A chatbot works in three steps: it reads the question, determines what the visitor means and finds an answer to match. A simple chatbot does that with fixed rules, an AI chatbot with a language model and the business’s own knowledge. Whether the answer is right depends less on the technology than on two things around it: the knowledge source the chatbot draws on and the moment it brings in a person.
Below you will read exactly what happens between question and answer, how a rule-based chatbot and an AI chatbot differ and what a chatbot needs to give good answers.
How do chatbots work in short?
A chatbot receives a question, recognises the intention behind it and gives an answer, or passes the conversation to a member of staff. That happens in a few seconds, in six steps:
- The visitor types a question in the chat window.
- The chatbot determines what the visitor wants: the intent.
- It looks up the answer in the business’s knowledge.
- It formulates an answer in plain language.
- Where needed it retrieves data or records something, such as an appointment.
- If it cannot answer the question well, it hands the conversation over to a person.
Unlike a voice assistant such as Siri, a chatbot works with typed messages, on a website or in a messaging app. What a chatbot is and which types exist is covered in what is a chatbot.
Chatbot architecture: the six building blocks from question to answer
The architecture of a chatbot is the set of components that together turn a question into an answer. How do chatbots work under the bonnet? Every chatbot on a website consists of the same six building blocks. The difference between a chatbot that helps and a chatbot that frustrates almost always sits in one of them.
Input. The question comes in through the chat window on the website, or via WhatsApp or another channel. The chatbot reads not only the last sentence, but also what was said earlier in the conversation.
Intent recognition. The chatbot determines what the visitor wants. “Can I come in on Saturday?” and “Are you open at the weekend?” look alike, but the first is an appointment question and the second a question about opening hours. Recognising this is called intent recognition, and it is the core of every chatbot.
Knowledge source. The answer comes from the business’s own information: product details, terms, delivery times and frequently asked questions. A chatbot is never smarter than this source.
Language model. With an AI chatbot, a language model formulates the answer in plain language, based on what is in the knowledge source. A rule-based chatbot skips this step and shows a pre-written text.
Integrations. Through integrations with a diary, CRM or stock system, the chatbot can do something: book an appointment, look up an order status or record an enquiry. Which systems can be connected is set out on the integrations page.
Handover. If the chatbot does not know the answer, or if the question calls for a person, it passes the conversation to a member of staff, with the full conversation attached.
How does a rule-based chatbot work?
A rule-based chatbot works with fixed rules: if it recognises a keyword or the visitor chooses a button, a pre-written answer follows. There is no language understanding behind it, only a decision tree the organisation has drawn up itself.
If a visitor types “opening hours”, the chatbot recognises that word and shows the opening hours. If the same visitor asks “How late can I still come in this evening?”, there is no known keyword and the chatbot gets stuck. The best-known form is a click bot, where the visitor only chooses from buttons.
A rule-based chatbot is predictable, quick to set up and never says anything you did not put in it yourself. On the other hand, it only works as long as visitors ask their question the way you expected, and every new question requires a manual change to the script.
How does an AI chatbot work?
An AI chatbot understands free questions, even if they are phrased differently from what was expected or contain typos. It uses natural language processing (NLP) to determine the meaning of a sentence and a language model to formulate the answer.
A visitor types: “I saw a black staircase with lighting on your site yesterday, is it available in oak too?” There is no fixed keyword in it, but an AI chatbot understands that it concerns a specific product and a question about finishes. It looks up the answer in the product information and, if needed, asks about the dimensions of the staircase.
That makes an AI chatbot far more widely usable than a rule-based chatbot. It does require more preparation: the knowledge source has to be in order and there must be clear boundaries for what the chatbot does and does not answer itself.
Rule-based or AI: the difference at a glance
The difference lies in how the chatbot understands a question and where the answer comes from.
Where does an AI chatbot get its answers from?
A good AI chatbot gets its answers from the business’s own information, not from whatever the language model happens to know. For every question it first looks up the relevant passages, for example the delivery time of a product or the conditions for part-exchange, and then formulates the answer based on exactly that information. This approach is called retrieval augmented generation, or RAG for short.
The difference is large. A language model without its own knowledge source gives a general or invented answer, and says so with the same confidence as a correct one. A chatbot with a good knowledge source names the actual delivery time, and says honestly that it does not know when the information is missing. At that moment it should bring in a person.
That is why the knowledge source is the most important building block. Anyone who takes an AI chatbot connected to your own systems seriously prevents most wrong answers before the chatbot goes live.
How does a chatbot learn?
A chatbot does not learn by itself. Someone has to read back the conversations, see where the chatbot got stuck or gave a wrong answer, and adjust the knowledge source or the instructions.
With a rule-based chatbot, learning means adding new rules and answers. With an AI chatbot it is mainly about adding to the knowledge source, rewriting unclear information and adjusting boundaries. How to go about that is covered in chatbot training.
The first weeks after go-live matter most. That is when the questions nobody foresaw come in, and they show what is still missing from the knowledge source.
When does a chatbot hand the conversation to a person?
A chatbot should hand over the conversation as soon as it is unsure, with a complaint, with a question that deviates from the standard, with a concrete buying question and when the visitor asks for a member of staff themselves. A good handover is announced and takes the whole conversation with it, so the visitor does not have to repeat anything.
Regulators are watching this too. For businesses serving customers in the EU, the European Commission explains that the AI Act requires people to be told from the start that they are interacting with an AI system. A chatbot that hides behind a human name undermines trust and, in the EU, breaks the rules.
How a chatbot and live chat with people relate to each other is covered in chatbot vs live chat.
How we set up a chatbot
At Bconnect we spend most of our time on the two building blocks where things go wrong in practice: the knowledge source and the handover.
We build the knowledge source together with the client, from the website, product information, terms and the questions that already come in. After go-live our chat specialists read along with the conversations and sharpen the answers. The handover depends on the model: with Operator Assisted AI our chat specialists hold the conversation and the AI supports them, with AI First + Human Handoff the AI holds the conversation and a specialist or the client’s own team takes over where needed.
An example from automotive. A visitor asks in the evening about the range of an electric car. The AI answers that from the dealer’s knowledge source. Then the visitor asks whether their current car can be part-exchanged and whether they can take a test drive on Saturday. A specialist then takes over, records the test drive and passes the request with the whole conversation to the right branch.
More examples per sector are in the chatbot examples. What a chatbot delivers for your business is in the benefits of chatbots. If you are unsure whether to build it yourself, read building a chatbot yourself or outsourcing it.
A chatbot is only as good as its knowledge and its boundaries
A chatbot reads a question, recognises the intention and gives an answer, with fixed rules or with a language model. For most businesses the technology is not the problem. The difference lies in the knowledge source the chatbot draws on and in the moment it brings in a person. Get those two right and you have a chatbot that really helps visitors. All the articles on this topic are in the overview of chatbots.
Frequently asked questions about how chatbots work
Can you explain in one sentence how a chatbot works?
A chatbot recognises what someone is asking and automatically gives an answer, based on fixed rules or with artificial intelligence and the business’s own knowledge.
How does an AI chatbot work?
An AI chatbot uses natural language processing to determine what a visitor means, looks up the answer in the business’s knowledge source and formulates it in plain language with a language model. If it does not know the answer, it should hand the conversation to a person.
What is the difference between a rule-based chatbot and an AI chatbot?
A rule-based chatbot responds to keywords and buttons with pre-written answers. An AI chatbot understands the meaning of free questions and formulates the answer itself, based on the business’s knowledge.
Does a chatbot learn from conversations?
Not by itself. Someone has to read back the conversations, correct wrong answers and add missing information to the knowledge source. After that the chatbot gives better answers to the same questions.
Can a chatbot give wrong answers?
Yes. An AI chatbot without a good knowledge source can give an answer that sounds convincing but is wrong. A well set-up chatbot only works with the business’s information and says honestly when it does not know something.
Does a chatbot work without the internet?
A chatbot on a website or in a messaging app needs an internet connection, because the question goes to a server where the chatbot determines the answer.
How can I build a chatbot myself?
You choose a platform, collect the questions customers ask with the right answers, build the knowledge source and test the chatbot with real questions. After that, someone has to keep reading the conversations and updating the chatbot.