Any developer can connect GPT-4 to a web form in an afternoon. Making that work within a bank, in a secure, regulated way, with customer context and accurate answers about their real account, is a completely different system architecture problem.

In this article, we’ll break down that end-to-end architecture: from the message the customer writes to the response returned to their screen, authentication, generative AI orchestration, core banking integration, and regulatory compliance. All with real services, available today.

A customer of the bank wants to know, at 11 p.m. from its mobile app: “How much did I spend in supermarkets this month and when does my card expire?”

That innocent question involves: identifying the user, extracting intent with NLP, retrieving data from two different systems, formulating a coherent response in natural language, recording the interaction, and doing it in less than three seconds. All without exposing sensitive data.

The chatbot doesn’t live only on the web. In a real bank, the same conversational engine serves through the web, mobile app, WhatsApp Business, Microsoft Teams (for companies), and even voice channels through intelligent IVR.

Azure Bot Service acts as the universal hub for channels. Its Direct Line component  normalizes the protocol of each channel to a single flow of messages that the backend processes identically. This means that the business logic is channel agnostic: you write once, you deploy it to all.

For WhatsApp, the integration goes through the WhatsApp Business Cloud (Meta) API, which connects to the Bot Service via a webhook. For Teams, Microsoft’s native connector eliminates that complexity. For speech IVR, Azure Speech Service does real-time speech-to-text, converts customer audio to text, processes it into the same conversational pipeline, and transforms the response back to speech using text-to-speech with neural voices.

Before any message reaches the application’s logic, it passes through three security filters that operate in parallel.

Azure Front Door with WAF (Web Application Firewall) is the global entry point. It absorbs DDoS attacks, applies OWASP rules, geo-filters if the bank operates only in certain countries, and terminates TLS before traffic enters the bank’s private network. The latency it introduces is minimal because Front Door operates from Microsoft’s 100+ points of presence around the world.

Azure API Management (APIM) is the API governance layer. Here, rate limiting is applied per user or IP to prevent abuse, JWT tokens are validated, headers are transformed, and versions of the chatbot API are managed. APIM acts as the gatekeeper that decides whether a request is allowed to exist before it touches any backend service.

Microsoft Entra ID (the new name for Azure Active Directory) handles identity. Clients authenticate using OAuth 2.0 with Conditional Access: If the client accesses from an unrecognized device, MFA is required. If you attempt sensitive operations such as querying card details, re-authentication may be required. Access tokens are short-lived and refresh tokens are managed with sliding expiration.

This is the most technically interesting layer, where the already authenticated message is converted into an intelligent conversation.

The initial NLP engine: Azure CLU (Conversational Language Understanding) analyzes the message and classifies the user’s intent. Want to know a balance? Report fraud? Check the status of a transfer? CLU also pulls entities: “grocery stores” as the spending category, “this month” as the date range, “my card” as a reference to a specific customer product. This step occurs before calling GPT-4o because it is faster, more deterministic, and allows routing decisions to be made without spending tokens.

Azure OpenAI with GPT-4o is the generative brain. This is where modern architecture differs radically from the rules chatbots of previous years. Instead of building decision trees for each possible question, a function calling system is used: the model receives the user’s message plus a catalog of available functions (consultar_saldo, obtener_movimientos, verificar_vigencia_tarjeta), and decides which ones to call and with which parameters. The backend executes those functions against the actual core banking and returns the results to the model, which synthesizes them into a natural, coherent, and personalized language response.

The  model’s system prompt includes: role instructions (“you are Bank X’s banking assistant”), security rules (“never reveal information about a customer other than the authenticated one”), topic limits, and the conversation context stored in Cosmos DB so that the customer can ask follow-up questions without repeating context.

Azure AI Search with RAG (Retrieval-Augmented Generation) complements the model with knowledge of the bank: fees, branch hours, product requirements, FAQs. The bank’s documents are indexed with vector embeddings. When the user asks something that doesn’t require transactional data but bank policy information, the system does a semantic search in the index and the results are injected as context into the prompt before calling GPT-4o. This eliminates hallucinations about specific bank policies and keeps responses up to date without the need to retrain the model.

Integration with legacy systems is where most banking chatbot projects fail or become incomplete. The right architecture uses Azure Logic Apps or Azure API Management as an adaptation layer between the modern chatbot world and the bank’s transactional systems, which can be mainframes, SAP, COBOL, or any decades-old stack.

Function calls from GPT-4o reach microservices in Azure Kubernetes Service or Azure Functions (serverless, simpler for medium-sized banks), which make the calls to the core banking, transform the data into the format the model expects, and handle errors in an elegant way: if the core banking does not respond within 2 seconds, the chatbot informs the customer and suggests alternatives instead of hanging up.

Azure Service Bus handles asynchronous flows: When the customer triggers an alert or notification, the event goes to a Service Bus queue that processes the action without blocking the conversation. It also handles escalation to a human agent: if the model’s confidence in the correct answer drops below a threshold, or if the customer expresses frustration, an event is published to Service Bus that transfers the conversation with all its context to a real agent in Dynamics 365.

Azure Key Vault with Managed Identity is the most critical piece of security in this layer. No service stores credentials in code or configuration. Each service has a managed identity that requests access to Key Vault only when it needs it, with least-privilege access. The certificates of connection to the banking core rotate automatically. This is a de facto requirement for PCI-DSS compliance, which applies to any system that touches payment card data.

The chatbot needs several types of storage with different characteristics.

Azure SQL Database stores structured transactional data when the bank doesn’t want to directly query the core for certain read-only use cases, with read replicas so as not to impact production systems.

Azure Blob Storage stores the documents that feed the RAG: contract PDFs, product manuals, regulatory circulars. A recurring indexing pipeline (Azure Functions + AI Search Indexer) processes new documents automatically.

Cosmos DB is the conversational session store, where each user’s conversation history is saved with configurable TTL (time-to-live), multi-turn conversation status, and user preferences. Its per-userId partitioning model ensures sub-10ms latencies on reads, which is essential so that context recovery is not the bottleneck.

The Azure AI Search index  is the vector store where the embeddings of all the bank’s documents live. When a question arrives, the embedding of the question is generated with the same model (text-embedding-ada-002 or text-embedding-3-small) and cosine similarity search is done to retrieve the most relevant fragments. This is RAG in its most practical form.

In a bank, what is not audited does not exist. This layer is not optional.

Application Insights captures every conversation, latencies per step, error rates, and usage metrics. Azure Monitordashboards  allow you to see in real time how many conversations are active, how many have escalated to human agent, and which intents are the most frequent. Alerts automatically fire if the chatbot’s latency exceeds thresholds or if the rate of low-confidence responses increases.

Microsoft Defender for Cloud applies a continuous security posture over all Azure services used: it detects misconfigurations, monitors for anomalous behavior, and acts as a SIEM when integrated with Microsoft Sentinel. In banking context, a relevant alert could be an unusual volume of inquiries about a specific account, which could indicate automated scraping.

Azure Purview catalogs all the data processed by the chatbot, automatically classifies sensitive information (account numbers, personal data, financial information), and provides the lineage of data that banking regulators can audit. Together with Log Analytics, where immutable logs of all interactions are stored, this covers the traceability requirements of regulations such as GDPR, the local Fintech Law, or regulatory frameworks for banking supervision.

To be completely clear, this is the route of that 11 p.m. message:

The customer writes from their mobile app. The message travels over HTTPS to Azure Front Door, which validates the certificate and enforces WAF rules. It goes to APIM, which validates the client’s JWT against Entra ID and applies rate limiting. The Bot Service receives the normalized message.

Azure CLU classifies the intent (expense query and card validity) and extracts the relevant entities. The conversation history is retrieved from Cosmos DB. The prompt is constructed with the session context and the bank’s system prompt. A semantic search is done in AI Search to see if there are any relevant policies on expense categorization.

GPT-4o receives the full prompt and responds indicating that it needs to call two functions: obtener_movimientos(category=”supermarkets”, period=”mes_actual”) and verificar_vigencia_tarjeta(product=”tarjeta_principal”)). Azure Functions executes those calls against the core banking through Logic Apps, which does the protocol translation. The actual customer data is returned as a function result.

GPT-4o synthesizes the answer in natural language, coherent, and in the bank’s brand tone. The response travels back through the same stack, is recorded in Application Insights, the history is updated in Cosmos DB, and arrives on the customer’s screen.

Total time: between 1.8 and 3.5 seconds, depending on the latency of the legacy banking core. Customer perception: You spoke to someone who knew your information and responded in human language.

Some third-party services are not Microsoft but fit perfectly into this architecture:

Twilio or Infobip for confirmation SMS and push notifications when the chatbot triggers a trade. Segment as a Customer Data Platform to enrich the user’s context with their behavioral history. Dynatrace or Datadog as an additional observability layer if the bank already uses them. Okta instead of Entra ID if the bank prefers a separate Microsoft IdP. Everything is connected via standard APIs without breaking the base architecture.

Each component described exists as a productive and documented service. There is no pending applied research, there are no experimental integrations. Azure OpenAI has been in general availability for more than two years. Azure AI Search with vectors has more than one. Azure Bot Service has more than five years of maturity in banking production. The RAG pattern is standardized and documented by Microsoft with solution templates available on GitHub.

What separates the banks that already have this working from those that don’t, is not technology: it is a decision of architecture and internal change management. Architecture is here. The question is how much it costs the bank not to have it.

What happens when the customer types “I want to talk to a person” (or the system detects frustration, complexity, or low confidence in the response) has three distinct moments: detection, routing, and context transfer.

There are two types of triggers and both are already covered by existing components in the architecture:

Explicit triggers — the customer literally says “I want to talk to an executive,” “I need someone,” “this doesn’t work for me.” CLU (Azure Conversational Language Understanding) already has escalation intent trained from the base design. No additional logic is needed.

Implicit triggers — here’s what’s interesting. GPT-4o can detect signs of frustration in the tone of the message, and the system prompt can include an explicit instruction: if confidence in the response is low, or if the user has been trying to resolve the same issue for more than N turns, return a special function escalar_a_agente() instead of continuing to try. Application Insights, which is already in the architecture, can track the number of turns and sentiment score per conversation.

Here come the components that are already in the architecture and one that is new.

Azure Service Bus (already exists) publishes a ConversationEscalation event  with the full payload: userId, conversation history, original customer intent, and session metadata. That event is consumed by the contact center system.

Dynamics 365 Customer Service / Contact Center (mentioned in the article as a component of human escalation, but it is worth developing it) is the CRM used by the executive. It has a queue of incoming conversations, agent availability, and skills-based routing rules. If the customer asked about loans, the event includes that category, and Dynamics 365 automatically routes to a credit executive, not an investment executive.

What is new is  a new component that was not explicit in the base architecture: Azure Communication Services (ACS).This service is the one that enables real-time communication between the customer and the executive, whether it is live chat, voice over IP call, or video. ACS is the “tube” of communication. Dynamics 365 Contact Center consumes it internally, but if the bank wants to build its own chat experience in the app, it can call ACS directly from the customer interface.

This is the step where most banking chatbots fail. The customer has been explaining their problem to the bot for five minutes, is transferred to an executive, and has to start from scratch. That cannot happen.

The architecture solves it like this: at the time of the handoff, the event traveling through the Service Bus includes the full history of Cosmos DB — not the raw transcript, but a summary generated by GPT-4o at the time of escalation. Something like: “Customer consults discrepancy in supermarket charge of 07/12. You have an active Visa Platinum card. It has been 3 turns without satisfactory resolution.” The executive sees that summary in the Dynamics 365 dashboard before writing their first message.

The customer in their app sees a transition screen: “We are connecting you with an executive. Estimated time: 4 minutes.” That estimate comes from the Dynamics 365 Availability API. The previous conversation is visible in the same chat thread.

Component	Status in Base Architecture
Azure Service Bus (escalation event)	Already exists, only the event type is added
Dynamics 365 Contact Center	Mentioned but not detailed — requires D365 licensing
Azure Communication Services	New if you want native chat/voice in the bank app
Cosmos DB (context to transfer)	Already exists
GPT-4o to generate handoff summary	It already exists, just that specific prompt is added
Application Insights (escalation tracking)	Already exists

In practical terms, if your bank already has Dynamics 365, the cost of enabling this flow is low: it’s configuring connectors and event logic in Service Bus. If the bank uses a third-party contact center (Genesys, Avaya, Salesforce Service Cloud), the pattern is identical — Service Bus publishes the event and the provider’s connector consumes it. The architecture is contact center agnostic.

Once the executive resolves the issue and closes the conversation, the bank may choose to return the customer to the bot for a satisfaction survey or to continue with other queries. ACS and Dynamics 365 can publish a close event that is consumed by Bot Service to retake the channel. The loop is closed and the entire flow is tracked in Application Insights from end to end — including the time it took the executive to resolve what the bot couldn’t.

Decision flow (when and how escalation is detected)

This diagram shows what happens once you decide to scale — the components that execute the handoff, context transfer, and cycle closure.

The two diagrams together tell the full story. The first shows the decision logic — how the system knows it should escalate, differentiating the explicit path (the client asks for it) from the implicit path (the system detects it by metrics). The second shows the mechanics of the handoff: GPT-4o generates the summary, Cosmos DB delivers the history, Service Bus fires the event in parallel to Dynamics 365 (which finds the right executive), and Azure Communication Services (which opens the actual communication channel). The executive receives all the context before writing their first message, and when they close the conversation, the closing event returns to the bot to complete the cycle with survey and registration in Application Insights.

Azure API Management (APIM)

Microsoft service that acts as a centralized gateway for all APIs in an organization. In the context of the banking chatbot, APIM receives every request that comes from the input channels and applies a set of policies before letting it through: it validates that the client’s JWT token is legitimate, applies rate limiting to prevent a malicious actor from saturating the system with thousands of requests per second, transforms HTTP headers when backend systems expect them in a different format,  and records every call for auditing. APIM also allows versioning of APIs, which means that the bank can launch a new version of the chatbot without breaking existing integrations. It is the gatekeeper that decides whether a request has the right to exist before it touches any internal service.

Azure Bot Service

Microsoft platform specifically designed to build, deploy, and manage chatbots at scale. What makes Azure Bot Service valuable in a banking architecture isn’t the AI itself—that comes from OpenAI—but the ability to standardize the communication protocol from multiple channels to a single flow of messages. When the customer writes on WhatsApp, the message arrives with Meta’s own format and metadata; when you type through Teams, you arrive in the Microsoft format; when it writes through the bank’s app, it arrives as a REST call. Azure Bot Service, through its Direct Line component and the channel SDK, translates all these formats into a unified message that the backend processes identically. This allows the chatbot’s business logic to be written only once and work across all channels without modification.

Azure OpenAI Service

Enterprise deployment of OpenAI models—including GPT-4o and the embedding model family—deployed directly within Azure infrastructure, subject to Microsoft security and compliance controls. The critical difference from using OpenAI’s public API is that the processed data does not leave the contracted Azure region and is not used to train future models, a non-negotiable requirement in data-regulated banking environments. In chatbot architecture, GPT-4o is used for two distinct purposes: as a natural language generation engine that synthesizes coherent responses from real customer data, and as a tool orchestrator using function calling, a capability that allows the model to decide which backend functions to call and with what parameters, without the developer having to program that routing logic manually.

Azure AI Search

Cognitive search service that combines traditional keyword search with vector-based semantic search. In the banking chatbot, it fulfills the role of an institutional knowledge base: the bank’s documents—rates, product policies, FAQs, regulatory circulars—are indexed in Azure AI Search. When a customer asks a question about policies or conditions, the system generates a vector representation of that question and compares it against the vectors of all indexed documents, retrieving the most semantically relevant fragments. Those fragments are injected into the GPT-4o prompt as additional context before generating the response, a pattern known as RAG (Retrieval-Augmented Generation). The result is that the model responds with real, up-to-date information from the bench rather than making up answers or being limited to what it learned during its training.

Azure Cognitive Language Understanding (CLU)

Microsoft’s natural language processing service that allows you to train models to classify intents and extract free language text entities. In the chatbot pipeline, CLU operates before GPT-4o as a fast, deterministic filter: when a message arrives from the customer, CLU analyzes it in milliseconds and determines whether the intent is to check balance, report fraud, ask about products, or escalate to a human executive. It also extracts entities: if the customer writes “how much did I spend in supermarkets this month?”, CLU identifies “supermarkets” as the merchant category and “this month” as the time range. Using CLU before GPT-4o reduces token costs, improves speed of response, and allows routing decisions—such as human scaling—to be made without consuming the generative model for tasks that don’t need it.

Azure Cosmos DB

Microsoft’s globally distributed NoSQL database, designed for sub-10 millisecond latencies in reads and writes at any scale. In the banking chatbot, it fulfills the role of conversational state store: each active customer conversation – the message history, the context accumulated during the session, the preferences detected – is stored in Cosmos DB partitioned by the user ID. When a new message arrives, the system retrieves the history of that partition in microseconds and includes it in the context sent to GPT-4o, allowing multi-turn conversations where the client can ask follow-up questions without repeating information. Documents have configurable TTL (time-to-live) so that inactive sessions expire automatically, complying with data retention policies.

Azure Service Bus

Queue and topic-based enterprise messaging service that decouples components of an architecture asynchronously and reliably. In chatbot architecture, Service Bus is the nervous system for events that don’t need an immediate response: when the customer triggers an alert, when the system decides to escalate the conversation to a human agent, or when an operation in core banking takes longer than expected, a message is published to Service Bus. That message is durably glued up—it survives system reboots—until the corresponding consumer processes it. The most critical component in the human escalation flow is the EscalationConversation event, which brings the entire session payload into Dynamics 365 Contact Center. Service Bus ensures that that event is not lost even if the target system is momentarily down.

Azure Key Vault

Microsoft cryptographic keys, certificates, and secrets, management service. In any banking system, the problem of credentials is one of the most frequent attack vectors: connections to databases, communication certificates with the core banking, API keys from external services. The unsafe practice is to store those values in environment variables or configuration files. Azure Key Vault eliminates that problem: services in the architecture never see credentials directly; instead, each service has a Managed Identity—an identity automatically managed by Azure—that requests access to Key Vault at runtime with a short-lived token. Certificates rotate automatically before expiring. The accesses are audited. This covers a core PCI-DSS requirement, which mandates strict controls over access to payment method data.

Microsoft Enter ID

Microsoft’s identity and access management system, formerly known as Azure Active Directory. In the banking chatbot, Entra ID is the source of truth about who the customer is who is sending messages. When a customer authenticates to the bank’s app, Entra ID issues a JWT token with a short shelf life that travels on each subsequent request. APIM validates that token before letting the message through. Entra ID also handles conditional access: if the customer tries to access from an unregistered device, or from an unusual geographic location, they can require a second factor of authentication (MFA) before proceeding. For sensitive operations within the chatbot – such as querying the full number of a card – a policy can be configured that requires re-authentication even if the session is already active.

Azure Front Door with WAF

Azure Front Door is Microsoft’s global traffic distribution layer, operating from more than 100 points of presence around the world to route user requests to the nearest and most available backend. The built-in WAF (Web Application Firewall) is the perimeter security component: it applies OWASP-based rulesets to block common attacks—SQL injection, cross-site scripting, malicious bots—before traffic enters the bank’s infrastructure. In the banking context, Front Door also handles TLS termination, ensuring that all communication between the customer and the system travels encrypted, and allows geo-filtering to be configured to reject traffic from countries where the bank does not operate, reducing the attack surface.

Azure Application Insights

Microsoft’s Application Performance Monitoring (APM) service, part of the Azure Monitor suite. In the chatbot architecture, Application Insights automatically captures the trace of each conversation: how long each step in the pipeline took, what function GPT-4o called, what the core banking latency was, how many turns the conversation had before being resolved or scaled. Those distributed traces allow the engineering team to identify bottlenecks with surgical precision: if responses are taking longer than normal, Application Insights can pinpoint exactly whether the problem is in the model, in the database, or in the core bank. It also feeds proactive alerts: if the rate of low-confidence responses increases by 20% in the last 15 minutes, the team gets notified before customers start complaining.

Microsoft Defender for Cloud

Microsoft’s cloud security platform that combines three functions into a single service: security posture management (CSPM), cloud workload protection, and SIEM capabilities when integrated with Microsoft Sentinel. In the banking chatbot, Defender for Cloud continuously monitors the configuration of all the Azure services involved—if a Cosmos DB is mistakenly publicly exposed, if an Azure function doesn’t have authentication, if a container has an image with known vulnerabilities—and generates risk-prioritized recommendations. Threat detection analyzes behavior patterns: An abnormally high volume of queries about the same account in a short period of time can indicate a data scraping attack, and Defender detects it before the human team notices it.

Azure Purview

Microsoft Data Governance Service that catalogs, classifies, and traces the lineage of data across all systems in an organization. In a bank, where customer data flows between the chatbot, core banking, session databases, and audit logs, knowing exactly where each sensitive piece of data lives and how it moves is a regulatory requirement, not an option. Azure Purview automatically scans connected data sources, identifies personal information, account numbers, card data, and classifies them with sensitivity labels. When a banking regulator requests an audit on a specific customer’s data processing, Purview can answer that question with the full lineage: which systems touched that data, at what time, and for what purpose.

Azure Communication Services (ACS)

Microsoft’s real-time communication platform that provides infrastructure for chat, voice over IP, video, and SMS, available through APIs that developers can integrate directly into their applications. In the human escalation flow of the banking chatbot, ACS is the component that opens the direct communication channel between the customer and the executive. When Dynamics 365 Contact Center assigns an available agent, ACS establishes the real-time chat session visible to both — the executive on their Dynamics dashboard and the customer in the bank’s app — without the customer having to switch screens or download any additional apps. ACS also supports IP voice and video calls for cases where the complexity of the problem requires richer communication than text.

Dynamics 365 Contact Center

Microsoft module within the Dynamics 365 ecosystem aimed at managing customer interactions in multichannel service environments. In the banking chatbot architecture, Dynamics 365 Contact Center is the system that receives the escalation event published by Service Bus and converts it into a work task for a human agent. Its skills-based routing capability is what ensures the customer reaches the right executive: if the escalated conversation was related to a loan inquiry, the system identifies that category in the event payload and assigns the conversation to a credit agent, not an investment or general service agent. The executive sees in their dashboard the summary generated by GPT-4o, the complete history of the conversation with the bot, and the customer data, all available before writing the first message.

RAG (Retrieval-Augmented Generation)

Architecture pattern for generative AI systems that combines the language generation capability of a model such as GPT-4o with the retrieval of updated information from an external knowledge base. The problem it solves is fundamental: a language model has frozen knowledge at the time of its training and knows nothing about a bank’s specific policies, its current rates, or its current products. RAG solves this by intercepting each user’s question, searching a vector database for the document fragments most relevant to that question, and injecting them into the model’s prompt as context before asking for the answer. The model then generates a response based on real and updated information, without the need to be re-trained every time the bank changes a rate or launches a product. In the banking chatbot, Azure AI Search is the retrieval engine, and vector embeddings are the mechanism that makes semantic search possible.

Vector embeddings

Mathematical representations of text in the form of high-dimensional numerical vectors—typically between 768 and 1,536 dimensions—generated by specialized language models. The property that makes them useful is that texts with similar meanings produce vectors that are mathematically close to each other, regardless of the exact words used. In the banking chatbot, each document snippet from the bank—a paragraph of the fee policy, a section of the card agreement—is converted into a vector and stored in the Azure AI Search index. When a customer’s question arrives, that question is also converted into a vector with the same model, and the system searches for the closest document vectors geometrically. That mathematical distance is equivalent to semantic similarity: “how much do you charge me to withdraw abroad?” you find the paragraphs on international commissions even if they do not share any exact word with the question.

Function calling

OpenAI’s GPT-4 and GPT-4o models are capable of generating, instead of free text, a structured instruction to call an external function with specific parameters. In the banking chatbot, this is the central mechanism of integration with the core banking: instead of programming a decision tree that identifies the intent and calls the corresponding function, the developer gives the model a catalog of available functions —consultar_saldo, obtener_movimientos, verificar_vigencia_tarjeta— with their parameters and descriptions in natural language. The model reads the client’s message, decides which functions are needed and with which arguments, and returns that structured instruction instead of a text response. The backend executes the functions against the actual systems, returns the results to the model, and the model synthesizes a natural language response. The developer writes less routing logic, and the system naturally adapts to questions that combine multiple needs.

Azure Logic Apps

Microsoft Workflow Integration service that allows you to connect heterogeneous systems using pre-built connectors and visual logic without the need for extensive code. In the banking chatbot, Logic Apps acts as an adaptation layer between the chatbot’s modern world of microservices and REST APIs and the bank’s legacy systems—mainframes, SAP, SOAP services, COBOL systems—that expose decades-old interfaces. When GPT-4o decides to call the consultar_saldo function, that call arrives at a modern microservice that invokes a Logic App. The Logic App does the translation: it converts the REST call into the format that the core banking understands, handles authentication specific to that system, handles retries in the event of a transient failure, and transforms the response back to the JSON format that the microservice expects. This decouples the chatbot from the technical quirks of legacy banking systems.

JWT (JSON Web Token)

Open standard for transmitting information between systems in a compact and cryptographically verifiable way. A JWT is a string of text divided into three parts: the header with the signature algorithm, the payload with the user’s data and permissions, and the digital signature that guarantees that the token was not altered. In the banking chatbot, when the customer authenticates to the app, Entra ID issues a JWT with the user’s ID, roles, and a short expiration time. That token travels in the header of each request. APIM verifies the token’s signature against the Entra ID public key—without the need to query any database—and rejects any request with an invalid, expired, or signed token with a different key. This ensures that no message can reach the chatbot without a verified identity.

PCI-DSS

Payment Card Industry Data Security Standard: A set of security standards defined by the major payment card networks—Visa, Mastercard, Amex, Discover—that apply to any organization that processes, stores, or transmits credit or debit card data. In the context of the banking chatbot, PCI-DSS defines specific requirements that directly affect the architecture: cardholder data must be encrypted at rest and in transit, access to that data must be limited to the minimum necessary and audited, credentials connecting to systems that process payments cannot be embedded in code,  and access logs must be retained for a defined period. Azure Key Vault with Managed Identity, Azure Purview for sensitive data classification, and Log Analytics for audit log retention are the architecture components that directly cover these requirements.

OAuth 2.0

A standard authorization protocol that allows an application to gain limited access to resources from another application on behalf of a user, without the user having to share their credentials. In the banking chatbot, OAuth 2.0 is the protocol underlying the authentication flow with Entra ID: when the customer logs in to the bank’s app, the app redirects to Entra ID, the user authenticates, and Entra ID returns an access token with a specific scope—for example, permission to read transactions but not to initiate transfers. That scope is hardcoded into the token and APIM can inspect it to allow or deny specific operations within the chatbot without consulting any additional systems. The protocol also defines how access tokens are renewed when they expire, without requiring the user to re-enter their password.

MFA (Multi-Factor Authentication)

An authentication mechanism that requires the user to prove their identity using two or more independent factors: something they know (password), something they have (registered device, authenticator app), or something they are (biometrics). In the banking chatbot, MFA isn’t applied to every interaction—that would make the experience unbearable—but is selectively activated by Entra ID using conditional access policies. Factors that can trigger MFA include: the customer is accessing from an unrecognized device, the geographic location is unusual from the user’s historical pattern, or the operation requested within the chatbot exceeds a risk threshold. When MFA is activated, the bank’s app displays a challenge—an SMS code, a notification in the authenticator app—and the chatbot is paused until the user completes the additional factor.