Getting Started with Chatbots and NLP: Part Two

Chatbots Part 2

Editor’s note: This is the second post in a series that explores the challenges and opportunities involved with chatbots and natural language processing to solve customer service needs. You can read part one here.

In my first post, I described some of the practical problems faced by companies interested in adopting chatbots. With that background, I described the requirements of a fully featured chatbot that could address those challenges using a combination of natural language processing and human escalation. I then described a high-level architecture and some tools that could be used to implement that architecture. In this blog post, I pick up where the last post left off by describing an implementation we can use to begin building a chatbot.

Prerequisites

There are a couple of steps necessary for this code to work locally. If you want to run the code on your own machine, you’ll need to:

  1. Register a bot with the Microsoft Bot Framework website and generate an application ID and password.
  2. Set up a Redis Labs Cloud instance or run your own local redis instance.
  3. Have node.js (version >= 7.3.0) and npm installed locally.
  4. Complete the setup steps outlined in the README found in the Github repository.

Implementation

In order to exchange messages between the customer via a chatbot and a human agent using a web UI, we have to have a messaging layer that works between the browser and the chatbot server. While there are many options for this, we will use the socket.io library. A full explanation of socket.io is beyond the scope of this post, so we will focus on the key aspects to our implementation. Socket.io provides the concept of “rooms”, which scope the communication over a socket. Think of this as a chat room. When a socket initially connects, it is joined to a default channel with all other connections. This will be where a group of human agents will be connected and waiting for a customer request.

When a customer sends a message to our chatbot (1), we will notify all the agents in the default room that a new conversation is starting (2). One of our agents will click a link that signals to the chatbot server that they intend to assist the customer (3), causing their socket to be joined to a room specifically for a conversation with that customer.

Once the agent has been joined to the room, the message from the customer is sent to the agent (4). At this point, the agent can respond, which sends a message back to the room (5). The chatbot server then broadcasts that message back to the customer (6). This style of communication can continue until the interaction is complete.

All of the source code for this example is available here. A complete explanation of node.js applications and all of the libraries and modules used in the application is beyond the scope of this article. Instead, I’ll focus on details specific to the functionality we are building by walking through file by file.

.env

appId=
appPassword=
redisPassword=
redisUrl=:
redisHost=
redisPort=

This file configures the Microsoft bot ID/password and redis connection details used by the rest of the application.

app.js

var env = require('node-env-file');
var app = require('express')();
var http = require('http').Server(app);
var io = require('socket.io')(http);
var redis = require('redis').createClient;
var adapter = require('socket.io-redis');

env('.env');

var pub = redis(process.env.redisPort, process.env.redisHost, { auth_pass: process.env.redisPassword });
var sub = redis(process.env.redisPort, process.env.redisHost, { auth_pass: process.env.redisPassword });
io.adapter(adapter({ pubClient: pub, subClient: sub }));

// start bot server with IO for socket notifications
var botServer = require('./lib/botServer')(io, http);

// start socket listener for chat agents
var socketManager = require('./lib/socketManager')(io, http);

app.get('/', function(req, res){
  res.sendFile(__dirname + '/index.html');
});
app.get('/join_chat.html', function(req, res){
  res.sendFile(__dirname + '/join_chat.html');
});

This is the code that runs to set up the socket.io server and the bot server. It also provides the routing information to server two HTML files that are used to provide the agent UI that we will discuss in more detail in a moment.

socketManager.js

module.exports = function(io, http) {
  const channelSpecificMessageTypes = ['agent joined', 'channel message', 'customer message', 'agent message'];

  io.on('connection', function(socket){
    console.log('a user connected');
    socket.on('chat message', function(msg){
      console.log('socketMgr: chat message: ' + msg);
      io.emit('chat message', msg);
    });

    channelSpecificMessageTypes.forEach(function(msgType) {
      socket.on(msgType, function(msg){
        console.log('socketMgr: got msgType = ' + msgType + 'in room: ' + socket.room);
        if (socket.room !== undefined) {
          io.sockets.in(socket.room).emit(msgType, msg);
        }
      });
    });

    socket.on('join channel', function(msg){
      console.log("joining channel " + msg);
      socket.room = msg;
      socket.join(msg);
    });

    socket.on('leave channel', function(msg){
      socket.leave(msg);
      socket.room = null;
    });

    socket.on('disconnect', function(){
      console.log('user disconnected');
      socket.room = null;
    });
  });

  http.listen(3000, function(){
    console.log('listening on *:3000');
  });
}

Here, we implement much of the logic needed to manage our web socket communication that acts as the glue between the bot and the web agent interfaces. We create a basic chat message protocol that consists of a couple message types. Messages have a type (e.g. customer message, agent message) and may also have data that accompanies the message. Some messages are room/channel specific, while others are more generic or control the behavior of the client/server interactions.

Non-Room Specific Message Types

  • Chat message: used to communicate general messages to a group of agents in the default room
  • Join channel: signal from the client to the server that it is requesting to join a specific room
  • Leave channel: signal from the client to the server that it should be removed from a specific room

Room Specific Message Types

  • Agent joined: sent by the agent web UI to signal to the bot that an agent is present to assist with an interaction in a specific conversation with the bot
  • Channel message: used to communicate general messages to the agent and customer in a room
  • Customer message: used to carry messages from the customer via the bot
  • Agent message: used to carry messages from the agent via the web UI

botServer.js

var restify = require('restify');
var builder = require('botbuilder');
var env = require('node-env-file');
var client = require("socket.io-client");


function findRooms(io) {
  var availableRooms = [];
  var rooms = io.sockets.adapter.rooms;
  if (rooms) {
    for (var room in rooms) {
      if (!rooms[room].hasOwnProperty(room)) {
          availableRooms.push(room);
      }
    }
  }
  return availableRooms;
}

//=========================================================
// Bot Setup
//=========================================================

module.exports = function(io, http) {
  // Setup Restify Server
  var server = restify.createServer();
  server.listen(process.env.port || process.env.PORT || 3978, function () {
     console.log('%s listening to %s for bot requests', server.name, server.url);
  });

  // Create chat bot
  var connector = new builder.ChatConnector({
      appId: process.env.appId,
      appPassword: process.env.appPassword
  });
  var bot = new builder.UniversalBot(connector);
  server.post('/api/messages', connector.listen());

  //=========================================================
  // Bots Dialogs
  //=========================================================

  bot.dialog('/', new builder.SimpleDialog(function (session, results) {
    if (findRooms(io).indexOf(session.message.address.conversation.id) > -1) {
      console.log("found existing room for conversation");
      // create a socket connection for this dialog`
      var socket = client.connect("http://localhost:3000/");
      // join a channel that scopes the discussion between the bot and the
      // agent using the session conversation ID
      socket.emit("join channel", session.message.address.conversation.id);
      // once the agent joins, send the message we received from the
      // customer via the bot
      socket.emit("customer message", session.message.text);

      // TODO: AI/ML processing goes here

    } else {
      var socket = client.connect("http://localhost:3000/");
      socket.emit("chat message", "A new customer/bot discussion has been initiated. " +
        "Click here to join.");

      // join a channel that scopes the discussion between the bot and the
      // agent using the session conversation ID
      socket.emit("join channel", session.message.address.conversation.id);

      // setup a callback that will be invoked once an agent joins
      // the discussion
      var agentJoinedCallback = function() {
        session.channelInitiated = true;
        // once the agent joins, send the message we received from the
        // customer via the bot
        socket.emit("customer message", session.message.text);

        // TODO: AI/ML processing goes here

        // setup a callback to be invoked once the agent responds so that
        // we can reply to the customer using the text
        var respondCallback = function(msg) {
          console.log("botChatStart dialog: call back invoked with msg:" + msg);
          session.send(msg);
        };

        // setup an event handler so we respond to the customer when
        // the agent responds
        socket.on("agent message", function(msg) {
          console.log("botChatStart dialog: got channel message: " + msg);
          // here we invoke the response callback
          respondCallback(msg);
        });
      };

      // setup an event handler so we know once the agent joins
      socket.on("agent joined", function(msg) {
        // invoke our callback from above
        agentJoinedCallback();
      });
    }

  }));

};

The Microsoft Bot Framework SDK is built around a concept of a dialog, which represents an interaction between the customer and the bot. Dialogs follow a routing pattern similar to websites or MVC frameworks; the default dialog is represented as “/” and is the entry point for all interactions originating from the bot. This dialog is where all of the logic that deals with exchanging messages between the bot and the web UI occurs.

Dialogs maintain their state via a session. Embedded within the session object is the message that is being passed into the dialog, and each message carries a conversation ID that is persisted across the duration of the conversation with the bot. This conversation ID is accessed through the session object passed into the dialog as session.message.address.conversation.id. We use the uniqueness and persistence of the conversation ID to name a room where the agent using the web UI and the customer interacting with the bot can exchange messages.

When a conversation is initiated by the customer, the “/” dialog is invoked. We first check to see if a room with the conversation ID already exists. When a new conversation is started, no room will exist, so we establish a connection from the bot server to the socket.io endpoint. We then broadcast a message using the chat message type to all the agents notifying all agents that a new conversation is starting with a customer. An agent can then click the link in the message to enter a dedicated room to converse with the customer.

When the agent joins the channel, the web UI sends the agent joined message. This invokes a callback that informs the bot server that communication with the agent can begin. At this point, we send the message from the customer to the channel so the agent can see what initiated the conversation. We then create a callback that is invoked whenever a message from the agent is received over the socket that sends the message back through the bot to the customer.

After the first message from the customer, every subsequent message received from the customer can be sent directly to the existing room. Therefore, if a room exists for a given conversation ID, the bot can simply join the room and broadcast the message without waiting for an agent.

index.html



  
    Socket.IO chat
    
  
  
    

    This is the entry point for the agent UI. JavaScript on this page establishes a socket.io connection to the server and receives the general chat message notifications from the chatbot. Embedded in the message body is a link to another page, join_chat.html, that includes a URI query parameter (“id”) that contains the conversation ID of the new conversation.

    join_chat.html

    
    
      
        Socket.IO chat
        
      
      
        

      When this page loads, it establishes a socket.io connection to the server, joins the channel name specified in the “id” query parameter, and sends an agent joined message. Any messages typed by the agent are then sent to the socket.io server with a type of agent message. The bot server joined to that channel receives the agent message and in turn sends it to the customer.

      Next Steps

      This implementation is very simple right now, but it creates a foundation we can build upon to integrate machine learning into the bot. In our next blog post, we’ll introduce natural language processing (NLP) that will be used to determine the intent of the customer’s request. We’ll also train our NLP model to extract certain elements from the message that we can use to find the correct answer for the customer’s question.

      Chris Hart

      Chris Hart

      CTO

      Chris Hart is a co-founder and CTO of Levvel. He has more than 15 years of technology leadership experience and has led software development, infrastructure, and QA organizations at multiple Fortune 100 companies. In addition to his enterprise experience, Chris has helped start or grow multiple early-stage technology companies. In the five years before starting Levvel, Chris was focused on financial technology solutions in the consumer, commercial and wealth management space. His technical expertise and enterprise-scale, global program management background helps Levvel’s clients transform their businesses.