Amazon Connect Data Sources - Part 1 ~ VoiceFoundry

In this Blog post, we’re going to take a look at the kinds of Data generated by Amazon Connect. This will be the first of a 3 part series that takes a deep dive into this important element – Data. In Part 1 we’ll look at the 7 different types of Data generated by Amazon Connect and how you can assist in its creation. Part 2 will show how to get access to this Data in real-time via event-driven processes as well as how to use that data in real-time to trigger other processes. In Part 3 we’ll look at options on how to store this Data for analytics and reporting.

Why this is so important

Data within the contact center is a rich source of information. It can provide you with insight into why customers are calling, the performance of your IVR, repeat callers, customer sentiment and troubleshooting, to name just a few. Knowing what information can be generated by Amazon Connect is critical in helping you understand the architecture you require and how to design your contact flows and processes. So let’s get right into it.

Type of data generated by Amazon Connect

We’ll start off at 10,000 ft. The first item you’ll likely notice is that there are more data sources than you first considered – 7 in fact! We’ll go into each of these in detail shortly. The other striking point to note is the varied use of other AWS Services. Amazon Connect is a building block. Sure, you can use it all on its own, but when used with the other AWS Services, it becomes so much more. The final takeaway you will spot is the real-time capabilities of the platform…50% of the data sources are streamed in real-time, the other 50% within a minute or 2.

Here we’ve expanded on the overview and added some more depth to the data that’s available.

NAME	PURPOSE	ACCESS VIA	AVAILABILITY
Contact Trace Records	Information about the contacts made/received within Amazon Connect	Kinesis Streams or Kinesis Firehose	60 seconds after creation
Contact Flow Logs	Detailed logging for the contacts as they navigate the contact flows	CloudWatch Logs	Real-time
Agent Event Streams	Events for Agent State Changes	Kinesis Streams	Real-time
Call Recordings	The audio recording of the conversation between the Agent and the Customer	S3	1-3 Minutes after call ending
Live Media Streaming	Customer side audio within the IVR/ LEX and talking with an Agent	Kinesis Video	Real-time
Scheduled Reports	Contact center reporting	S3	Few minutes after scheduled to run
CloudWatch Metrics	Instance level metrics, calls in queue, packet loss etc..	CloudWatch Metrics	Every 60 seconds

Let’s take a closer look at each of these in turn…

Now you have a solid understanding of the data sources including what they provide and when. We’ll go into more detail of each. I must admit, I was considering a separate blog post on each of the 7 data sources as there is more than enough to discuss on each. Instead, I’m whetting your appetite and we can discuss further over a coffee if you wish.

1. Contact Trace Records (CTR)

These are generated by Amazon Connect for each contact within your Instance. For example, an outbound call will generate a CTR as well as an inbound call. CTRs are likely to be the items that you will first consume and use within your Instance. CTRs are generated as a JSON Object that consists of Key Value Pairs (KVP). An example CTR can be seen below.



{

"AWSAccountId": "111122223333",

"AWSContactTraceRecordFormatVersion": "2017-03-10",

"Agent": {

"ARN": "arn:aws:connect:us-west-2:111122223333:instance/8ca95e2f-891b-4e2f-9353-35d1e893233e/agent/6ce798d2-7fa8-4578-827a-5b43ae17b33f",

"AfterContactWorkDuration": 15,

"AfterContactWorkEndTimestamp": "2019-02-13T10:14:23Z",

"AfterContactWorkStartTimestamp": "2019-02-13T10:14:08Z",

"AgentInteractionDuration": 19,

"ConnectedToAgentTimestamp": "2019-02-13T10:13:32Z",

"CustomerHoldDuration": 17,

"HierarchyGroups": null,

"LongestHoldDuration": 15,

"NumberOfHolds": 2,

"RoutingProfile": {

"ARN": "arn:aws:connect:us-west-2:111122223333:instance/8ca95e2f-891b-4e2f-9353-35d1e893233e/routing-profile/12e39435-6c3d-468d-975f-08ad53fc4f9d",

"Name": "Basic Routing Profile"

},

"Username": "dan"

},

"AgentConnectionAttempts": 1,

"Attributes": {

"greetingPlayed": "true",

"customerName": "Donna",

"accountType": "VIP",

"language": "es-US",

"accRef": "ABCD1234"

},

"Channel": "VOICE",

"ConnectedToSystemTimestamp": "2019-02-13T10:12:19Z",

"ContactId": "e2e44c3e-9d27-41b7-b581-247f83789bb2",

"CustomerEndpoint": {

"Address": "+441111111111",

"Type": "TELEPHONE_NUMBER"

},

"DisconnectTimestamp": "2019-02-13T10:14:08Z",

"InitialContactId": null,

"InitiationMethod": "INBOUND",

"InitiationTimestamp": "2019-02-13T10:12:19Z",

"InstanceARN": "arn:aws:connect:us-west-2:111122223333:instance/8ca95e2f-891b-4e2f-9353-35d1e893233e",

"LastUpdateTimestamp": "2019-02-13T10:15:31Z",

"MediaStreams": [

{

"Type": "AUDIO"

}

],

"NextContactId": null,

"PreviousContactId": null,

"Queue": {

"ARN": "arn:aws:connect:us-west-2:111122223333:instance/8ca95e2f-891b-4e2f-9353-35d1e893233e/queue/121588f7-596c-4147-9248-6858ed704a73",

"DequeueTimestamp": "2019-02-13T10:13:32Z",

"Duration": 27,

"EnqueueTimestamp": "2019-02-13T10:13:05Z",

"Name": "BasicQueue"

},

"Recording": {

"DeletionReason": null,

"Location": "mybucket/connect/vf-connect-demo/CallRecordings/2019/02/13/e2e44c3e-9d27-41b7-b581-247f83789bb2_20190213T10:13_UTC.wav",

"Status": "AVAILABLE",

"Type": "AUDIO"

},

"Recordings": [

{

"DeletionReason": null,

"FragmentStartNumber": null,

"FragmentStopNumber": null,

"Location": mybucket/connect/vf-connect-demo/CallRecordings/2019/02/13/e2e44c3e-9d27-41b7-b581-247f83789bb2_20190213T10:13_UTC.wav",

"MediaStreamType": "AUDIO",

"ParticipantType": null,

"StartTimestamp": null,

"Status": "AVAILABLE",

"StopTimestamp": null,

"StorageType": "S3"

}

],

"SystemEndpoint": {

"Address": "+442222222222",

"Type": "TELEPHONE_NUMBER"

},

"TransferCompletedTimestamp": null,

"TransferredToEndpoint": null

}

The vast majority of the data in this JSON Object is system generated, with fields like the SystemEndpoint.Address, which is “+442222222222“. This is the number that the caller dialed to get into Amazon Connect. However, there is also a place where you have total control of what information is added to the Object. Enter, Contact Attributes, within our CTR Event and we have a few key-value pairs set already:

"Attributes": {

"greetingPlayed": "true",

"customerName": "Donna",

"accountType": "VIP",

"language": "es-US",

"accRef": "ABCD1234"

},

Contact Attributes are incredibly powerful and will very likely be pivotal to your IVR and Data Strategy. We shall explore these in more detail in a later blog post.

2. Contact Flow Logs (CFL)

Contact flow logs are audit level logging for the contact within the IVR (i.e., they are only created during self-service and queuing. Once connected to an agent they stop, unless the customer is transferred etc.)

CFLs are streamed in real-time to CloudWatch where they are enabled by default for new instances. However, you need to START and STOP them by using the Contact Flow Editor.

CFLs are super useful and I highly recommend you enable them, even if you think you don’t need them yet. You can never re-create the data, and they provide a wealth of insight and troubleshooting capabilities. One of my personal favorites is using CloudWatch insights (superb tool!) to run real-time analytics on your Flow Logs:

The image on the left above is the CFL showing you the message the caller heard, the image on the right is using CloudWatch Insights to search and visualize against your CFLs.

3. Agent Event Streams or Agent Trace Records (ATR)

Agent trace records are created immediately when an agent state change is detected. For example, a record is created if an agent moves from Available to Lunch, or as in the example below, a state change occurs from CONNECTING to CONNECTED.

{ "AWSAccountId": "111122223333", "AgentARN": "arn:aws:connect:us-west-2:111122223333:instance/8ca95e2f-891b-4e2f-9353-35d1e893233e/agent/6ce798d2-7fa8-4578-827a-5b43ae17b33f", "CurrentAgentSnapshot": { "AgentStatus": { "ARN": "arn:aws:connect:us-west-2:111122223333:instance/8ca95e2f-891b-4e2f-9353-35d1e893233e/agent-state/b0b8d607-2317-4299-a475-28d88f83cef5", "Name": "Available", "StartTimestamp": "2019-02-13T11:11:38.038Z" }, "Configuration": { "AgentHierarchyGroups": null, "FirstName": "Dan", "LastName": "Bloy", "RoutingProfile": { "ARN": "arn:aws:connect:us-west-2:111122223333:instance/8ca95e2f-891b-4e2f-9353-35d1e893233e/routing-profile/12e39435-6c3d-468d-975f-08ad53fc4f9d", "DefaultOutboundQueue": { "ARN": "arn:aws:connect:us-west-2:111122223333:instance/8ca95e2f-891b-4e2f-9353-35d1e893233e/queue/121588f7-596c-4147-9248-6858ed704a73", "Name": "BasicQueue" }, "InboundQueues": [ { "ARN": "arn:aws:connect:us-west-2:111122223333:instance/8ca95e2f-891b-4e2f-9353-35d1e893233e/queue/121588f7-596c-4147-9248-6858ed704a73", "Name": "BasicQueue" } ], "Name": "Basic Routing Profile" }, "Username": "dan" }, "Contacts": [ { "Channel": "VOICE", "ConnectedToAgentTimestamp": "2019-02-13T11:11:53.053Z", "ContactId": "e7919b58-9a32-4c48-a3a8-9521fa2dec2c", "InitialContactId": null, "InitiationMethod": "INBOUND", "Queue": { "ARN": "arn:aws:connect:us-west-2:111122223333:instance/8ca95e2f-891b-4e2f-9353-35d1e893233e/queue/121588f7-596c-4147-9248-6858ed704a73", "Name": "BasicQueue" }, "QueueTimestamp": "2019-02-13T11:11:46.046Z", "State": "CONNECTED", "StateStartTimestamp": "2019-02-13T11:11:53.053Z" } ] }, "EventId": "fd663051-a2bf-4d33-8ea0-1e9443abfb3a", "EventTimestamp": "2019-02-13T11:11:53.053Z", "EventType": "STATE_CHANGE", "InstanceARN": "arn:aws:connect:us-west-2:111122223333:instance/8ca95e2f-891b-4e2f-9353-35d1e893233e", "PreviousAgentSnapshot": { "AgentStatus": { "ARN": "arn:aws:connect:us-west-2:111122223333:instance/8ca95e2f-891b-4e2f-9353-35d1e893233e/agent-state/b0b8d607-2317-4299-a475-28d88f83cef5", "Name": "Available", "StartTimestamp": "2019-02-13T11:11:38.038Z" }, "Configuration": { "AgentHierarchyGroups": null, "FirstName": "Dan", "LastName": "Bloy", "RoutingProfile": { "ARN": "arn:aws:connect:us-west-2:111122223333:instance/8ca95e2f-891b-4e2f-9353-35d1e893233e/routing-profile/12e39435-6c3d-468d-975f-08ad53fc4f9d", "DefaultOutboundQueue": { "ARN": "arn:aws:connect:us-west-2:111122223333:instance/8ca95e2f-891b-4e2f-9353-35d1e893233e/queue/121588f7-596c-4147-9248-6858ed704a73", "Name": "BasicQueue" }, "InboundQueues": [ { "ARN": "arn:aws:connect:us-west-2:111122223333:instance/8ca95e2f-891b-4e2f-9353-35d1e893233e/queue/121588f7-596c-4147-9248-6858ed704a73", "Name": "BasicQueue" } ], "Name": "Basic Routing Profile" }, "Username": "dan" }, "Contacts": [ { "Channel": "VOICE", "ConnectedToAgentTimestamp": null, "ContactId": "e7919b58-9a32-4c48-a3a8-9521fa2dec2c", "InitialContactId": null, "InitiationMethod": "INBOUND", "Queue": { "ARN": "arn:aws:connect:us-west-2:111122223333:instance/8ca95e2f-891b-4e2f-9353-35d1e893233e/queue/121588f7-596c-4147-9248-6858ed704a73", "Name": "BasicQueue" }, "QueueTimestamp": "2019-02-13T11:11:46.046Z", "State": "CONNECTING", "StateStartTimestamp": "2019-02-13T11:11:47.047Z" } ] } }

ATRs are used for various reasons, mainly for Workforce Management, and this is where Amazon Connect Technology Partners get their information from. You can also use them to create your own real-time dashboards as we have shown in a previous blog on Kibana and below.

4. Call Recordings

Customer interactions are by far the most valuable of all data available in any call center system. The reason for the call, the sentiment – this data can provide immense feedback loops.

And with Amazon Connect, to record 100% or a subset of your calls becomes a footnote! Calls are recorded in an open WAV format.

Bitrate: 256 kb/s
Stream Audio: pcm_s16le, 8000 Hz, 2 channels, 256 kb/s
Use approx 2Mb of storage per minute

They are stored in S3, and are generally available within a couple of minutes after the conversation has finished. Note: if the conversation lasts 3-5 minutes you’re going to see the recording much more quickly than a conversation that lasted say, 50 minutes. Once your calls are in S3 you have so many options including:

Leave them and use the native Amazon Connect UI to playback recordings.
Use a Technology Partner to collect the calls for their WFO Solution.
Run them through AWS services like Amazon Transcribe to get the transcription of the conversation, as we show here.
Do all of the above!

5. Live Media Streaming

Live media streaming captures the customer audio but differs from Call Recording in 3 very unique ways. First, it will only record the customer audio; you cannot capture the agent audio. Second, you can capture the customer audio during the IVR Menus, Queue Music and LEX Interaction. Third, it is streamed in real-time to Kinesis Video.

We will discuss more Live Media Streaming in an upcoming Blog, but for now, Amazon has created a peek at what’s to come. Below is a real-time conversation where the customers’ speech is in one language, and then it’s converted to another, also in real-time. The unhappy face emoji is the real-time sentiment of the customer, and the suggested actions are contextual based on the conversation!

6. Scheduled Reports

Contact Center reporting is pivotal to many businesses. With Amazon Connect you can use the CTRs and ATRs from above, along with the CFLs, to populate your data lakes and thirst for data by utilizing your own analytical tools like tableau, Kibana and Quicksight. However, to get up and running quickly, built-in reporting provides access to real-time and historical reporting that can be scheduled. The results are then saved in S3.

Technology Partners use these reports for workforce management as they provide rich data to feed their systems for intra-day reporting.

7. CloudWatch Metrics

Last and no means least is CloudWatch Metrics. Amazon documentation provides good details on exactly what information is generated. But once you have the data what can you do with it? Well, you can create Alarms when an event happens, like calls in queue, or a sudden spike in calls per interval.

And you can create Cloud-Watch Dashboards like our example below.

In conclusion…

One of Amazon Connect’s key principles is ‘openness’, and they have made this come to life in abundance. After all, it’s your customer and your data. The challenge now becomes how you best utilize the features and determine which are the right tools for the job. Why don’t you contact us and we will be sure to understand what you’re trying to achieve and find the right way forward for you?

Tidbits…

Thanks to Mike Haney for the CloudWatch Dashboard code!

I don’t really have much for you this time, other than, test and learn! Be aware of what’s available and start small, build your knowledge and capabilities and before you know it you’ll be unlocking the potential on a regular cadence.

Thanks Again! —Dan

Blog