Part 1: Amazon Connect Reporting - Data Sources
In this blog post, we're going to take a look at the kinds of data generated by Amazon Connect reporting. This will be the first of a three part series that takes a deep dive into this important element - data. In Part 1 we'll look at the seven different types of data generated by Amazon Connect and how you can assist in its creation. Part 2 will show how to get access to this data in real-time reports via event-driven processes, as well as how to use that data in real-time to trigger other processes. In Part 3 we'll look at options on how to store this data for analytics and reporting.
Why is this so important?
Data within the contact center is a rich source of information. It can provide you with real time or near real metrics into why customers are calling, the performance of your IVR, repeat callers, customer sentiment and troubleshooting, to name just a few. Knowing what information can be generated by Amazon Connect reporting is critical in helping you understand the architecture you require and how to design your contact flows and processes. So let's get right into it.
The types of data generated by Amazon Connect reporting
We'll start off at 10,000 ft. The first item you'll likely notice is that there are more data sources than you first considered - seven in fact! We'll go into each of these in detail shortly. The other striking point to note is the varied use of other AWS Services. Amazon Connect is a building block. Sure, you can use it all on its own, but when used with the other AWS Services, it becomes so much more. The final takeaway you will spot is the real-time capabilities of the platform. Fifty percent of the data sources and metrics reports are streamed in real-time, the other 50% within a minute or two.
Here we've expanded on the overview and added some more depth to the data that's available.
| NAME | PURPOSE | ACCESS VIA | AVAILABILITY |
| Contact Trace Records | Information about the contacts made/received within Amazon Connect | Kinesis Streams or Kinesis Firehose | 60 seconds after creation |
| Contact Flow Logs | Detailed logging for the contacts as they navigate the contact flows | CloudWatch Logs | Real-time |
| Agent Event Streams | Events for Agent State Changes | Kinesis Streams | Real-time |
| Call Recordings | The audio recording of the conversation between the Agent and the Customer | S3 | 1-3 Minutes after call ending |
| Live Media Streaming | Customer side audio within the IVR/ LEX and talking with an Agent | Kinesis Video | Real-time |
| Scheduled Reports | Contact center reporting | S3 | Few minutes after scheduled to run |
| CloudWatch Metrics | Instance level metrics, calls in queue, packet loss etc.. | CloudWatch Metrics | Every 60 seconds |
Let's take a closer look at each...
Now you have a solid understanding of the data sources, including what they provide and when. We'll go into more detail of each. I must admit, I was considering a separate blog post on each of the seven data sources as there is more than enough to discuss on each. Instead, I'm whetting your appetite and we can discuss further over a coffee if you wish.
1. Contact Trace Records (CTR)
These are generated by Amazon Connect for each contact within your Instance. For example, an outbound call will generate a CTR as well as an inbound call. CTRs are likely to be the items that you will first consume and use within your Instance. CTRs are generated as a JSON Object that consists of Key Value Pairs (KVP). An example CTR can be seen below.
{ "AWSAccountId": "111122223333", "AWSContactTraceRecordFormatVersion": "2017-03-10", "Agent": { "ARN": "arn:aws:connect:us-west-2:111122223333:instance/8ca95e2f-891b-4e2f-9353-35d1e893233e/agent/6ce798d2-7fa8-4578-827a-5b43ae17b33f", "AfterContactWorkDuration": 15, "AfterContactWorkEndTimestamp": "2019-02-13T10:14:23Z", "AfterContactWorkStartTimestamp": "2019-02-13T10:14:08Z", "AgentInteractionDuration": 19, "ConnectedToAgentTimestamp": "2019-02-13T10:13:32Z", "CustomerHoldDuration": 17, "HierarchyGroups": null, "LongestHoldDuration": 15, "NumberOfHolds": 2, "RoutingProfile": { "ARN": "arn:aws:connect:us-west-2:111122223333:instance/8ca95e2f-891b-4e2f-9353-35d1e893233e/routing-profile/12e39435-6c3d-468d-975f-08ad53fc4f9d", "Name": "Basic Routing Profile" }, "Username": "dan" }, "AgentConnectionAttempts": 1, "Attributes": { "greetingPlayed": "true", "customerName": "Donna", "accountType": "VIP", "language": "es-US", "accRef": "ABCD1234" }, "Channel": "VOICE", "ConnectedToSystemTimestamp": "2019-02-13T10:12:19Z", "ContactId": "e2e44c3e-9d27-41b7-b581-247f83789bb2", "CustomerEndpoint": { "Address": "+441111111111", "Type": "TELEPHONE_NUMBER" }, "DisconnectTimestamp": "2019-02-13T10:14:08Z", "InitialContactId": null, "InitiationMethod": "INBOUND", "InitiationTimestamp": "2019-02-13T10:12:19Z", "InstanceARN": "arn:aws:connect:us-west-2:111122223333:instance/8ca95e2f-891b-4e2f-9353-35d1e893233e", "LastUpdateTimestamp": "2019-02-13T10:15:31Z", "MediaStreams": [ { "Type": "AUDIO" } ], "NextContactId": null, "PreviousContactId": null, "Queue": { "ARN": "arn:aws:connect:us-west-2:111122223333:instance/8ca95e2f-891b-4e2f-9353-35d1e893233e/queue/121588f7-596c-4147-9248-6858ed704a73", "DequeueTimestamp": "2019-02-13T10:13:32Z", "Duration": 27, "EnqueueTimestamp": "2019-02-13T10:13:05Z", "Name": "BasicQueue" }, "Recording": { "DeletionReason": null, "Location": "mybucket/connect/vf-connect-demo/CallRecordings/2019/02/13/e2e44c3e-9d27-41b7-b581-247f83789bb2_20190213T10:13_UTC.wav", "Status": "AVAILABLE", "Type": "AUDIO" }, "Recordings": [ { "DeletionReason": null, "FragmentStartNumber": null, "FragmentStopNumber": null, "Location": mybucket/connect/vf-connect-demo/CallRecordings/2019/02/13/e2e44c3e-9d27-41b7-b581-247f83789bb2_20190213T10:13_UTC.wav", "MediaStreamType": "AUDIO", "ParticipantType": null, "StartTimestamp": null, "Status": "AVAILABLE", "StopTimestamp": null, "StorageType": "S3" } ], "SystemEndpoint": { "Address": "+442222222222", "Type": "TELEPHONE_NUMBER" }, "TransferCompletedTimestamp": null, "TransferredToEndpoint": null }
The vast majority of the data in this JSON Object is system generated, with fields like the SystemEndpoint.Address, which is "+442222222222". This is the number that the caller dialed to get into Amazon Connect. However, there is also a place where you have total control of what information is added to the Object. Enter, Contact Attributes, within our CTR Event and we have a few key-value pairs set already:
"Attributes": { "greetingPlayed": "true", "customerName": "Donna", "accountType": "VIP", "language": "es-US", "accRef": "ABCD1234" },
Contact Attributes are incredibly powerful and will very likely be pivotal to your IVR and Data Strategy. We shall explore these in more detail in a later blog post.
2. Contact Flow Logs (CFL)
Contact flow logs are audit level logging for the contact within the IVR (i.e., they are only created during self-service and queuing. Once connected to an agent they stop, unless the customer is transferred etc.)
CFLs are streamed in real-time to CloudWatch where they are enabled by default for new instances. However, you need to START and STOP them by using the Contact Flow Editor.
CFLs are super useful and I highly recommend you enable them, even if you think you don't need them yet. You can never recreate the data, and they do a good job of providing a wealth of insight and troubleshooting capabilities. One of my personal favorites is using CloudWatch insights (superb tool!) to run a real time metrics report on your Flow Logs:
The image on the left above is the CFL showing you the message the caller heard. The image on the right is using CloudWatch Insights to search and visualize against your CFLs.
3. Agent Event Streams or Agent Trace Records (ATR)
Agent trace records are created immediately based on agent state change. For example, a record is created if an agent moves from Available to Lunch, or as in the example below, a state change occurs from CONNECTING to CONNECTED.
{
"AWSAccountId": "111122223333",
"AgentARN": "arn:aws:connect:us-west-2:111122223333:instance/8ca95e2f-891b-4e2f-9353-35d1e893233e/agent/6ce798d2-7fa8-4578-827a-5b43ae17b33f",
"CurrentAgentSnapshot": {
"AgentStatus": {
"ARN": "arn:aws:connect:us-west-2:111122223333:instance/8ca95e2f-891b-4e2f-9353-35d1e893233e/agent-state/b0b8d607-2317-4299-a475-28d88f83cef5",
"Name": "Available",
"StartTimestamp": "2019-02-13T11:11:38.038Z"
},
"Configuration": {
"AgentHierarchyGroups": null,
"FirstName": "Dan",
"LastName": "Bloy",
"RoutingProfile": {
"ARN": "arn:aws:connect:us-west-2:111122223333:instance/8ca95e2f-891b-4e2f-9353-35d1e893233e/routing-profile/12e39435-6c3d-468d-975f-08ad53fc4f9d",
"DefaultOutboundQueue": {
"ARN": "arn:aws:connect:us-west-2:111122223333:instance/8ca95e2f-891b-4e2f-9353-35d1e893233e/queue/121588f7-596c-4147-9248-6858ed704a73",
"Name": "BasicQueue"
},
"InboundQueues": [
{
"ARN": "arn:aws:connect:us-west-2:111122223333:instance/8ca95e2f-891b-4e2f-9353-35d1e893233e/queue/121588f7-596c-4147-9248-6858ed704a73",
"Name": "BasicQueue"
}
],
"Name": "Basic Routing Profile"
},
"Username": "dan"
},
"Contacts": [
{
"Channel": "VOICE",
"ConnectedToAgentTimestamp": "2019-02-13T11:11:53.053Z",
"ContactId": "e7919b58-9a32-4c48-a3a8-9521fa2dec2c",
"InitialContactId": null,
"InitiationMethod": "INBOUND",
"Queue": {
"ARN": "arn:aws:connect:us-west-2:111122223333:instance/8ca95e2f-891b-4e2f-9353-35d1e893233e/queue/121588f7-596c-4147-9248-6858ed704a73",
"Name": "BasicQueue"
},
"QueueTimestamp": "2019-02-13T11:11:46.046Z",
"State": "CONNECTED",
"StateStartTimestamp": "2019-02-13T11:11:53.053Z"
}
]
},
"EventId": "fd663051-a2bf-4d33-8ea0-1e9443abfb3a",
"EventTimestamp": "2019-02-13T11:11:53.053Z",
"EventType": "STATE_CHANGE",
"InstanceARN": "arn:aws:connect:us-west-2:111122223333:instance/8ca95e2f-891b-4e2f-9353-35d1e893233e",
"PreviousAgentSnapshot": {
"AgentStatus": {
"ARN": "arn:aws:connect:us-west-2:111122223333:instance/8ca95e2f-891b-4e2f-9353-35d1e893233e/agent-state/b0b8d607-2317-4299-a475-28d88f83cef5",
"Name": "Available",
"StartTimestamp": "2019-02-13T11:11:38.038Z"
},
"Configuration": {
"AgentHierarchyGroups": null,
"FirstName": "Dan",
"LastName": "Bloy",
"RoutingProfile": {
"ARN": "arn:aws:connect:us-west-2:111122223333:instance/8ca95e2f-891b-4e2f-9353-35d1e893233e/routing-profile/12e39435-6c3d-468d-975f-08ad53fc4f9d",
"DefaultOutboundQueue": {
"ARN": "arn:aws:connect:us-west-2:111122223333:instance/8ca95e2f-891b-4e2f-9353-35d1e893233e/queue/121588f7-596c-4147-9248-6858ed704a73",
"Name": "BasicQueue"
},
"InboundQueues": [
{
"ARN": "arn:aws:connect:us-west-2:111122223333:instance/8ca95e2f-891b-4e2f-9353-35d1e893233e/queue/121588f7-596c-4147-9248-6858ed704a73",
"Name": "BasicQueue"
}
],
"Name": "Basic Routing Profile"
},
"Username": "dan"
},
"Contacts": [
{
"Channel": "VOICE",
"ConnectedToAgentTimestamp": null,
"ContactId": "e7919b58-9a32-4c48-a3a8-9521fa2dec2c",
"InitialContactId": null,
"InitiationMethod": "INBOUND",
"Queue": {
"ARN": "arn:aws:connect:us-west-2:111122223333:instance/8ca95e2f-891b-4e2f-9353-35d1e893233e/queue/121588f7-596c-4147-9248-6858ed704a73",
"Name": "BasicQueue"
},
"QueueTimestamp": "2019-02-13T11:11:46.046Z",
"State": "CONNECTING",
"StateStartTimestamp": "2019-02-13T11:11:47.047Z"
}
]
}
}
ATRs are used for various reasons, mainly for Workforce Management, and this is where Amazon Connect Technology Partners get their information. You can also use them to create your own real-time dashboards as we have shown in a previous blog on Kibana and below.
4. Call Recordings
Customer interactions are by far the most valuable of all data available in any call center system. The reason for the call, the sentiment, the time range of the call - this data can provide immense feedback loops.
And with Amazon Connect reporting, to record all or a subset of your calls becomes a footnote! Calls are recorded in an open WAV format.
Bitrate: 256 kb/s Stream Audio: pcm_s16le, 8000 Hz, 2 channels, 256 kb/s Use approx 2Mb of storage per minute
They are stored in S3, and are generally available within a couple of minutes after the conversation has finished. Note: if the conversation lasts 3-5 minutes you're going to see the recording much more quickly than a conversation that lasted say, 50 minutes. Once your calls are in S3 you have so many options including:
- Leave them and use the native Amazon Connect UI to playback recordings.
- Use a Technology Partner to collect the calls for their WFO Solution.
- Run them through AWS services like Amazon Transcribe to get the transcription of the conversation, as we show here.
- Do all of the above!
5. Live Media Streaming
Live media streaming captures the customer audio but differs from Call Recording in three very unique ways. First, it will only record the customer audio; you cannot capture the agent audio. Second, you can capture the customer audio during the IVR Menus, Queue Music and LEX Interaction. Third, it is streamed in real-time to Kinesis Video.
We will discuss more Live Media Streaming in an upcoming Blog, but for now, Amazon has created a peek at what's to come. Below is a real-time conversation where the customers' speech is in one language, and then it's converted to another, also in real-time. The unhappy face emoji is the real-time sentiment of the customer, and the suggested actions are contextual based on the conversation.
6. Scheduled Reports
Contact center reporting is pivotal to many businesses. With Amazon Connect reporting you can use the CTRs and ATRs from above, along with the CFLs, to populate your data lakes and thirst for data by utilizing your own analytical tools like Tableau, Kibana and Quicksight. However, to get up and running quickly, built-in reporting provides access to a real time metrics report and historical reporting that can be scheduled. The results are then saved in S3.
Technology Partners use these reports to create custom reports for workforce management. They provide rich data to feed their systems for intra-day reporting.
7. CloudWatch Metrics
Last and by no means least is CloudWatch Metrics. Amazon documentation does a good job at providing details on exactly what information is generated. But once you have the data, what can you do with it? Well, you can create Alarms when an event happens, like calls in queue, or a sudden spike in calls per interval.
And you can create Cloud-Watch Dashboards like our example below.
In conclusion...
One of Amazon Connect's key principles is 'openness', and they have made this come to life in abundance. After all, it's your customer and your data. The challenge now becomes how you best utilize the features and determine which are the right tools for the job. Why don't you contact us and we will be sure to understand what you're trying to achieve and find the right way forward for you?