What kind of data can you use, and should you use, for an investigation?

Why is data important

Data is important in an incident investigation because our goal is to find out what really happened—to learn more about the things that create success, and the things that may have been more trying. We do this as any investigator would: by collecting evidence. The clear evidence is what helps make an investigation as comprehensive and unbiased as possible.

We have two main sources to help guide us in discovering what happened: hard evidence (or the data that we can find and analyze), which includes things like chat logs, screenshots of graphs, tickets from customers, video call recordings. And then there’s soft evidence, which is information gathered from participants themselves (like interviews). We use data based on actual actions taken (or not taken) to drive recollection and help reconstruct the perspectives of responders as they progressed through the event.  

Data itself doesn’t tell the whole story, so typically, we’ll use the first to help inform us about the second. Once we examine data, we get a sense of where to look next and who to talk to.

We can recognize sources of data about an incident because they help us answer questions like:

• What was the responder seeing?
• What did the responder have access to?
• What made this confusing or difficult?
• How did the system respond to a particular input or intervention?

Depending on your organization, some sources are going to be available while others won’t be. Sources may be available for some incidents and not others, or some sources may be easier to retrieve than others. At the end of the day, it’s best to simply use whatever evidence that you have access to!  

Ease of availability vs investigative effort

You can think of many of these sources falling somewhere on a quadrant. You have:

1. More commonly available and less effort for the investigator to analyze.
2. Less commonly available and less effort for the investigator to analyze (or work with).
3. More commonly available and more effort for the investigator to work with.
4. Less commonly available and more effort to get and hard for the investigator to work with.

Note that for these quadrants we’re not really talking about the quality of the evidence itself. Like for video call transcripts, they might be available (or might not be), but the quality might be suspect even if it’s easier for an investigator to go through.  

There are lots of sources of data, but they’re not all equal in utility for your investigation or effort required on your part to extract information. Using this view, you can better prioritize what sources of data you investigate and prioritize your time as far as what data you analyze. Depending on your organization and the constraints of your investigation (especially time), you may have to choose to examine some data over others.

As you gain skill and experience as an investigator you may find that you are better able to utilize types of evidence that were previously prohibitive based on the constraints of your investigation.

In this article we will explore two prime examples of data to use as evidence in your investigation: chat logs and timelines.

Chat logs

Chat logs are probably one of the most important forms of evidence that you’ll have available to you. They hit that sweet spot between highly available, and very easily gathered.

Chat logs are in that first quadrant so they’re very good.

They give us an opportunity to see exactly what was said in the moment. We get an idea of not just the broader content, but the actual language that was used.

This can give us insight into other aspects of the incident, like what may have made things difficult and where things were confusing.

It also allows us the opportunity to understand how the communication took place. Was it all centralized? Did the responder have to hop around a lot?

Chat logs are also time stamped which can help us build a timeline later.

Another source of data, in the slightly less available quadrant, are video call recordings. Depending on your organization, these may only be intermittently recorded and if they are, they require a significant amount of time to extract information from, because investigators have to watch the entire video (even if they do it at 2x!).

Video call transcripts, while not always produced, offer an easier way for the investigator to sort through calls. However, they may be of lower quality. If the transcripts are consistently of lower quality, with issues like incorrect wording and unclear attribution, that can necessitate a lot of going back and listening to what was actually said, resulting in higher effort than initially expected.

Why build timelines with your data

Timelines are important because they provide a way of examining an incident that helps reveal how the event unfolded over time.

Timelines aren’t evidence, they’re not something that we find on our own, they’re something we construct and reference to help us make sense of the data we have.

Every person has their own view and model of what happened and how it unfolded. Even though we can now look back and see many things that those responders could not in the moment, this is not how the event took place for them. By creating a timeline, we’re able to help show each individual or team’s understanding of the incident. This in turn allows a more insightful discussion which includes different views.

Timelines are an effective way to organize data, and they also can reveal more data to use as evidence in an investigation. They can help us see gaps in response perhaps where resources, like responders, were scarce. And also document the way the event actually took place for responders over time, which can lead to teams making effective changes in the process over the course of multiple incidents.

You can find evidence for your investigation in all sorts of places like chat logs, video calls, or transcripts, ultimately, it’s best to use whatever you have access to. Now that you have a framework on how to find and evaluate evidence, you’re ready to go out and make it a part of your next investigation.

For more detailed information on these and other topics, you can always check out Jeli’s Howie: The Post Incident Guide