The CISO’s Response Plan After a Breach
Webinar Transcript | Air Date July 19, 2022
Watch the Webinar | Read the Blog Post
Sam Humphries:
Thank you for joining, we will get the webinar started momentarily. Good morning, afternoon, or evening dependent on where you are watching our webcast from. We will get started in just a minute, but first I wanted to cover some housekeeping information. Today’s webinar will be recorded and we will email you the link at the end of the live event. We will have Q&A at the end. So if you do have questions, please submit them in the sidebar. Be welcome to add them as we chat, but if we’ve missed them for any reason, we will review them at the end. Thank you, we will start momentarily.
Welcome to our webinar entitled The CISO’s Response Plan After A Breach. I’m Sam Humphries and I’m joined today by my excellent friend, Steve Moore. So as I mentioned, I’m Sam Humphries, I am head of security strategy for EMEA here at Exabeam. With me, is Steve Moore. Steve, can you introduce yourself, please?
Steve Moore:
Certainly. Thank you, Sam. I’m Steve Moore, chief security strategist at Exabeam, former customer, former advisor, and now employee.
Sam Humphries:
Thank you, Steve. So both of us have had the joy of working through many incidents during our careers. Steve has also had the joy of a full bone breach to deal with. So we are going to share some of our experiences, advice along the lines of the agenda that you see here. Smart folks among you will remember that there was a seventh bullet point around audits and kind of the fallout afterwards. We’ll cover that during the presentation. So don’t worry, we’re not going to miss it.
First of all, though, the term breach is I think an emotional term for some people and a confusing term. In fact, depending on where you look and what you want to read, you may find different descriptions of what a breach is. Now this is from my side of the pond here, the NCSC. Talk about cyber incidents and breaches almost in the same sentence, but talk about breaches of security policies, different attempts to gain unauthorized use, access, make changes, cause disruption or denial of service.
When we cross the pond to NIST, NIST do have two separate descriptions here. In fact, they have five different descriptions of incident and you’ll see them. There’s a lot of commonality in the words here. So, I mean, Steve, would you agree? This is confusing for most people.
Steve Moore:
One of the things that plagues our industry is sort of coming together for a common definition of good, or how do we define certain actions and activities? And this is an example of that. I think if you were to go and speak with 10 different organizations, even ones that I would consider mature, the nuanced definitions and examples they would have for an event, an incident, and a breach would be vastly different. And the way they would report on them would be vastly different. And so it’s important to get that language not only correct for your company, but then also when you’re describing it to external parties that it’s sort of tracking with what they expect as well. So it can be a bit tricky. And as you’ve outlined here to most, there’s an overlap that we need to work hard to clarify.
Sam Humphries:
Agree. Well, I remember when the good old GDPR, my favorite four letters, and when that first launched, the ICO here in the UK were absolutely swamped with calls because people who were experiencing events, intrusions, weren’t even sure what was happening, decided that this was probably a breach and the best thing to do would be to call the ICO. And they had like 500 phone calls pretty much the day that GDPR came out, I think it was in the first month, the vast majority of those were not classically breaches. So there was a massive issue with over reporting because of the lack of clarity around this. Now I like the way you’ve put this. This is a Steve Moore, T.M. Quote. Steve, do you want to walk us through this?
Steve Moore:
This was an attempt. The Genesis of this was to help maybe simplify some of this language. For me, you detect intrusions with capabilities, meaning this doesn’t always follow the case, you can be notified that you’ve had a problem externally, which happens more often than we would like to admit in our industry. But for the most part, you have a capability that allows you to identify the problem. You manage incidents with a process, meaning it’s a deviation from something that you want in your environment. You’re going to manage those accordingly via the processes you have defined.
And then this is the piece that I think people get the most wrong. When we talk about a breach, it’s typically a decision that’s made, meaning there’s more than just the security team involved that you call a breach. And that’s an important distinction.
So based on your internal governance and security program and privacy and legal and all the rest, you make the decision to say, okay, we’re going to call a breach, which sets an action, all sorts of other tasks, such as notification, different incident response and crisis management processes. So it’s sort of a tiered approach. Notice I didn’t mention events or any kind of technologies. We talk about capabilities, looking at an intrusion, which could involve many events. Again, the management of incidents could take many different forms. They don’t always have to be technical in nature, but again, the breach is called with the wider organization. You call a breach, you don’t detect a breach.
Sam Humphries:
I really like this. I think. And you’re welcome to steal this, if you like, if this works for your organization. I’ve heard a lot of CISOs talk about the fact that once they get to the point of needing to notify or thinking they need to notify, that’s the point where that group decision goes in. So I think this is nice and clear and that it was always that kind of, I think the muddy line between, is it an incident? Is it a breach? But I think this clarifies this really well.
All right. So response plan after a breach. We’re going to take you back in time a little bit. So the response plan really has to come in way before you say the emotional word breach. So we’re going to talk now about planning and playing well with others. So this seems like a very obvious meme, but if you are starting to make friends through the organization after the breach word has happened, that may not be the best time, right, Steve?
Steve Moore:
The joke I make is you don’t want to have to make introductions during a crisis. Now you always will need to, and there’s reasons why that we’ll get into deeper into our conversation here, but the more introductions you make during a crisis, it’s an indicator. So sort of keep count of that or reference other people’s problems. The greater that number, the lower level of preparation you’ve had. And I can’t state it any clearer. It means you either interviewed poorly. So you’re in charge, you’re the director or CISO or whatever, and you don’t have the visibility that you should, or you don’t have the influence in the organization or kind of third or tertiary to that is they just don’t care.
And so now there’s a problem that they have to be involved in. They have to get updated and have answers to these terrible problems that are happening during a cyber breach. So it’s just a higher friction situation that we want to avoid. But the joke is, don’t make an introduction in a crisis.
Sam Humphries:
Really not. I think, we’ve got to look at this from the fact it’s not the security team that gets breached, ultimately it’s the organization. So we need… Go ahead.
Steve Moore:
To me, I think it’s a question you have to, it’s a thought experiment that you have to get into as well if you’re in a position. From analyst all the way up to CISO or even if you’re not a security person, to say, if we had a breach, is there someone that I might need to speak with that I don’t presently know today? That’s part of planning. That’s part of thinking through how do we resolve these kinds of issues and I’ll even go further to say, it’s okay to have a breach. We don’t want to have one, but it’s okay. But if we have one, how ready are we to address it?
And the first thing is people. So if there’s people I don’t know, if there’s organizations we don’t have connections into, that’s a huge red flag. Someone should care deeply about that. And I can’t stress that enough, both from my own personal experience and doing advisory work for others and helping the security community. It’s amazing how many of those introductions are made too late.
Sam Humphries:
So who should we be bringing together in those situations? Do we need our full Jedi Council?
Steve Moore:
Right, right. Sort of the message here and this works kind of in two forms. The question I ask technology companies and is similar to the question I ask security teams and it’s what’s the greatest story that you can tell? Like for the past year, what’s the greatest story you can tell? And then once you told it, how high did that story go, meaning to whom were you able to directly tell it? And that tells me many things. First, you have to be able to find interesting things. You have to be able to then wrap a consumable story around it. And then hopefully, you have the audiences of the powerful that will listen to say, okay, I’m interested in the story you tell, I’m interested in the outcome and the risk associated with it. And hopefully that drives to greater influence and cooperation when we need to solve a problem before it’s an incident.
So there’s an important thing there. There’s a connective tissue. So that’s another thought experiment for those that have joined us to say, okay, what’s the best story we’re able to tell as a team, as a division, as a CISO, whatever that might be, and then who listens? And the same thing if someone’s attempting for, maybe it’s a security platform, again, what’s the greatest story you can tell with this and who listens? So if you’re wanting to evaluate like a vendor, it’s another way that if I’m doing advisory work, same thing applies.
And that’s something we should all practice, storytelling is incredibly important. But as an example, let’s say we found amazing things. Let’s say we identified some type of an attack. And we have the team identified an event of least occurrence that’s high risk. We have mimic cats on a system. And we believe that there’s been sort of a breakout from that system and credentials have been stolen. And all these wonderful… We’re telling this, sort of this movie scenario, right.
But if we write that up, maybe our story’s not good enough. Maybe we didn’t articulate it in print well enough, maybe it’s too technical. So maybe the story’s right, the capability’s right, but the way we’ve delivered it is wrong. Or maybe all of that’s perfect, but no one gives a damn and the highest the story went is the CISO or the director of security and no one else above wants to… There’s no outlet, there’s no mechanism to share what those details are, what those…
So it’s something that, again, if you’re in a situation where it’s not working, evaluate yourself against that to say, can I tell a good story? Can I identify things that are noteworthy? Who listens? And if you’re failing, take it to yourself first, but ultimately then the organization has to engage in that process as well. So continue to get better at your storytelling.
Sam Humphries:
There is a lot of power and stories. To the end, I’m a big fan also of tabletops and simulations, indeed Magic the Gathering, or indeed Magic the Gathering, depending on how it turns out, right? These can be super, super powerful in your organization. So you don’t want to practice on the day, I think is a fair statement, right?
Steve Moore:
And first off, I have no idea what this card is.
Sam Humphries:
Me neither.
Steve Moore:
I’m probably too old or the wrong kind of nerdy or something. But it falls in line though, with what we’re chatting about. I mean, the good thing about tabletops to be quite frank is even if you have no money, no budget, you can do a tabletop. Make sure it’s relevant, make sure that the theme of what you’re covering is something that’s not just contrived. So don’t waste the tabletop on something that’s sort of developed in a vacuum of experience. There’s plenty of people in a community that have sort of adoptable information you can use. I would suggest the easy ones out of the gate, phishing, credential theft, compromise credentials, all these sorts of things that happen, endpoint exploitation. If it doesn’t involve that, I would say you’re sort of missing the point.
You want to generate a scenario that gets attention that’s either similar to the problems that you have or similar to issues that have happened, and maybe in your vertical, in your industry, also supply chain, third party risk is becoming certainly riskier and more popular, but that’s sort of the first part. So don’t create it in a vacuum of experience. It’s okay if you have no budget, run it yourself and then keep a kind of a journal if you will, of who has been invited and who engages. So if someone doesn’t show up, that’s unfortunate, but make sure that then ask them then for a delegate to say, if they’re not available, if they’re not prioritizing, who is their delegate?
And as the, kind of the quarterback of this or the person who’s planning this, just keep track of that to say, hey, what’s my engagement rate against different verticals in my company, right? So I have, let’s say a high degree of participation from HR and legal, but not from manufacturing or whatever that might be, to say, okay, where am I lacking? Or maybe low engagement from legal.
And that kind of goes back to the prior slide where you have the Star Wars slide it. And again, the other thing I really like, the other one I advise from an executive tabletop, which is a little bit different. There’s a whole presentation on this, but it’s writing your own breach notification letter before you’ve had a breach. So do that as the exercise and look to see how uncomfortable everyone is when you sort of write your own sort of preemptive note, it’s your sort of public failure, but see who’s uncomfortable. Listen to the questions that are asked and talk about where do we host it? How do we notify? How do we contact? What’s the tone? All these things are incredibly important.
Sam Humphries:
We’ve got some more coming up on that, which is awesome. I reiterate, again, I think these practice things are so important. I did one recently on ransomware and you speak to people who’ve never been through a ransomware event and a lot of people will get quite on their high horse about, well, we don’t pay the ransom. We don’t engage with criminals, yada, yada, yada. And if you’re in a situation where things are falling down around you, ultimately, if you’ve got a business to run or you’ve got customers or people who rely on you, the decisions that you make may change based on the situation you’re in. So that’s so good to do.
Steve Moore:
The important thing to remember is it’s not their choice, it’s probably not their choice if it’s paid or not. And a security person that says we won’t, or we can’t, they may not have the grace, the opportunity to approve or deny that. And that’s the important thing to remember. It may not be your choice. Just like early on when we talk about what’s a breach, you call a breach and it’s through a wider team. Now, the state side here, there’s other rules, depending on where the money may go and other involvements and there’s recommendations, all that still applies. But again, it might not be, maybe at that point out of the hands of security.
Sam Humphries:
Right. Generally, sorry, I’ve not spoken to a security person who’s been able to say yes or no, we’ll pay the ransom in those circumstances. All right. Let’s talk about communications a bit more then. We’ve done some practicing. We’re going to get into SITREPS. So situation reports, for those who’ve never heard the term SITREP. So depending on where you are in the world, you may have various authorities that you need to notify at the point of breach. Now here in Europe, we’ve got GDPR, UK’s got some additional rules, again, for reasons.
But if you look at this map, pretty solid coverage of countries with some form of legislation or drafting… Say that word again, hang on. If we look at this map, you’ll see here, there’s pretty good coverage of countries. See why they have legislation in place, or have draft legislation going through. So depending on who you are and what you do, you may well have various authorities you need to notify. They may have different requirements as well. So this is not necessarily straightforward. And that’s just the external authorities part around it. And their rules may differ around how to notify customers, third parties and the like.
Steve Moore:
I mean, the point I’ll say here is that it’s the management of the delivery of notification and the artifacts that support it and the interested parties, whether it’s government or otherwise, and not even getting into customers yet, or partners, state governments, it’s amazing the amount of effort and how much overlap you will be answering questions over and over and over again. And it’s important to know who you’re going to have to contact and what does that matrix look like? And ultimately, then who’s going to manage that message? Because it can’t always be the same person, you’re going to have to delegate that sort of authority, because there’s going to be so much heat on you and on your team to say, okay, who’s going to manage the 11 different groups, the 20 different groups, the 25 different… And again, we’re not even into customers yet.
So it’s and then what information and how quickly do we need to notify, that’s another, speed is very important in many cases. The inverse of that though, in many cases, in state side, depending on the type of breach, you may be asked not to notify for a certain period of time, if it’s an ongoing, especially in cases of espionage. So you have to have no… If you have the right partners, whether it’s with FBI or Secret Service, they may tell you intentionally not to notify. But then for 60, 90, somewhere in there, right? So it all depends, right? There’s no one straight shot at this. So speed is important, clarity, purpose of direction, incredibly important.
Sam Humphries:
This is also why privacy lawyers get paid big bucks, right? It’s a conversation for another day, I’m sure. But when you get into those circumstances where you’re told not to notify, and then you’ve got another country saying you should, I don’t know, that must turn into some sort of living hell, but again, that’s why we pay the lawyers the big money. So we’re going to do our SITREPS, things we need to think about as part of the situation reports. I think it’s important here. We’ve got to think about ensuring that the person who knows what’s going on is going to be the source of truth. Now what’s going on is a broad topic, I would say. But at the same time, we need to be able to get a source of truth that is correct for the organization. But then the message gets more complex depending on who you are actually notifying, who you are talking to, what their role is going to be after they’ve received your message, there’s a lot more to this.
And also then we need to look at how do we manage all those different channels? Because the way that we communicate will change potentially between some of these teams, keeping things as simple as possible. Because most of the folks on the list here, if we go down to that second set of bullets, aren’t technical. So they’re not going to want like code level detail as to necessarily what’s going on, but things around impact and risk and what are we doing, that needs to be in good, simple, plain language that people could understand.
Steve Moore:
So when you’re working, if you’ve about to call or have called a breach, there’s going to be a small number of people who actually know what is actually happening, the status of everything. And there’s sort of rings of truth. And then that ring zero, that’s your source of truth. There’s typically a technical team, very small. There might be some outside help from people who do breach response. There may be some assistance from law enforcement, maybe, and they are going to define the truth and it will change by the hour. So that’s an important thing, it’s changing and sort of morphing and being added to, and it’ll change a hundred times, it’ll change a thousand times as the investigation grows. But what you don’t want is multiple sources of truth. Meaning you have to keep that core group very tight and you have to keep the message even cleaner.
So then it goes through layers of abstraction, much like any other case of where you might write up a status report and it gets changed five times before it makes it into the board deck, same kind of thing. So you want to make sure as it goes out, as you see in the middle of the slide, that there’s someone who is representative, primarily of legal, but also maybe senior security that then doesn’t share too much. You don’t want to do anything nefarious. You’re not trying to hold information back, but that source of truth doesn’t need to be shared with everyone. And that includes the internal company as well. They don’t need to know everything. They’re going to be real curious because it’s gossip, it’s company, it’s interesting. They don’t need to know everything and that’s okay. And it’s not meant that because you don’t trust them, it’s that you have to manage the message.
I would encourage share as much as you’re able. I’d say, say as little as possible, but be as truthful as you can because, with the caveat, to say, look, this is an ongoing investigation, variables associated with it can change and are changing daily. But this is the mission that we have. This is the information that we know at this point in time, and this is how we’re going to resolve it, right? And so you could do a class, you could do a day long class just on this slide, but getting this right is incredibly important.
The other thing is as you’re working on these problems, people of authority will want updates. Some of them by the hour. And so you have to set in your cadence, okay, here’s how you will get an update. Here’s when you’ll get an update, otherwise they’re going to be asking other people of power, where’s my update? And you don’t want that because then it just gets people irritated. So that’s the important thing. The other thing I’ve seen work well in the past is look, senior people will get briefed on all of this and their ability to comprehend what they’re hearing will vary and they’re going to be without sleep. They’re going to be speaking with media and lawyers and everyone else. What they should and shouldn’t say is going to get mixed up. And I think actually creating, I’ve talked about this in the past, you have green phrases and you have red phrases and you want to keep them focused on the green phrase. These are the themes that we’re going for.
And you may have heard that it involved a certain piece of technology or a certain process, or was associated with a certain nation state. Don’t say any of that, that’s a red phrase, that’s a red word, and we’re not going to include that in any of our external communication at this point because they can get confused. And I’ve seen it firsthand. I’ve seen it happen actually in multiple occasions. And so you may need to develop literally an executive tool because you’ll spend a lot of time. The other thing I’ll add to this is if you’re in this source of truth, you’re going to spend a lot of your time briefing people that are more senior to you to cover these topics, to get them comfortable. So they’re going to carry the banner.
Sam Humphries:
I think you raised a really great point on the tiredness point as well as really it’s amazing like three days into something, let alone a week, like how broken people can be. And the wrong things said up to the press can have huge ramifications. So I think I love the red and green phrases. I think that’s an awesome way of looking at it. The other thing I think we need to be aware of is around when you are sharing the message internally, making sure that people are clear on what they can share because…
And I’ve seen this, I’ve seen my words that I wrote in an incident email. I know we’re talking about breaches, not incidents, but I’ve seen those literally land on a very popular technology website after an incident happened at a previous company and that was, included customer names. It was quite detailed. And someone in sales had sent that email out to a partner to help them understand what was going on. So it’s done in good faith. And then next minute, that partner sent it on to said website. And that was not a fun moment. I will tell you.
Steve Moore:
That’s a big issue. And the way that that is prevented is you’re very careful as to what you put in print. You may even use a separate messaging platform because depending on, you may not use your traditional messaging or email platform for a couple reasons, but that’s one of them, but you have to be very guarded on what’s shared and what’s sent because anything that goes outside of that, here’s the sort of circle of trust. Whether it’s on purpose or accidentally, will end up someplace you don’t want it. And there’s rings of sort of trust.
I mean, this is basic investigative tenants that we have to sort of remember and think about, there’s need to know. And if you don’t need to know the details. The other thing I’ll say is this is important for everyday operations. And we often get caught up in it when we’re dealing with incidents and sort of managing the problems. We can put words, snark, jokes, whatever in old, in emails, as we communicate, all that has no place in incident messaging, in email, in chat, because all of that can become discoverable if there’s a lawsuit and you don’t want your bad joke showing up in the hands of opposing counsel or in a government official or an auditor, it just looks like it’s poor taste, even though we’ve all done it, you don’t want that in there. So there’s kind of two pieces. One is stay quarter of the message, don’t share any more they need to and keep opinion and snark out of it. It’s got to be fact based all the time.
Sam Humphries:
There’s always that risk also that it’ll be taken very literally even if you don’t mean it that way, so with you. So this is actually a nice segue into when we should communicate with and without emotion. This is literally one of my favorite sentences ever, she says smugly. I have seen this on every breach notification, I think since ever, pretty much. And for those of you that are listening and watching at home or in the office or wherever you may be, I’m sure when your data has been breached, because it would be very strange, I think at this point, if you’ve never experienced your data being breached, that you have received a letter from an organization or you’ve seen this on the website that starts with this glorious sentence, we take the security of our customers very seriously.
Honestly, when I read this now, it makes me want to throw up in my own mouth. It has been overused and I’m going to ran for a bit here. I make no apologies. It has been overused, any sense of emotion and it seems to have gone now, it’s become a running joke, honestly, in the security community because it doesn’t feel like that it’s actually true and well meant. So please stop using this. There are other ways of saying this. And now I’m sure that you do take the security of your customers very seriously. That’s not the issue, but it’s been almost used as a throwaway sentence at the beginning, especially when, in the cases of some breaches where there have been some pretty big failures across the organization that maybe it doesn’t ring completely true. So top tip from Sam, please stop using this sentence in your breach letters. Steve, what are your thoughts on this?
Steve Moore:
I mean, this is a very tough one. This often gets used not only in breach notification letters, but also just general information that’s found on the websites, information that’s provided to customers. So in general, sort of discourse, it gets used by those that are sort of answering questions for third party risk or trying to get a deal done. But then also included on breach notification letters. It’s often one of the first or second sentences that you would see. Now, I will say that for many breaches and massive failures, there’s a security team and individuals who are very passionate and they do this job even when it’s impossible. And so they do take it very seriously. It is beyond their career. It is their life. And so they do care. And so this absolutely does apply to them.
The problem is typically not them, it’s often the larger organization and you can tell that by the response that typically happens directly after the breach. You can look at, then the discussions of, well, we take it very seriously, but we’ve brought in outside help or we’ve added more money to the budget or we’re hiring more people. All these things are well and good, but if you took it seriously to begin with and why is there such a delta in the way the security team looks post-breach, it’s good that they’re getting more help, but what was the cooperation like before the event? Was their advice being listened to? So the phrase itself often sounds cheap and that’s a shame.
There’s other ways that you can sort of prove or disprove, you know, that you have a great security program and that it’s something you truly do support. If you’re in the position where you have to communicate this, again, outside of a breach scenario, list facts of the great capabilities you have. So rather than sort of just sharing platitudes, focus on the capabilities that you’ve built, focus on what you’re able to do and how well you do it. What is your maturity and efficacy along that curve, rather than just making these sort of grandiose statements, they will be used against you. So if you use them frequently, you’ll be evaluated against them. And if you have to use them in a breach notification letter, at this point, they sound cheap. So I would argue to also use other words.
Sam Humphries:
A hundred percent. I mean, don’t use the opposite of these words, that’s not a good look either. But at the same time, like you say, there are ways of demonstrating how good your security program is and your capabilities and your people without just saying seriously, like what does that mean? Like we take it jovially? No, let’s not do that either. There’s a sentence there that looks like it’s got emotion in it, but doesn’t feel like it, so those of us on the end of it. Should it be a case of facts versus emotion or can we have both things kind of tied in together when we’re notifying?
Steve Moore:
I think it goes back to the identity of the company. I know this sounds a… So part of the answer is it depends, how do you face, if you’re in a difficult situation, if the company faces difficult times, how do they behave? Are you the type of company let’s say, and this is a little bit extreme, but let’s say there’s a downturn in the economy. Are you the kind of company that would go out of the gate and just do layoffs immediately? Or maybe you’re the kind of company that would do, maybe the executives take cut salaries, so everybody sort of shares and we don’t have to eliminate jobs. And we all just make a little less.
The reason why I mention that is that same type of energy to say, look, we’re going to come together and we’re going to fight through this and this is our objective. This is the problem that we currently face, decide the language. How do you want to be seen? And maybe if you’re not strong enough to have that kind of message, maybe you’re working in the wrong place, but you got to own the failure too.
One of the other things I think the best organizations do often you’ll see the CISO sort of own the letter, but better organizations, the CEO owns it. So ultimately it’s their responsibility. And I think that’s admirable that they put their name on, in this case, the failure, because they’re sure as hell going to put their name on the success. And so if there’s a problem in there, that’s sort of the burden and blessing of leadership, not to say that the CISO isn’t responsible, because there’s a plenty of things that they will need to do. But owning the failure I think is also sort of owning the owner there too to say who’s going to represent this?
The last bit here on external messaging, I want to caution this. Many organizations retain crisis managers and crisis communications and experts, and that’s well and good. And you need training for that. Even getting down into, if you’re doing video, what words you choose, how should you sit? How do you… All these variables that most people don’t think about. How are you perceived on camera? But none of these folks back to the sort of the ring of truth, source of truth, they’re going to need coaching. So you’ve got to identify your security team members that are going to coach them on the security message. This is incredibly important. And this needs to be, this is almost another type of tabletop.
Meaning we have a crisis, we have experts in communication, we’ve got an executive that’s going to deliver the message back to owning the failure, maybe it’s the CEO. Well, how comfortable is that CEO in these concepts? And you don’t want them to stumble over these concepts. You want it to be muscle memory. So the best organizations practice this. And so being able to deliver it calmly and maybe with less emotion, in that regard, in terms of the practice of the delivery is important, but having some emotion like any other situation, that’s excellent internally to say, hey, this is our time to define ourselves. This is an awful situation, but we will be known more for the way we respond to the breach than the breach itself. So this is our time, you’re going to learn.
This is like taking a million dollar training class. If you could sign up for a million dollar class and go take it, it won’t be as good as what we’re going to go through now. I said those very words to my team, not all that long ago. And it’s true. So that is full of emotion, but you want to subtract some of it and make sure you apply it the right way and smartly when using it externally.
Sam Humphries:
Yep. Agreed. I think you’ll also find that you’ll have lawyers saying, don’t say this, don’t say that, that’s submitting culpability, like the word sorry has a lot of issues around potentially. But ultimately, I mean, you’ve got to show some empathy. If it’s going to go out and be wooden, but I wouldn’t then also skip out the door smiling at the same time. So this is so key. There was recently a big discussion on LinkedIn around this, about, would you trust your CEO to deliver the message? And a lot of people said, no. Now that’s something the organization has to work with. And it’s about that training, about that support, making sure that ownership is there and that that person is not going to get shut down into pieces by saying the wrong things, because they’ve been given the wrong information.
Steve Moore:
I think the answer there is, I think it’s fantastic that the CEO would want to deliver it, because some of them will hide behind others. So credit to them, if it’s a CEO that’s going to deliver it. And I think it’s, yes, I would support them. I will trust them and I’ll give you any and all time available to make sure that when that situation occurs, that the message is delivered as cleanly and clearly and as correctly as possible. So that’s part of their training that they can… Hey, it’s a team assignment at that point. And so they need to be ready, it takes practice.
Sam Humphries:
Absolutely. The organization is breached, not the CEO. So when we’re getting into facts though, making sure we’ve got some clarity and this will change depending on who you are communicating with. Some of these won’t necessarily apply to all groups, but if you can stick to these kind of areas around like, this is what we know, this is what we’re doing about it. What we don’t know is fair to say. What can be shared externally, if you’re talking to internal people. For customers, for internal folks, what are the recommendations? What should we be doing next? When will you get that next update? I know you’ve already mentioned this a little earlier on and how will that be delivered is really important.
I’ve seen before with execs running around, trying to get more information out of people whilst they’re waiting for the next update, then that can turn into like a cascade of maybe half information being shared amongst execs or into other teams, and it gets very messy, very quickly. So people that can trust the fact that you are going to provide them with the right information on time, every time in a way that they can consume is really, really important. Anything to add on this one, Steve?
Steve Moore:
I’ll give a quick example. If you’re in the middle of a large response, you will live on bridge calls. If you’re not all able yet to get in person, even if you are in person, there’s going to be bridge calls, because people are going to be out working on things and pulling together data and billing timelines and getting answers to things. And you may have 10 or 12 different bridge lines open and executives will lurk. In many cases, those bridge lines, there’s technical people working. So you’re going to hear keyboards and not very many words. You may go an hour or two before hearing anything, but work is happening. And okay, what are the DBAs doing? What are the… Back and forth. Cloud ops. What’s the sec security team doing? What are we… All of that.
And one of the things I think that’s important to do is, now you got to fan out, you got to bring in outside. You got to bring in help, you got to maybe even recruit some conscripts, but you have the people working on the issue. You have a scribe that writes down and documents any change that’s necessary that happens, so some sort of event. And then you have a person that’s sort of the speaker. And sometimes the scribe and the speaker are the same, but because you have lurking executives, I would sort of ask that the speaker or the scribe that even if there’s no update, change, change in status that they repeat the prior status every 15 minutes. And you note that. So right now, it’s 12 noon, this is the status. The next update will be given at 12:15. Any changes will be updated at that point in time so feel free to take a break and come back. We give updates every half an hour, every 15 minutes. And if there’s no change, you just repeat it.
And that was one of the best things that I accidentally figured out to keep nerves calm amongst people who were curious that had access to these bridge lines, that had access to these communication channels, but wanted to know what was happening. Rather than having a technician get interrupted. Because someone would join the call, barge in and say, “What’s going on here, right? Give me a status.” And then, you don’t want that. You want to avoid situations where you have, it’s called distraction avoidance. And that’s what we want to make sure that we always have for the technical teams that are working through this. They’re stressed enough. So that’s a little anecdote that I’ll share.
Sam Humphries:
A hundred percent. I’ve been on the same sorts of calls and we’ve done it, we did exactly the same thing because also you get execs joining that haven’t been on earlier. So for them, they are coming through. So to get that status to say, this is where we’re at, that, it also preempts any questions of them exactly coming in and being like, what’s going on. What’s this, what’s that? You give them, like this is where we’re at. This is the situation. And then the updates come and this is where you need to come to get them yada, yada health and they will be.
Ultimately, this comes down to optics. Now shouts out to my 90s kids in the house. I’m sure everybody now has their nose on the screen. So I won’t leave this up for too long, nor will I spoil for you what it is. But I did check that this was not an HR violation.
But the reason this is up here is those optics are so, so important, both externally and internally to make sure that, as you mentioned earlier, Steve, this can turn into, I wouldn’t say it’s the most positive event that will ever happen in your lives, but the difference between somebody remembering your organization for doing an awful job of handling a breach and thinking, well, I’ll never ever deal with you again, or I’ll never deal with you in the future, if they’ve never [inaudible 00:44:00] customer before, but they’ve seen this happen publicly. And these things get very public very quickly. So the importance of this is huge.
Steve Moore:
Also, I mean, look, get a little bit selfish about it too, make it about you. You have a mission, you have a team, you are part of this organization. You’re proud to be a part of it, hopefully, but you also have to manage the optics of your brand as well. And so one of the things I used to talk about, did a presentation on leadership and sort of career development and leadership during a breach. And it sounds a little strange, but that matters. What is your brand? How are you seeing… If you roll up all the stuff we’re talking about here, you need it to scale across an organization, but also have more or less confidence in your ability to lead and make decisions and articulate your thoughts. Is that greater because of the breach or did your value drop?
So the breach happened, you can’t change that, that already happened. We’ve already missed out on the opportunity to prevent that. But once it’s happened, then what do you make of it? Like any other crisis or a situation in life that’s massively important.
Sam Humphries:
Agreed. All right. We’re going to communicating with consistency, which is going to start with a history lesson from Steve.
Steve Moore:
Look, we hinted at this earlier. What you see on the screen, there’s kind of two things, if you enjoy history. One of the reasons why you would have a drummer, if you go back far enough, is it to keep kind of the beat and the cadence of soldiers that are in often approaching withering fire and that’s to keep them focused on the beat of the drum versus the bad things that are flying at them. The reason why you would have, you see the use of a bugle is something a little bit similar, but it was your mechanism of control. So is this, do we rest? Do we retreat? Do we advance? Do we fall back? And before that, you would have flags that represented the same things, right? You’d have certain indicators.
And this is done to keep teams, in this case, this is a military function, but in general, you need to ask yourself, what are my ways of communicating the same things during a crisis? And I don’t want to get too sort of hyperbolic on this, because most technical people are working a lot anyways, but when you have a major incident or a breach, it’s often kind of a 24 hour thing, it doesn’t go to sleep. It encompasses everything you do for a period of time, often years of work, which is misunderstood and I think under reported in terms of the effort you must put in.
And so the drum beat. So the drum beat is back to the cadence of communication. So how often, what do I share? How do I keep people calm? How do I tell them where they need to go back to the trumpeter or the bugle? So what is it that we do? When do we do it? What are the content elements we need to create and how often? And they get into this sort of comfort and confidence in what you’re doing if you can sort of be the team that provides that and stay, even in the way you communicate, notice that I’ve slowed my speech and I’ve flattened it out a little bit, same thing.
And so back, even the example on the call from the prior slide or so, it’s, hey, your next update will be in the next 15 minutes. The present status of this particular work stream is X, it’s 83% complete. We expect it to be done in one hour, but if there’s any changes that need to be made or updates, you’ll get that in the next 15 minutes. Oh, I just heard someone join, we’re 13 minutes from the next update or whatever that is, right. And any updates or changes that go into this are rolled up to the nightly executive, typically there’s a nightly call that’ll happen too, or a morning call, status call that all the executives join.
So all elements of progress and barriers to success from this call will be shared and rolled up into the executive call. So if you miss something here, know that you can collect it, the stat. So you’re constantly giving data and updates. And so that’s why we have drummers and little soldiers on the screen.
Sam Humphries:
And the British are coming apparently [crosstalk 00:48:50].
Steve Moore:
Not so much anymore.
Sam Humphries:
No, really not. We’ve got our own problems. All right. So I think this has kind of come off as we’ve been talking, but this is an interesting point. And I’ve seen this, I’ve experienced this first hand. When you’re on those calls, the chain of command kind of goes out the window a little bit. So where you might have quite a hierarchical organization, the spokesperson that call may be quite junior in the general scheme of things and be dealing with people that they have never really crossed paths with. They may have seen them on a website, they may have seen them at an all hands, but all of a sudden they’re responsible for providing information to people much more senior than them.
So, as you were saying about making sure those updates are clear and consumable, that piece also comes with an additional requirement for that. Whoever is going to be the spokesperson. And this is another Steve Moore is the strength to communicate truth to power. If you’re going to [inaudible 00:49:48], you shouldn’t have somebody who’s going to get nervous of there being execs on the line. I’ve seen execs crumble, more senior execs who’ve been on the line and be scared almost of delivering bad news, which in some cases, well, in many cases, things will get worse before they get better. Having the right person to deliver that message is key.
Steve Moore:
You covered as well. I mean, I don’t know I’ve much to add, but it’s a, for early days of the breach, especially, there’s a flattening of the organization and there’s this intermingling of levels. And so you have to think about how well am I explaining this, not going into too much technical detail, getting the point across to say, look, we have a plan for this. This is how it’s going to be managed. This is exactly what we’re working on right now. And this is exactly when we expect it to be done.
And if there’s an area where we need more cooperation with, make that very clear. We’re presently working, right? So it’s these types of messages. And the other thing I’ll add is you’re going to need to develop spokespeople and you covered it, but getting comfortable, start your junior staff, even in the SOC, even if they’re interns or new hires, they should all be getting up and presenting their findings once a month. Get them able to communicate, have them answer the question. This sounds kind of corny, but why are you here?
Sam Humphries:
I think that’s really good advice.
Steve Moore:
Why are you here?
Sam Humphries:
I’d ask myself that every day. I think the other thing that can happen on calls that’s important to be able to manage is sometimes you can get folks on who can be quite disruptive and that can also be quite senior people, especially if you might have the folks who’ve not really been that engaged with doing the tabletops or the simulations that have never been through a situation like this before and have come in kind of panicking. So be able to handle that piece as well is another good kind of string to the bow as part of whoever’s leading that call.
All right. So we’ve got our internal calls going. We also need to be able to make sure that the folks who are front of house and thank you for my front of house picture there, that I’m sure you’re all enjoying, that the people who are friends of house, so customer facing. And that can be everybody from support, customer services, folks on your front desk, it may well be someone’s going to turn up literally at your building and the person on the front desk is going to need to know something about what’s happening. If they’ve got an angry mob coming in for any reason that they should be able, they should know what they can and can’t communicate.
So again, this should be part of your practice. And part of your planning is to making sure people understand how to deliver a message in a crisis that you are providing a deliverable script to them. And also we mentioned earlier, kind of the green phrases, red phrases, how should they deliver that message? Now it shouldn’t just be a case of reading it and saying, I can’t tell you anymore. That’s it, in some cases.
If you’ve got somebody screaming at you down the phone or on a front desk, what can you do next? What are you empowered to do? What’s your escalation point? Where can you point someone out to get further information? Or how can you get further information on behalf of that person? Do you have like an automated communication service that customers can sign up to to be able to get those next updates, rather than having to sit on a phone queue, which is going to drive them crazy because your whole music is horrible. All those sort of things need to be ironed out way before you end up in a breach situation. And again, the clarity on when they’ll get the next update and that you stick to that is really, really important.
There’s a couple of schools of thought on this one, and I think we should need to clarify a little bit around containment remediation and what are those things really mean. So there is the school of thought of, there’s kind of the mundane school of do the big remediation event in certain circumstances, especially if you think you’ve got an adversary live in the environment versus kind of going through it slowly. In many cases, that adversary will be long gone by the time that you find out, which is unfortunate, because we’d rather catch them early, but if you’re at the point of breach and you found out afterwards, they may well not be in your environment or at least not as active as they were.
But I think, we talked about the differences between breach and incident. Containment remediation wins it over. I think there’s a lot of gray areas here. And especially if you’ve been working on something for weeks or months, it’s tiring, absolutely, it’s emotionally draining and any first chance to say right, we’re good now, let’s just go and do what we normally do. It’s easy to kind of take that option although it’s not the best option to go down. What would your advice be, Steve?
Steve Moore:
Well, I think that the expectation has to be, if you’re working on this kind of problem, that you explain to those that are less familiar with these types of issues that we first need to know what has been modified, changed or stolen in our environment? And there’s a reason why typically people, you mentioned mundane, but any other organization that you might bring in, you bring them in, people falsely might say to throw out the adversary, that’s completely incorrect. You bring them in because you lack the capability to understand where they have been and how they got in. And they’re going to build you a way to see your environment, to know what has been changed. So where has the adversary gone? Are they still in the environment? They’re going to build a timeline of an attack and they’re going to do that because you lack the capability to do so.
And then from that list of hosts and credentials and everything else that have been affected, only with that, can you then have in a remediation event. Now, as part of this, you’re also going to discover what are the weaknesses that allowed this to occur? So is it flaws on the endpoint? Is there some other web issue? Typically, it’s endpoint exploitation, it’s some sort of phish that happened, not in every case, but the vast majority, if you look at the latest DBIR, it’s phishing in credential theft, or the vast majority of kind of number one, number two.
So that then allows you to get to the point you’re at a remediation event, but the breach is nowhere near over. The joke that I would say is, I had someone ask me this live. I was on a stage speaking. They said, “How do you know when the breach is over or when the breach is through?” And I kind of thought about it. And I said, you know the breach is over when the executives begin behaving the way they did before the breach.
Sam Humphries:
It’s so true.
Steve Moore:
That’s the snarky thing I would say. But behaviors matter. And what I mean by that is when cooperation begins to draw closer to zero to solve organizational issues, you know it’s over.
Sam Humphries:
So we were talking about this the other day, and we were like, when is it physically over versus emotionally over? I think we can carry scars from incidents and breaches long, long into the future. Very much so. And it’s not just for the CISO, it’s your team, it’s the folks that have worked on the breach, it’s your customers and your future customers. And like we said earlier, how you behave and how you respond and how you communicate can absolutely have an impact, not just on… The relationship you have with your customers can actually strengthen. But it can also have a huge impact on whether or not somebody wants to do business with you in future.
Steve Moore:
That’s an important point to make. And I’ll say if you’re in management, in security, make sure that you have a relationship with your sales team. Most companies work for, you have to build and sell things in order to have a company. And so figure out who those people are that are in charge of sales, be an asset to them before there’s a breach, go in and spend 15 minutes and say, the phrase I like to repeat is you don’t have to spend a lot of time, but just let them know that this is the way that your security organization differentiates itself from your industry peers, just that one phrase. And then whatever that is say, we believe we’re the best at, we’re most proud of, whatever that is, right. And just have the relationship, give them some talking points. When people are asking about security related items, have a cheat sheet.
We have a third party risk programs, and there’s all these kinds of things, but have a relationship, go to their QBR, go to their, wherever they have their sales kickoff or SKO, be friends with the people that make money for your company. You’ll be surprised at the benefits that come from it. And know that if there’s a breach, that there’s hell to pay on their side. So both on customer support, customer success and sales. So be ready to have assets available to help them through this. And you’ll be well rewarded.
Sam Humphries:
A hundred percent. All right, we’re getting there. We’re nearly at the end. Thank you for staying with us on this because so much we can talk about here. And as we’ve already said, like some of these things could be a day training course, let alone in a couple slides. So this is really important. So the dust is settling and you need to understand like what’s happened, why it’s happened and what you can do to improve your posture moving forward as an organization.
So blaming the intern, generally, is not the right answer. I think we will all agree there. Although in this case, when I was looking for a fun intern picture, I liked this. I thought this was rather lovely where this was a, not a breach by any means, but a small mistake by an intern and HBOMaxHelp, good on you for supporting your intern. I like this a lot, but generally, it’s not going to be one person’s failure. And looking at it as purely a failure is not going to get you into a good spot. Now this is an opportunity to improve ultimately. So treat it as such. And again, it’s about working together as a team to be able to understand what went wrong, how it went wrong and what you can do to be better the next time around.
Steve Moore (01:00:12):
Totally agree.
Sam Humphries:
I am a big, big fan of the joy of root cause analysis. So you have to do this. I can’t stress this enough. I’ve seen enough times where there’s been a breach or an incident at an organization where they have got back to business as usual, and then kind of shut the book and said, all right, well done everyone. Thanks. Let’s just get on. And then same place next week is pretty much where we’re going to be at. You have to follow this circle. If you ignore the post-breach activity, which can take months, months into years sometimes, you will find that you are in a world of pain very, very quickly. And I say this, having seen it many, many times over.
There’s a whole separate who should run the RCA conversation we could be having. This is again, a training course in itself, but if you don’t do the post-breach activity, it’s going to screw you up for your preparation in future, and you’re just going to end up in a world of detecting problems, trying to recover from them, not really doing so. And you’ll be in this horrific state of never learning and never improving.
Steve Moore:
Right. I think the quick thing I’ll add is if… So set breach out of this, focus in your daily operations. If you were SOC or the people that are responsible for finding bad stuff and responding to bad stuff, aren’t writing up observations that lead to future change. Meaning if they’re saying, okay, there’s a reason why we think this failed, or there’s a reason why we can’t answer definitively why this happened. We have a gap in visibility, we have a gap in capability, or we know that this is an issue with permissions in temporary directories on workstations or whatever it might be, like get to the point where you know where the failure happened. Or we have poor visibility into credential management or credential behavior, whatever that might be, focus on that.
And those observations should roll up into some kind of risk register. And that allows for future cooperation, maybe investments, organizational change. So your observations, your intelligence should drive maneuvers, is what they say. So focus on that.
Sam Humphries:
Agreed. We have covered a lot. I see this. We do have a great guide for you. If you’d like to do some reading afterwards, we’ll make sure this is sent out to everybody who’s attended. There is a guide that literally covers a lot of the topics that we’ve already talked about. Some in more details. It’s a nice compliment to the time you just spent with us. Thank you very much. We also have… And Steve hasn’t mentioned this, but when he did his radio voice earlier, Steve is the host of our marvelous new CISO podcast. This is I think a great, great episode to listen to. So when you’re in your car with your family, make an event out of it, but Dave Damato has some really, really great points about like what happens just after a breach happens. And how does that feel like for a CISO? So really good bit of external homework for you. Steve, anything to add on this one? I know you’re a big fan of Dave.
Steve Moore:
Huge fan of Dave. Credit to him. Listen to his show, you’ll learn a lot. I promise.
Sam Humphries:
All right, we’ve got time for a couple of questions and that’s, but I think we’ve got a few here that have come through. So first question is around what would you recommend for execs who seem to be not really get engaging with the whole kind of preparation process, maybe saying like I’ve got 30 minutes to devote to this, or don’t even show up at all when you’re doing your preparations and they should listen to Dave Damato is what they should listen to. Does it stop?
Steve Moore:
I mean, calendars are always busy. If they can’t show up, send a delegate. If you are the one that is trying to run a tabletop or get engagement, just keep good notes and send the invitee list to your leadership and send the participant list. And then if there’s a gap in areas, make sure you track the areas that lack representation and report that into someplace outside of IT.
So you want to include people typically in the risk or privacy departments and say, look, this is, I would like to report to you the status of our tabletop exercise for incident response on this subject matter included in that are the groups that have participated, so give them credit. And we lack participation from these areas, which keeps us from being successful in this way, right? So our next event is next month or next quarter, and we will reinvite them. We’re sharing this information with you for visibility and accountability purposes, and that’ll be enough to politely set the tone and just stay at it, stay at it, ask the delegates if they can’t show up, ask them to send somebody and just keep working way down.
Sam Humphries:
I think I’d just add to that as well is kind of, we talk to them and I ask them why aren’t they attending if they can’t send somebody else? And I said flip a little at the beginning, getting to listen to that podcast. I think if you can help them understand a little bit more what as to why this is important. It’s not like we’re stuck for examples of breaches anymore. I mean like what, way back when we just talked about the target breach all the time, now it’s breached Azure. There’s probably somebody notifying while we’re talking.
Steve Moore:
It’s important to just make it human, have relationships, don’t make the first request that. Like, if it’s someone important enough to have… If they’re senior enough, especially if it’s an executive table top, have you ever had lunch with them or drinks, or have you ever talked to them outside of this? Again, don’t make the introduction be the ask, right? There’s a curse at the call, take that away.
Sam Humphries:
I like it. There’s much to be said with that. Last little question then, and it ties to this. How frequently should we run these types of exercises?
Steve Moore:
I think it’s more important that you have levels of exercises, so you have some that are executive based, some that are technical in nature. I think you need to have a broad based incident response team that meets, I would recommend twice a month at least, where there’s representatives from several different groups. You may not be doing a tabletop, but instead of that, you’re going to be reviewing sort of persistent incidents or issues of concern where you’re going to cover what has happened, and then you’re going to cover what you would like to work on in the future. So that’s sort of the machine that runs and then that collective group will use those observations to create yearly, quarterly, every six months, tabletops that are representative to the needs of the company and do them at a couple different levels.
I think that you can… The other thing that’s important is make sure that there will be takeaways from these tabletops, things that you miss, questions, mistakes, observations of needs, and those should fuel future requests, budget requests, and capability requests and things. So timing is less important, but inputs and outputs are maybe more so and doing it at multiple levels. So it’s kind of dependent, but meet frequently with sub-teams.
Sam Humphries:
I think, also having the… I mean there’s no kind of pass fail on these things. There’s always more we can be doing. And certainly, I don’t know a team that sat there sitting on budget thinking, cool, I wish I had something to spend this on. So if you can use it into your advantage there as well, then that’s a good thing. All right. If you’d like to stay in touch with myself and Steve, here’s how you can get hold of us on the LinkedIns. We have a wealth of information also on our seize the breach hub. We will be in touch with the guide that we mentioned. And with that, thank you very much for attending, we appreciate your time.