ERP043 - Attacks in the MSP Channel w/ Kyle Hanslovan

Welcome to Evolved Radio where we explore the evolution of business and technology. I'm your host, Todd Kane, this is Evolved Radio and we're talking about everyone's favorite topic in the MSP channel today, security. There have been a string of coordinated attacks on MSPs recently. Some large players like Wipro, Fujitsu and IBM, as well as a number of smaller regional MSPs. Today I have Kyle Hanslovan, CEO of Huntress Labs to join me on the podcast. To give his expert viewpoint on risks these attacks pose and how IT service companies should be securing their environments to avoid being the next victim. If you enjoy the show, be sure to subscribe on iTunes, Stitcher or wherever you get your podcast from. Also, be sure to check out the webpage evolvedmgmt.com/podcast for show notes, links to my guests and to check out previous episodes. Now, let's get started. Joining me on the podcast today is Kyle Hanslovan, CEO of Huntress Labs. Welcome, Kyle. Stoked to be here, Todd. Thanks, brother. Awesome. So, uh, it has been a very busy and a very scary month in the MSP channel. So, um, I really, really appreciate the work that you guys have been doing to, uh, put shine some light into the channel and to give some people some perspective and background and and displayed research, uh, in what what's been going on. We'll definitely get into, uh, into some of the details of the MSP, uh, attacks that have been going on. But, uh, maybe just to to give us a a launch in, if you you have a very interesting background. I don't think, uh, it's often you come across, uh, legit NSA operatives, so you definitely have the real credentials as far as a a security operator. You want to give us a bit of your background on the security side? So, I I started in the Air Force about 15 years ago. I just happened to be one of those kids that, uh, was pretty good at breaking things and there was a pretty cool job available. I pivoted into like the defense contracting, still supporting in, you know, intelligence gathering at NSA. And then, uh, use kind of that offensive expertise to pivot into defense. It seems like after a little while you get bored and it's time to, you know, kind of make a difference instead of just cause some trouble. Yeah. So, you're you've, uh, worked for the government, then as a contractor, working in the in the security field. And then formed, uh, the company that you you run now, Huntress Labs. And for I think a lot of people in the MSP channel will be familiar with Huntress. Um, for those that are not or people, um, not in necessarily in the MSP channel, give us a quick background on Huntress. So, the whole idea is my job was long-term persistent access at NSA. And it really bothered me that once I slipped past preventative products, it remained unchecked. So, when I looked at how I could give back with my co-founders, we said, look, there's this whole market that usually enterprise doesn't even, you know, help, let alone help them focus on what happens if prevention fails. So, the long story short would be Huntress is there to find an incident and quickly get it the heck out of your network before the situation escalates. Yeah. And I think that that that's a sort of a good touch point on the as well. I think it feeds into other parts of the conversation that we'll get into. So much of security threats now are invisible. And I think that that's a big departure, uh, maybe not recently, but over time. Where it used to be, uh, hacking attacks and things were much more about visibility and notoriety. And now the most successful hack attacks are the ones that you are completely unaware of. And they they sit in the background and do nefarious things that, uh, and the longer it stays dormant, the longer those threat actors know that they have a legit source that they can continue to pump. Do you what are your thoughts on that? So, I think most people don't ever think about the economics of hacking, you know, you always think of this shady person in a room somewhere. But a lot of times, especially in crimeware or even the advanced nation state attacks, like there's a lot that goes into saying, if I'm constantly going into a network and getting kicked out and having to go back in. It's really inefficient just from a financial resources standpoint. So, I think the reason we see way more undetected lurking attacks or even just supply chain where you might not even have insight because it happens somewhere further up the supply chain. That's the economics are driving that. Yeah, and we, uh, we touched on that a bit before turning on the recording. But the another good point is so many of these attacks recently. Especially on the big guys that are happening very, very recognizable large sort of Fortune 500 industry names. Uh, they're actually receiving some of these security breaches from the supply chain. Where there's either contractors, uh, vendors that they utilize for their their production purposes and things like that. Where it's a softer target within a larger target. That's definitely a growing trend as well where they kind of analyze who are the subcontractors, who are the companies that work with this larger company. And can we get into one of these in order to push ourselves up into the bigger company, right? Yeah, and I, you know, any security person that has been around long enough, usually when they're not affiliated with the government. They think of the target case, right? Where it was supply chain, I think it was Hvac, uh, you know, subcontractors that led into the the compromise of some of those point of sale terminals. But I I think we we've now realized in 2019, it's a little bit different. Supply chain is anything and everything. It's who provides the IT, who provides and stores the data. Who enables our businesses to be more efficient or also some of the people who can really be the ones that hinder us at the same time. Yeah. So, that leads well into, uh, the the topic for today, which is, uh, attacks in the MSP channel. Um, so I I think this is this is, um, an interesting reflection point for people. Where people in in the IT services channel, especially in the MSP channel. They've spent the last two or three years talking a lot about ransomware. And in a lot of ways it's been kind of a a bit of a sales boon for them. Where there's enough visibility around these types of attacks that that, uh, most people are aware of them. And know someone that's been hit by them. Uh, and it it's been good for business in a lot of ways because people are rightfully paranoid about ransomware and getting attacked. And I think the shift that we're seeing now is why go and hack 100 companies when you can hack the MSP and then use their tools to control and dominate those 100 companies, so a single attack vector rather than a single a 100 different, uh, hacks. It's become very juicy targets, so my feeling and I I definitely want your input on this as well. That, uh, MSPs are becoming very, very visible targets and people are right to be to to for their growing paranoia about these types of attacks as well. Yes. So, US Department of Homeland Security did their pitches earlier this year where they kind of blew the the I don't want to say it, uh, too specifically, but they more or less threw China under the bus and said, hey, this, uh, this APT has been attacking MSP since 2014. For a lot of folks, they were like, oh, I I thought this might have been the case. But I think we tend to lose focus that MSP is a big word, right? And often times it went after, like you mentioned, some of those bigger, you know, the IBMs, the HPs, etcetera. And I think Brian Krebs just did an article about this this week about eight of them that were targeted. Uh, but that's changing. That's changing dramatically now that attackers are not just doing this go deep into the biggest customers, they're also going very broad. And that's important to me. Because ransomware to me, if you would have asked me two years ago, I would have told you, hey, it's down trending. We're seeing it down trend in favor of crypto coin. Turns out once again, economics play into a big thing. Crypto coin crashes in value. And now we see once again ransomware being used for extortion. Um, maybe something that, uh, that that can give the audience, you know, a thought is. We're seeing this go broad where MSP tools are used to attack large amounts of customers, so that's clearly going to continue. But what happens when these attackers realize that maybe if I go broad for an MSP, I could probably extract a little bit more value after, you know, out of a company that's. Maybe if I encrypt the right data, if somebody's working on a million dollar proposal for like maybe a a contract. Are they willing to pay $50,000 to get that back? If you encrypt them right at the right time. So, I think that's kind of where we're going to see this evolution continue. Yeah. And we've even seen that there was a a story this week of an MSP that paid the ransomware. And then charged the client, uh, sort of an upsale on that that that decryption saying, hey, we rescued your data for you, right? Here's here's the fee for that. Uh, so. I definitely don't advise that. Um, and it it's maybe it touches on an important point. I don't think there's necessarily a good answer on this or maybe you're as black and white as I tend to be on this. Do you pay the ransomware? I I think I think we all know it just encourages more. Right. Um, you right, you at the same time, I think it's real easy to have that opinion like I would never pay a kidnapper to get my kids back till it's actually my kid. Right. Yeah. Yeah. Yeah. So, that's uh, yeah, I think I am black and white. But, um, I think some of that perspective is from somebody who's not facing. Well, I just sunk a million dollars into a proposal. And if I just have to pay $50,000 to get it back, the ROI might actually be paying. Which unfortunately encourages the attackers to do more. Yeah. So, you and you mentioned, uh, crypto coin. Um, the the the distinction between the ransomware and the crypto coin. Um, by that, do you mean where there's, uh. Stuff that gets encrypted and people hold the the data hostage versus leaving malware that's doing like mining and sort of crowd sourcing individual devices. Is that what you mean by the distinction in those two? Yeah, that you you hit the nail on the head. And thanks for clarifying that. Um, these same attackers instead of installing ransomware on hundreds or thousands of computers. They could have left, you know, a small piece of software that eats up the CPU and mines, you know, whatever the new Bitcoin dejour is. Or Monero or whatever one's best for privacy, so they didn't. They chose the economic decision to use this to immediately extract value. Which I found very interesting. And the, uh, the I think the the immediate thought for people, uh, is. Well, you know, I've got a good antivirus. Um, and I think we've seen certainly through some of these attacks. You guys detailed a bunch of this in your research. You had a really awesome webinar. This this past week, I'll I'll link to it as well, so people can register and check that out. But you go you showed some of the tradecraft as well. And some of the things that I I found interesting was that, um, you actually captured some, uh, video and screenshots of people actively disabling the AV as a part of their attack in these, uh, these things. So, I wanted to have you touch on that around fullness of, uh, of the AV, but also secondarily, uh, some of the attacks, the way that they were doing it were simple and sophisticated. Where they were using memory resident PowerShell scripts, so that there was never anything actively on the machine unless it saw tax come through memory. It's not like there was a virus that's uploaded in an executable that needed to be run. So, I thought that was interesting as well. Can you touch on both of those? Yeah, so I'm going to give you kudos first for saying the word memory resident. Hey, people continue to use that fileless misnomer. Where sometimes there's often files used and abused. Like PowerShell is a file. But huge kudos on that. Um, as for the the AV side, especially related to these. Folks continue to ask like, Kyle, can I just stop? And I think that would be the the biggest foot gun we could give our clients as an, you know, as advisors. Would be remove your antivirus. Like prevention matters. I see the most sophisticated and I see the most basic malware every day. So, from my perspective, like, you need to be running antivirus. The next question is, you know, that's kind of like saying, I should be wearing sunscreen. There's a lot of manufacturers of sunscreen. There's a lot of different, uh, you know, SPF and how much protection do I need and how often do I need to reapply? It's a hard question to answer and I think a lot of time we focus so much on just the technology of, can you give me the sunscreen that doesn't get me sunburnt? Every one of these cases of the videos that we shared, which by the way, that was an ironic benefit of the attackers. They used and abused a tool, didn't didn't exploit it, but just got credentials to a tool that happened to record their own tradecraft. So, we were lucky the attackers chose that way or we would have never got to see their actual techniques. But pivoting back to that antivirus, um, yeah, you need it. I see more and more people asking themselves the questions of I'm running Windows 10 or the new fancy server 2018 or excuse me, 2019 or 2016 server. Can I get away with just the built-in antivirus? I'm seeing a lot more conversation on there. And I think that's something that is worth thinking about. Um, I see a lot of partners that are using great next generation antivirus and EDR products to give them more telemetry. And that's so focused on the technology side. I often ask, where's the risk? Like we can talk about technology all day, but we should probably separate the risk as well. And say, maybe for some clients, they have just such tight budgets that they have to accept the risk of getting hacked. And that's hard, that's a really hard, hard decision to make. But, um, you know, the the second part you asked about was the sophistication versus simplicity. If you have the greatest next generation products in place, but you don't take care of like your basic foundations of hardening, you know, preventing access, giving people administrative privileges, those undermine some of your great investments. So, I really think we as practitioners, we need to take sometimes the technology hat off, take a step back and even though it's not sexy, ask ourselves, what is the real risk we're trying to solve and making sure our clients know at the end of the day, I'm accepting this risk. That that's something that needs to be a business decision, quit making it a technology decision. Yeah, and I think, um, you hit on exactly sort of what I think is important. As an underlying theme to this discussion and what people really need to take home from the types of events as they continue to grow is the fundamental support, right? Like people, uh, I think the especially technology people, they always want to look for a tool to solve their problems, regardless of what it is. It's what they're comfortable with, it's what what they they think should be available. But, you know, if you're the the house equivalent where like if you have a really fancy alarm, but you continue to hand out keys to everybody in the neighborhood, your house is not safe, right? So, focusing on the passwords and the the hygiene around the the basics of security management and prevention. I think are huge themes for this, right? It's huge. I mean, thankfully, I I I got a chance to go for a day to Kaseya's Connect conference. That was in, um, Las Vegas and the presenter that they brought on, she said the number one thing that she would like to stomp out are use of passwords. That's a very ambitious goal. That's that's typically what you hear from security researchers. But she was undermining a point that like you said, it's it's the key. Should I actually enable access? Like people who manage Airbnbs nowadays, they have a rotating keypad. That every time somebody checks in and checks out of an Airbnb, it's a new keypad on the door. That person can't come back with that same key the next day. And that's a great way of approaching that risk. So, yeah. Let's stop giving out the keys, let's stop or start rethinking about the architecture. That way, uh, you know, we could focus on the harder problems later. Right, so simple things like using password manager, complexity of password rotation, right? Like you said, always just be changing the key because passwords could be reused. Passwords simple, passwords could be compromised. And I think as an MSP, one of the things that I think rightfully people struggle with. But you can't ignore this as a problem is that password rotation becomes difficult. How do you rotate admin passwords 100 clients at a time, not break a bunch of stuff? Or have that that sort of lost information or the inefficiency of having to source that information and keep track of it. Yeah, it's complicated, but the alternatives are very, very scary. Right. Yeah, it's, um, so much so people are designing now software to rotate things automatically. If anybody watches that webinar, we call out Microsoft's local admin password solution. Laps. It's free. But that the Microsoft released it for free because the result of not doing it is so dramatic. Uh, that it it made sense to give it out. The exact opposite is actually happening too, which is cool when you think about it from a psychology side of. We're saying change the admin passwords automatically. But we've discovered for users, you know, a lot of the current advice is stop changing those passwords so frequently. Focus on a nice long password because automated programs. Those are very good at repeatable processes. We as humans are pretty bad at anything that we need to make sure that it happens perfectly on a schedule. Um, so yeah. I find it interesting that we're using a two-prong approach here. Not just hitting everything with a hammer, we're saying, use automation for the things that could be protected automation. For users that don't need it, let's focus on the user experience so they don't undermine the security themselves. Yeah, definitely. I I've never been a fan of password rotation. So, I'm glad that that one is is catching on. I think as long as you you combine a good password with with decent 2FA. Uh, SMS if you have to, but there's that's not secure at all. Uh, an authenticator is better. And having some type of physical key access like Ubi is. I think that your best measure. But, you know, it's not necessarily. So, as you said before, kind of pick your scale and absorb the risk as a result of which one you pick. Yeah. I mean, thanks for thanks for highlighting that piece too. Because, I mean, SMS, right, a a researcher online, we we kind of sparred with on Twitter for a little while talking about the issues of SMS being undermined. And I've seen it. Uh, I might have even in engagements got to play with undermining SMS-based two-factor. But, um, at the end of the day, it was way more important that things that I didn't think of, I immediately said everybody should have an authenticator app, non-SMS. And somebody hit me up, the security researcher that said, look, Kyle, you might be an expert at this. But what about those users who have dumb phones instead of smartphones that can only receive, you know, a text? How do you handle that? And that that was enough to put me on on on my, uh, on my laurels and make me rethink my strategy. That actually SMS was way more important than I realized, even if it's not perfect. Right. Okay. So, and, um, we'll get dig into the the attacks themselves. Um, and there's a few things that I want to set the record on. I sent an email, I have this weekly newsletter. I sent out an email this week and kind of underlining a few things that had happened. And wanting to make people aware of the risk that is out there as as an MSP. Um, but one of the pieces that I think I I really want to level set is that. People were afraid the tools were being compromised. So, there was there was a bunch of flack for a bunch of vendors in the industry that were getting, uh, hit. Um, so Wipro was, uh, sort of one of the first ones that really got a lot of attention as as a vendor that was under attack. Um, one of their tools was, uh, they were using, uh, screen connect or control and that was a an attack vector for them. And then there was, uh, Kaseya and Webroot were also sort of the the names were thrown out. And as before the information was available, a lot of people started talking about how. Uh, those tools were used in some malicious way. And the important point that we I think both of us want to level set here is that the tools were not the attack vector. The tools are not unsafe. It's that these tools have remote access to hundreds and thousands of devices. So, all you have to do is get inside the inside the the the MSP and then find one of these tools to start deploying payloads. So, they were using the RMM as an RMM, not the attack vector. So, I think that's a really important as well, uh, point to underline as well. Yeah. I'll I'll second it. We have not seen a single case in regards to the the Wipro or any of those incidents that happened afterwards. Of control being exploited or Webroot being exploited, meaning a vulnerability in those software or Kaseya. It appears that it's. Just like if you left your keys at a gas station and somebody took your keys and walked off, you know, because you left them in your ignition. And somebody drove your car away. It doesn't mean Ford or Fiat or whatever you're driving these days is, uh, you know, vulnerable. Like, yeah. They they could have maybe made it that fancy biometric is required and there's some of that argument if you really get into the minutia. But I think it's critical to understand that it on this case, in these cases, we have not seen any of those software vulnerabilities used. So, large largely the two attack vectors that I've seen, um, uh, for for people's awareness was, uh, just simply scraped password, right? And I think you you had a a good point here, um, that you noted in in the webinar. That someone may have reused a password from some other type of event registration, like all of us go to these events like IT Nation, Datacon, Kaseya Live, all of these events and people are registering online. And I could absolutely see the case where someone is using a password from their own environment to register for one of these websites. Like people should be savvy about this stuff. But when you said that, I was like, oh, that's actually a really, really smart, right? And that's, uh. That's that risk, right? You want ease of convenience. I want to remember all my passwords. But, uh, you know, at the end of the day, if you're using that same one over and over. This is a 10-year-old advice. But I think the difference that we're we're starting to finally come to terms as a channel is all this basic advice, nobody's really following it to the T that we need to, right? This is our our red alert situation. That it's time to say, all right, I know I needed to brush my teeth and floss every day, but now I have cavities. Yeah, we have cavities now as an industry. It's time to start brushing. Yeah. Yeah. And the a good analogy, brushing teeth or or health. Uh, they fit. You know, give me some complicated program or supplements to make me ripped. It's like, well, no, man, you just need to eat well and exercise. No, no, no, that's boring. What's the hack? Give me the idea, like what like what's the what's the shortcut, right? So. Start with the basics, not about the crazy complicated stuff that'll protect just. Audit your your environment, uh, have some basic controls and governance in place and that's probably 80% of the battle for you. Oh, yeah. Absolutely. If you could just unite behind some sort of, you know, industry accepted guidelines, even if you don't follow them at all. Like if you just went and united behind something and and made it part of your culture, even if it was once a quarter, once a year, you'd be better off than, you know, shooting from the hip and just assuming that it's all done. Yeah. Yeah. Um, so the other one that that I found interesting that you noted is. You saw some evidence, remember if it which which of the attacks this was, but you you saw some evidence that the the attackers were actually resident in the these environments for a while before they started launching payloads. Can you elaborate on that? Yeah, so we tend to see, uh, a case depending on which scenario it is. This most recent one with ransomware was very much smash and grab as we typically say. But other cases, they'll actually assess and say, is this worth smashing and grabbing? Or is there actually something worthwhile to maintain that persistent access? Um, that's both simple and sophisticated, right? We would do the same thing. If you were looking and saying, you know, is this HR computer, can I grab everybody's resumes and sell them? Like, no, that's not worth very much. You know, it's ransomware. But we're seeing people are actually making the economic decisions or the intellectual property-based decisions. To say, this computer or this network's worth a little bit more. So, we're seeing people say, I'm going to skip these ones. Um, some of it's risk-based. You know, we've actually seen some cases where they've taken a look at a product. They failed to disable it. That's that's some of the videos we didn't share that we we probably need to do a better job of sharing some of that. They actually took a look and said, oh, I tried to disable twice this antivirus. And they moved to the next host. That was a really cool video because you could actually see they were like, ah, this is just a a numbers game to us. I'm going to move on. That's that's like that outrunning the bear, right? They didn't outrun that hacker, they were just faster than the lowest hanging fruit next to them. Right. And some of the the attack devices, um, were interesting. Probably know about these, but it hadn't occurred to me. That you showed some videos of them snooping around on on point of sale devices. So, probably in a retail or a restaurant environment. You know, most of those things are still running Windows 7. Uh, luckily, luckily, most of them are hopefully upgraded from XP at this point. Uh, but that's not a typical attack vector that I think most people would think about. If you think about a device being hacked, it's usually a desktop or a laptop. Not necessarily, you know, the sales terminal sitting at the end of a bar, right? But as you guys noted, I didn't realize this and and sort of clarify that this is true. But those devices typically hold credit card numbers in memory before they're encrypted, is that right? Yeah, so a lot of the time if you think of this from a, so you're exactly right. First to confirm. But a lot of the time, when you scan that credit card, you have to read it. And immediately upon reading it, a lot of those time to be able to share or shuttle the data back and forth before it gets sent maybe over SSL or before it gets encrypted on disk. It's it's it's in actual memory and this is the same for ATMs. As well as point of sale machines. Almost all the tailored malware that go after these in memory is is the the new hotness. There's plenty of them that still try to just find places where the vendor didn't encrypt on disk. So, let's not make it seem like the the problem has to be even that elegant. Sometimes they can just get away with credit cards are stored on disk. But a lot of times they'll catch capture in transit, you know, the the POS machine has to be able to get this data to send it off to Amex or Visa or whoever's doing payment processing. And during that window, that's an opportunity for collection. Wow. Yeah. And to your point around the the software, not like a line of business software poorly written. That's unheard of, right? Oh, no, no, never. That never would this happen. Um, in some cases, it's just as simple as like, um, Microsoft didn't have the features. For instance, one of the biggest features they added for credential hardening came in Windows 8, 8.1 and afterwards. That it actually isolates processes into protected enclaves that mean like even if you have admin credentials. It's extremely hard to get some of this data. Those are things that when we we start joking about, nobody wants to upgrade to Windows 8 because right, they had that terrible metro interface. People didn't take note that it actually hardens credential grabbing as well. And software developers are even worse at using these new operating system features. So. Right. Yeah. Big problem whether supply chain or just local sys admin problem. Right. Yeah. Super important. Um, the other part is what I loved, like I think I maybe mentioned this earlier, but the the your guys's display of the tradecraft. I think it's really fascinating. Because even like I I don't know a ton about security. I know enough to be dangerous because I've been in the industry for a while. Uh, I used to run a professional services security department. So, you know, I I picked up some. So, I probably know more than most, but I, you know, I'm I'm an idiot for the most part. When it comes to like the details and of security operation. Um, so I I really found it fascinating seeing some of that that tradecraft stuff. And I was at a conference last week and they had Kevin Mitnick. As a one of the the the presenters and I found it absolutely fascinating. He it was interesting, he kind of ran it like a magic show where he he mentioned that he was a big fan of magic when he was a kid. So, the hacks and stuff he treated as if it were like a a magic show displaying an act. And it was really fascinating to watch the other side of the keyboard. And you always hear about attacks. And in our heads, I think we tend to think of, you know, guys like yourself or Russian actors and and China state and stuff like that. That are, uh, banging away on command line terminals and doing all kinds of crazy complicated stuff. And Mitnick's demonstration really dropped the veil on that for me where most of the stuff that he did was web-based. And it really highlighted to me that this stuff is not super complicated anymore because it's it's almost been sort of commercialized. There's user interfaces for a lot of these tools and that to me was scary as hell. Because it now it removes the barrier of entry for people to be able to do this stuff and it almost functions as a service. So, um, where I'm going with this is the the fact that you guys kind of exposed some of the tradecraft, I think was really interesting. Because I think it helps inform the channel about what this stuff looks like and how simple it it is and why you need to be a bit more paranoid than you probably are. So. I I mentioned in the beginning, right? I was started my life and started my career in the Air Force. And so, no matter where you are on the cyber side, you know, mechanical side or maybe you're actually a pilot, since everybody thinks everybody in the Air Force is a pilot anyways. Um. Everybody tends to joke about, uh, you know, to be an a pilot, you must be this top gun, top caliber. And there there is some truth behind that. But what's interesting, you you've even got the schematics, right, behind you on the actual demonstration. Or. On your wall there. And when I actually talked to a lot of people that started in the, you know, the early F-14 pilot days that ended up, you know, progressing to F-15, F-16 series. They said one thing that they were most proud about was they didn't make better pilots. They made easier to fly plane. And right now in in security, that's absolutely happening, especially on offensive security. It's they're making easier to hack. Like they're not making better hackers. They're making it so more junior people can conduct these. So, I think the more that our channel, especially as MSPs. That I look at. MSPs to me are the best ambassador for what I always call the iceberg, right? Fortune 500 is the tip of the iceberg, but most of like the US economy is ran by small businesses. MSPs could be shining that light, even doing some of these very easy basic hacking demonstrations to educate users. Because once you see that, right, once you see behind the Oz curtain and you say like, oh gosh, this is way easier. You now have like a visceral feeling of, okay, this isn't just the most sophisticated hackers who have to be going after me. This could be very basic hackers. Doing it, right, easier, uh, easier tools to use, more junior pilots, more people that could attack me. So, I I do think both Kevin's demonstration on the marketing side should even transcend not just educational material for MSPs. I really think MSPs are the perfect ambassador to take that message to the SMB market. Yeah, and you guys did some great work at that conference actually, you had a pre-day where you you showed some of those demos of being able to to to demo hacking of a of a device clients. Is that something that you guys do on a regular basis, is that only like a an online class or is that only available at at some of the conferences that that people attend? Yeah, so as of right now, we do, uh, two classes a year. They happen to align with the two biggest, you know, MSP shows, usually at, uh, you know, Connectwise's event as well as the Datacon event. That will teach like this year was 70 MSPs, instead of hearing death by PowerPoint, they spent four and a half hours with my team learning how to hack computers. And it was two-prong. Half the audience was, I realized that in order to defend, I have to know how attackers are getting in. How could you possibly defend if you don't know how they're attacking? The other half was, I have to get better at educating. I know there's this risk, but my customers aren't technical and I need to show them, not tell them. So, that was pretty exciting. So, yeah, we do it twice a year. Um, we're we're working on kind of commoditizing these, uh, videos just to give out for free, these instructional tutorials. Um, that's how you actually make a difference sadly in security, it's not by building the best technology, it's usually through educating first and then wrapping some good technology that nobody understands under the hood later. Yeah. Education is a good way to lead, right? I think, um. This is a good space for for information. Um, because I think, you know, as the the IT industry evolved is that, you know, everyone was treated like some grand master wizard around. You know, you you have some, uh, some knowledge that no one else holds. And and you know, I think anyone of us on the inside knows that's not necessarily true. Because I used to look at people who could who could, uh, take apart a car and put it back together. I have no idea. But they would consider me magical because I was able to reboot their machine and get it running again. Right. So, everyone sort of has their industry knowledge. But, uh, I think it's it's that education component is really important. Just around the the the exposure of how simple these things are. Uh, and you're right. Like Fortune 500 companies, people tend to think, oh, those are the guys that are going to get attacked. Because it's worthwhile, right? But, you know, they have hundreds of millions of dollars to spend on security. So, it's a hardened target. Whereas SMB, you know, the basics. Like we said, are not being done. So, those are the targets that are that if you collect 100 of them, they're probably end up being worth more than what you might get out of a larger company as well. So, the education becomes really critical. For for people to understand why a mom and pop shop of 10 employees at an accounting company is somehow going to be a a juicy target for an attacker. Yeah. Researchers from the Poneman Institute did one last year. A whole case study on the situation and they concluded that it makes no economic sense to go after somebody with two to five million dollars in annual security budget to extract $100,000 of fake payment when you could get 10 $10,000 payments in half the time. Like it just doesn't make sense for a I'm I can't believe I'm saying this. Hacker go to market strategy. You know. It is a business. I mean, let's let's be real about it. Cybercrime is a business, in fact, I I saw a stat, I don't know where this came from, so maybe I'm I'm pulling this out of the, but it sounds believable to me. That cybercrime now represents a larger economic pie than all other crime combined, right? And I think that that that sounds true because of the purely economic component to this. Like there's very, very low operating costs in cybercrime. And the returns are often extremely high multi. Whereas, you know, organizing physical crime takes a lot of time, takes a lot of people, takes a lot of involvement. So, this stuff, uh, I think the, you know, from a business perspective, the margins are extremely interesting when it comes to cybercrime. It's no wonder everyone's flocking to it, right? Yeah. It's it's 100% economic driven as well as risk driven. When you're having to physically rob a bank or physically dumpster dive or break into something. There is a real chance there will be physical security to throw you in handcuffs. I can do a whole lot from my house here in Maryland. To somewhere else in in the world. And I don't have a real risk of somebody kicking my door and locking, you know, putting me in handcuffs. Like maybe that comes after I get more and more, you know, pervasive. Yeah. But. Um. Did you I I don't know if you saw, Todd. That, uh, the Gancrab ransomware authors. They, uh, these were the ones that were hacking MSPs. They weren't actually doing the hacking. They were providing ransomware as a service. So, other people would install it and they would get a share, right? Revenue share. And they just announced, we actually saw during these MSP attacks, the actual hackers conducting. Moved from Gancrab to Sodi Nokobe. Um, ransomware. Because the Gancrab author said, we're good, we're retiring. We made enough money now and we don't want to go to jail. Like that's an actual thing that came out on their own hacker forum. Or, you know, uh, underground where they were selling these tools. So, that's just shows you, like this is real. That's incredible. I guess the the one element we haven't touched on. Is is, um, the user, uh, the the the major weak link in this. And we've we've kind of focused on on the technology and, you know, passwords to some degree. It's user related. Um, but I think maybe to touch on this, the risk around fishing, uh, pretty prevalent one as well. Uh, fishing attempts have gotten extremely sophisticated, like really, really detailed, good trade work the way that they're crafted. You can tell that they're spending a quite a bit of time on these to make them extremely believable. And I've heard from, uh, from several people where they've had, uh, someone email them pretending to be the CEO of a company. They send an email to the controller or the CFO saying, like, I'm in the middle of a deal. We need to close this vendor to to do this particular action. Uh, for for service and I need you to wire $10,000 to this account. Um, and I know in one instance, that happened multiple times. Where they said, no, sorry, it doesn't look like it came through and the person wired it again. Like those types of events are really, really scary. Because, you know, it's, uh, it's as simple as just, you know, crafting an email. Uh, maybe they have some insider insider info, they fire over an email and in that case, you know, they can they can pull the the the the jackpot. Bandit arm more than once, uh, so maybe, uh, if you can just touch a bit on. On your views on on fishing and and the user side of, uh. So, if I was to, uh, soapbox a little bit, the one thing I think that's usually done wrong. I think we forget that, you know, generally as humans, we should assume positive intent. I send to see some backlash on employees when they open the fishing email. I look at the exact opposite and say, every single employee. Whether they're my grandma who's still working as crazy as that sounds, um, if it's her behind the keyboard or somebody sophisticated in IT. Every single person is a sensor that could be vigilant if you use the right motivational words, right? The same reason that made people rally behind Martin Luther King. Is they believed in what they were what he was doing, right? Why can't you make your own employees believe in just a sexy spin on it? Every single person is an opportunity to hunt down a hacker and disrupt them for the company and celebrate it. Reward. You know, even if it's a a $50 Amazon gift card, that changes a culture. Um, I don't see anywhere close to enough of this. I do see the user education. I do see the measuring. I've even seen measuring used to fire people. I I 1000% disagree. If there's other issues, whatever. Do your thing. But, um. I tend to see people under use underutilized and, um, maybe that's just from the military experience. Where everybody is in a see something, say something culture because bad stuff happens when you don't. I think that needs to to really kind of turn on its head. So. I'll step down from the soapbox and get a little bit more, um, you know, pragmatic in regards to like what's the current state of things. Fishing is getting crazy good. Um, 2018. I had three of both my most sophisticated from a security standpoint of MSPs that I was aware of cases where the they had CEO, CFO type fraud. It wasn't wires though. It was actually, hey, I'm at a trade show or I'm at this event, please send me, I'm giving out a raffle card for iTunes gift cards. Can you please go buy iTunes gift cards, scratch them out so I can give out the code for this? And it worked. And sometimes it worked multiple times. Um. I know, Todd. The audience is going to hear us and they're going to say, all right, you're telling me, Kyle. Um, one thing I want to provide to you, Todd, at the end of this is there's an actual link to a US indictment recently where the FBI came out and there was actual hackers that were used, uh, to attack small businesses across kind of, uh, Pennsylvania area. And they actually detailed. Exactly how this happened in the indictment, how many times the successful fishing was used. And the actual amount that the attackers come to people to wire and what was crazy is it was dry cleaners. It was construction companies. It was accounting firms, law firms, everybody that our channel services and it actually gave an indictment and it said. Don't just hear it from my mouth, here's the actual printed 25-page report. Detailing every one of the companies redacted of the names, but how they operated. And that to me is the the the big eye opener. Of fishing is here to stay. We all know we need it. We need we know it's going to happen. But, uh, once I guess I I pivoted back to that user education. It's just nice to see what's actually happening to make people believe. Instead of just trust me, Todd. Yeah. People fish. Yeah, it's scary out there. Just trust me, I'll protect you, right? Like that, I think people people will be nervous about that type of, uh, uh, a pitch or push. Like that you're just trying to lean on fear to to drive a solution. I think people will be rightfully suspect of that, but if you provide some level of education. I think that that gives you more credibility and more ammo for us. And your your point on the the fishing and how that can be used in sort of negative ways. Employees. I I 100% agree, you know, I've had people ask me in the past. If, uh, you know, if people fail a fishing test, like should they be reprimanded? And I was like, absolutely not, they shouldn't, especially if you've not told them about it before, right? So, you can either launch a fishing an internal fishing campaign to test, uh, internally without telling people. Or you can tell them and say, this is going to come, like keep your eyes open, right? And there's different reasons to use different those those modalities, but, uh, to reprimand someone because they they opened a fishing campaign. I think it's completely wrong approach. Yeah, I think it could cripple culture. And I I've only got one or two cases where I've I know this have happened. And, uh, usually it it involves employee turnover. What what actually hurt your business more? That fishing email? Or the fact that some of your employees that were there for four years just quit because they didn't want to be worry about losing their income for their family next time they open the wrong email. Yeah, yeah, it's scary. Although, on the the the side of users. Every time this topic comes up, I share my favorite comic, probably have seen this. And it's a a boxing ring and the announcer is announcing the the boxing match and, uh, it says, in this corner, we've got firewalls, antivirus, all kinds of security devices all piled up in the corner. And in this corner, we have Dave. And like it's just like user on his T-shirt and he's standing the wrong direction looking out of the the boxing ring. It it to me it perfectly underlines the fact that, you know, you can amass all of this crazy technology you want. But the users are always going to be the single source of, uh, of, uh, your your threat. In fact, Mitnick's presentation. He talked about the fact that his company will 100% guarantee access that they can break into your company in in the white hat tests. Only if they're allowed to include social hacking the people. So, you can prevent the technological break-in, maybe, right? He's not 100% confident that he can get in that way. You include the users, he's 100% confident he can get in, right? And I think that that that puts a fine note on it right there. Yeah, I'm going to take the opportunity to just, I mean, even share, right, I I could have gone after any security problem, right? NSA was great time, my co-founders and I won the World Series of hacking. We were kind of at the top of our game. When we started Huntress. We chose detection and response instead of anything to do with prevention. Because we looked at it just as like massively unaddressed. So. And there's all kinds of stuff that need that type of. I mean, our whole entire preventative healthcare is based on early detection. And then responding before it kills you. Uh, I I don't know why we're such laggards when it comes to IT and security. To adopt that same methodology that medicine has had for 50 years. Yeah, agreed. Awesome. So, uh, maybe if you could just tell us a bit about the Huntress solution. And I think because a lot of people, initially when I heard about it, I was like, okay, another AV solution. And it's absolutely not. Uh, so, um, can you detail to people like if they have a pretty healthy security practice. Where Huntress would fit in the added element to that? Yeah, so I always recommend the first thing you have to have is you have to have your your your house buttoned up, right? Everybody wants to buy another product, but sometimes you can just better invest your money in the free things you're not already doing. Um, say you've got that or you're in the process, you've got that under control. We're a type of thing that's a complementary layer to your prevention. So, how it actually works, it goes on each computer. And its whole goal is we're not looking like if you look at the antivirus paradigm. I'm going to do a terrible job here and just say, hey, if you could summarize 25 years of antivirus efforts. They're asking the question, when a program runs, is it going to do something bad? Right. That that's super succinct. But that's effectively if you were to to explain it to my grandma. That's how you'd explain it. When a program runs, does it do something bad? We asked our ourselves the question, why are any of these programs running in the first place? So. Huntress in a nutshell is looking at this different layer of technology, different telemetry. No different than like healthcare, uh, saying, hey, we need to look at your blood, sometimes it's blood cell count, sometimes it's platelets. Two different ways to be able to say, are you sick? Our whole goal though is when something slips by and before it does something and spreads throughout your network. Can we find this quickly, whether it's a computer, a server, whatever, and then deliver you. And this is really the piece. Technology's exciting, great, but what's really more important is, all right, now you have an incident. How do you respond to it? So, that's really where our, you know, why we're so darn successful is we create you a ticket in your ticketing system. That says, here's exactly which computer it's compromised. Here's exactly the steps you need to take and you don't have to be a former NSA operator to be able to do it. Like that's that's the motivation that wakes us up every morning. Is how do you enable junior people to take these actions that make them look like security rock stars, especially to clients? Yeah, excellent. Okay. Uh, and in closing. Um, any call to action, obviously people encouraged them to check out Huntress Labs and, um, I'll link to the the demo this week. If people want to see some more detail on on the attacks that you guys, uh, uh, demonstrated there. Uh, any other call to action or or ask that you would you would suggest to the community? Yeah, it's, um. It really boils down to knowing thyself, right? And and this comes twofold, this is vendors and MSPs. This is a channel effort. So, um. You mentioned there's there's not been a whole lot of sharing of hacker tradecraft. If you know thyself and you're exposing these things. We need to share it. That's how we get better. That's how medical research works. We share sicknesses. Vendors when they're seeing these things, they need to come out and share. So, that would be my, uh, my call to action is let's know thyself and share what we know. Agreed, awesome. And if people wanted to reach out to you for more info or follow you on social, any channels that you're active that you would encourage people to reach out? Yeah, so I'm super active on Twitter. I'm Kyle Hanslovan or at Kyle Hanslovan. On LinkedIn, I'm there as Kyle Hanslovan as well. Our company is Huntress Labs, you can find us at Huntress.com. So, it's pretty simple to find us if you're interested on that side. Awesome. Really appreciate the info, fascinating conversation. And, uh, thanks for sharing with us, Kyle. Yeah, thanks, Todd, for having me, brother.