Using AI Safely At Your MSP - ERP142

This file was generated by Descript Todd Kane: Welcome back to another episode of the Evolved Radio podcast. Today we're talking about something every MSP is running into right now. How do you let your team actually use AI without turning your ticket data into someone else's training set? My guest today is Callen Sapien. Callen is currently building Synthrio, an AI infrastructure platform built specifically for MSPs and SMBs. Before that, he was chief strategy officer at MSP-Bots. Here's how I'm kind of framing this one, Callen. As we talked about, we spent a decade dragging clients to the cloud, then we spent a decade dragging them to security, and now our clients are dragging us into AI, whether or not we're ready for it. And I think we're missing a lot of opportunities, both on the client end as well as internally at the MSPs. So today, Callen and I are digging into how you actually give your whole team AI superpowers without blowing up your PII exposure in the process. Callen, welcome to the show Callen Sapien CEO Synthreo: Thank you very much, uh, for the warm welcome and having me. It, you, uh, opened up on everything that I do every day long, so Todd Kane: Perfect. Great guest. Callen Sapien CEO Synthreo: Yeah, there we go. We're aligned Todd Kane: Yeah. So, uh, I'll, I'll maybe reframe this again, like we talked about, um, and then I'll, I'll kind of set you up to, to, to, to give us your perspective on this 'cause I, I think you're particularly well-suited for this pr- this, having your experience with MSP Bots, integrations with PSA, now building a very, very AI-forward platform, so good exposure on kind of both ends of these, these, uh, the, these, uh, this topic. And the part that concerns me is I see sort of this multi-layered approach for people, especially service managers that I interact with a lot in the work that I do, and some of them are really leveraging AI in great ways. They're, uh… In particular, I've seen, uh, a lot of people dumping ticket data into a model and having it do some analysis and, uh, spotting some trends and giving it some insights, and that stuff is golden because doing a service review manually is time-consuming and requires a lot of sort of mental bandwidth and energy to kind of connect those dots. So being able to offload that to a model is super, super valuable. But rightfully, people are a bit nervous about where they give that data to, right? If you're gonna export all of your tickets, just dump it into a model and say, "Give me some insights," you gotta be pretty particular about how you're actually servicing that model, what information potentially goes in there. Simple things like the fact that, you know, I think you and I, you and I talked about this, is like a lot of people don't even recognize, like even if you have a paid model, there is a setting in your options to turn off whether or not it goes into training data. So just because you're paying doesn't necessarily mean that you're clean to give it kind of as much information as you want. So I recognize why people are cautious about this, but I see this as sort of a tale of two MSPs, the people that are leveraging AI in order to be able to do these things and hopefully doing it well, versus the people that are not able to take advantage of this because they're somewhat justifiably fearful about the best way to take, to go about it. So what's your, what's your take on how we go, we go about this in a safe fashion in order to maximize AI's potential inside the MSP? Callen Sapien CEO Synthreo: I, I, I-- That's a great, uh, overview and, and I break it down into really three areas. There's the technological, right? Did I uncheck that box? Uh, I have ZDR enabled, zero data retention, right? Uh, uh, we love to throw out acronyms in our space and Todd Kane: It's a new one for me. I like it Callen Sapien CEO Synthreo: and, and zero data retention is actually probably the most critical piece. I'll circle back to it because I don't wanna, like, do the ADHD thing where I, I, I go off and I say I'm gonna talk about three things and then, then jump down, which I almost did just now. So you've got your technical, which includes things like zero data retention, ZDR. That includes the, the, the actual checking of the settings, uh, and the-- and then the secure, and we've been talking about this for a long time, whether it's cloud security or even going back to the aughts where it's the right permission levels, uh, set up, right? The right identity and access policy. Uh, so there's the technological side. Then something that we, we overlook a lot is the contractual side. We, we have gotten used to these, like, giant SaaS agreements that we just, we just click, "Yep, I read it. Yep, I read it." And most courts have said, you know, it's not really a, a-- truly something that we would expect. It does limit liability, but, but you can't enforce every clause, right? Uh, but the AI, uh, terms are actually not as long as most SaaS terms, and they have some hooks in there. Uh, and then the last piece is cultural, right? Um, y- y-- we, we are-- You, you mentioned the, the push, push, now pull. We cannot suddenly, three years later, come in and say, "I've been handling your security for a really long time. I ignored AI. I just told you to ignore AI. I know you didn't ignore AI. Now I'm gonna shut everything down, and you can't use it," because people have been getting value. Even if that value has been a picture of my fridge and see what I'm gonna make today or what's in my fridge, they're getting value, and they haven't had the pain as much as they have in other areas that they haven't had the governance. And so one other thing that's kind of interesting on that last part is I was on with an MSP in Canada and, uh, the, the Canadian AI Privacy Security, uh, Act failed, right? It did not go through. But that doesn't mean the FIPS Act that, that exists around data, data retention and all of those things can't be enforced, right? GDPR counts, FIPS counts whether or not it was done by AI, SaaS or a human. And we, we-- when we're, when we-- we do need to bring that in as a governance piece, but, but those are the three kind of buckets. Going back to the zero data retention, because I do think that's the most important one, second is the contractual, but the, the, the zero data retention is critical because you can uncheck that box that says or check that box that says, "Do not train on my data." There is no box with any vendor right now that says, "Do not retain my data." And they propose in their contractual side that they have legitimate reasons for it, if you're r- abusing it, if they need to see if a model's misbehaving, if they have these other things. But every single model provider asks for or demands as part of the use of the product, the ability to keep data for as little as seven years, which is a long time, uh, and as much as, uh, indefinitely. When we look at like Google having shifted from don't be evil for the first 11 years and, and listening to, "I'm not gonna train on your data, I'm not gonna do these things," and still work within the, the confines, we know that if somebody has access and they keep access, it may not always be used for good things. I don't wanna be like a scary, Anthropic is using your data to destroy the world, but, but if they retain a copy of it, you know, things can change. But I, I-- Before I jump into the contractual, I'd love any thought that you have there. Todd Kane: Yeah, I guess like the biggest issue with that is gonna be breach, right? Like your data is retained somewhere and it's not necessarily-- 'cause originally like when the, the, when GPT was first sort of exploding and there was like these stories of like, uh, these guys at Samsung that loaded up a bunch of information in-into GPT and, you know, uh, nearly eliminated one of their patents because they'd, uh, in ess-in essence made the information public into one of the, one of the models. Or like your data kind of resurfacing as information for other people if it's a part of the training data. And I think a l- some of that is definitely, uh, not as much of a concern as it originally was, and it goes to sort of the f- like the first, uh, I would say edict of AI safety is don't use an unpaid model, right? Like the free model is there for training data, right? So if you're gonna give your data to anything, make sure it's a paid model at the very least and you're turning off some of this data. But to your point, like it's still there, right? Like there's some websites even for myself where I will tell it not to save my credit card data 'cause I'm like, "Yeah, it's not really that big of a company if it's not using Shopify or Amazon as a backend cart. Like do I really trust it?" Same idea. Like you're, you're essent-uh, essentially kind of giving this, uh, this information over for free forever. Uh, so yes, there's a risk of data breach. Some people think like, um, "I'm a small player, you know, my information, you know, would it… w-why would they possibly use my information for something," right? And it's not really that. It's like, okay, data exfil, they dumped a petabyte of information from, uh, AI model X. Your information's in there, right? So I think that's sort of the, the biggest risk is, is just data exfil from some scary event in the future and, you know, could be even sort of strangely benign. Like we had these conversations recently with Alex Dao on, on security where, uh, training models broke into other companies in order to gain access into stuff, and that's exactly how this could go down is like, you know, uh, Fable Six, uh, launches an attack because it needs more training data and starts breaking into all the other AI companies and s- and pilfers all of its information, right? Like, uh, you never know how this stuff's gonna come about. So I think that, that's a good point is just understand sort of where the data is resident and how it's being utilized right now. Callen Sapien CEO Synthreo: That's, that is a, uh… And that, and that even in-- So there's a large problem of people being incredibly empowered. Uh, uh, we, we, we have the ability to kinda host your own MCPs, right? And, and that may seem somewhat superfluous because there's a lot of MCPs out there, but the reason we have it is because of the data residency and the other pieces you're talking about. I've got a, a partner that needs data residency, uh, from regulatory reasons. Uh, they can use the official app in the store that exposes financial information and these other things, the official MCP for a marketplace, uh, through Claude. But Claude won't guarantee that that data doesn't on US servers or it's saved there and these other things. People need to understand what that flow of data is, and it's actually a tremendous amount of value that an MSP can provide. Uh, it's a huge value add, giving this visibility and building the services around it. And at the risk of sounding anti-security again, and I'm not anti-security, I've been a CISO, but at the risk of it, uh, the thing that I would like to actually point out related to this is you can't sell and protect AI the same way that we did endpoints and other pieces because it doesn't operate or act the same way. And in the last five years, six years, done a lot in security, but a lot of MSPs that thought they were gonna be MSSPs did not become MSSPs. They bought a SOC, they bought an MDR, they bought an outsourced helper to deliver these things. So they've outsourced the services side of this instead of standing it up because it costs a lot to stand up. the, the challenge with trying to sell AI that way is the management is where the margin is. The, the, the, the driving of the output, the guidance that, that's required to actually impact the P&L is where the management is and the guidance of, and that, that actually shifts into the contractual side. In the US, a lot of the court cases are up in the air still on whether something like attorney-client privilege, if someone's using AI, counts, right? Because for the same Todd Kane: Yeah. Callen Sapien CEO Synthreo: destroying your patent. Todd Kane: Mhm Callen Sapien CEO Synthreo: And the one thing through the cases that are pro and for and against and this and that, the one commonality through all of them when we look at what's becoming, uh, stare decisis, the, the let the, let the decision stand, is who had the governance and the stewardship of the data contractually, and was there anyone that could take this data for a legitimate or an illegitimate reason and shift it, uh, and use it? And, and that's where the contractual side comes in. Pretty much every main, main provider that's out there that we're going to use this data to, to, to, to learn something. Not train the model, but we're gonna learn something, and we can access it over the course of using it. And so that, that to me is a, a big area that we can get into. Todd Kane: Okay. Um, yeah, I think the, the governance piece, like that's, that's I, I think really interesting. I'll, I'll, uh, governance, I'm gonna write this down just so, 'cause we got-- we're gonna open up tons of ADHD threads here as we go, I'm sure. All right. Um, the, uh, I guess like the function around sort of the util-utilization for PSA data, like that, that to me, like I said, is like I, I wanna focus on this because I feel like there's an incredible amount of value for it, but it's such a fraught issue for doing it correctly. Um, so obviously, okay, you know, we're not using, um, a train, uh, uh, free model. Maybe we're, uh, handing it over to a paid model. Uh, I think understanding, uh, where the contractual I think lays into this is like being clear with the client that maybe you're using their data, right? Like if you're supporting a law firm and they have client information, uh, potentially in some of those tickets or the connections that you create to them, that creates a bit of a, a slippery slope as well. And maybe that's an interesting one as well, is how do we think about the PSAs that have AI capabilities built into them or other tools that are connected, right? Like that is, you know, you're not even necessarily giving something to a model, but the model is passing through, or sorry, the data is passing through another model in some fashion you don't necessarily have direct visibility to. So like, and I have to imagine this is incredibly common with most of the platforms creating some type of AI capability, whether or not it's a third-party module or just built into the system, right? So how do you think about that piece? Callen Sapien CEO Synthreo: Yeah, I think it's, it's, it's, it's a completely under-addressed problem that exists right now. Uh, it, it-- The, the, the, at least the use of AI within traditional SaaS products in a completely unmanaged way. Monday.com, for example, added a feature that basically put Lovable or Replit in it. You can build apps, build MCPs, connect things into your Monday as if you were building, building front ends. And that's not… That's-- It's very, very challenging to be able to get visibility into what people are doing in, in that area. And so I think that, uh, it, it's still kinda coming out in the wash. We're working on some ways of building visibility. There's some really cool apps out there doing that, but it's, it's, it's, it's, it's very heavyweight right now, and it, and it's across every single app. Uh, when you look at a standard MSP, they use 17 tools to deliver the, the output. Their customers use about 23 to 28 tools. All of them have AI in them now processing it, and most of them have updated their terms to make it an opt-out, not an opt-in. You're, you're, you're not… It's not like calling in used to be. I think it eventually will be, but, but it, but the data is being used, maybe not to improve things, but it's being pushed through a model that, that, that we don't know what it's gonna do with it. When we get back to kinda your core question, which is, you know, how are these things being used, and how are we able to do it in a, in a core way or in a, uh, in a strong way? One thing that, that I think we talk about a lot, and it's shifted with AI, people have asked me why I think that AI is so good at coding, right? This was a hard task that required a lot of brainpower and a lot of reasoning that exists. And the whole reason that it's great at coding is because open source software exists and, uh, Bitbucket and, uh, and, uh, and Stackbucket and Stackjack and all these, these things where I could go look at good code examples, bad code examples, and they train the models off of that. I think right now we're trying to protect the wrong things with our data, even the PIIs and the other, the other pieces. We have to do that because of compliancy. But the data Your MSP's data is not very different than this MSP's data. That lawyer's data is not very different from that lawyer's data. But what is different is your processes and the how. You're seeing more and more, uh, these attacks almost, uh, they-- I, I would actually say they're almost to a level of attack where, uh, Claude Design's a perfect example of this. Claude went and had a Figma integration. Claude learned how Figma works, and then they launched Claude Design. If someone doesn't put any PIIs in there, doesn't put any data, doesn't put anything in there around what the ticket actually contains, but they, they actually describe their process to the AI, now I've given the AI what makes me a 28% EBITDA MSP instead of a flat or an, a 5% EBITDA MSP. If I put that data on how the, the, the process of how this lawyer treats clients and, and runs through their process, now I've given away my secret sauce, and anybody who uses that model that it's improved itself on gets that. And I think that's one of the things that we really do have to, we have to talk through, and, uh, how do we protect the knowledge and the how? Because IP is, is, is shrinking as a, as a capability. Um, and that's something that I, that I think that is very interesting, and how do we protect that? And that actually goes back to that ZDR, right? If, if the model doesn't remember anything about what happened, then, then it can't learn. Todd Kane: Yep. Yeah. I, I almost feel like that one's unavoidable, and I almost feel like, uh, also, uh, like one of my favorite expressions that I've said forever since I've been consulting is, "Knowledge is easy, execution is hard." Uh, I don't know that there's a lot of proprietary information in the future that we can actually protect. Uh, uh, outside of things like the Coca-Cola recipe, right? Like sure, like there's some s- specifically proprietary information. But I think it's, it's a good example of what is really strange about the MSP industry that I call it a lot is we all fundamentally have the same business model and noob- no two businesses run even remotely similarly, right? Like, a- and this is despite the fact that like there's a huge incentive and a huge model for commoditization and forcing people to, to work the same way. But quite frankly, most organizations are a reflection of their owner, right? And anyone who has never noticed this, just like do a bit of a tour and think about the places that you've worked, the companies that you've looked at. They are always a reflection of the owner in this weird, weird way. So I think that's really only the, the only differentiation. So I don't know that like the information of, uh, how I do something is, is necessarily as sort of a secret sauce as maybe, maybe it would be, especially in the future. Because knowledge will just be so systemic and available, right? Yeah. So, um- Callen Sapien CEO Synthreo: I really like, I'm not a huge Sam Altman fan, but I do think he, he's been right on a lot of things, already built a trillion-dollar company. Uh, but, uh, and a trillion-dollar non-for-profit. Uh, but, uh, Todd Kane: If you're getting technical about it, yeah. Callen Sapien CEO Synthreo: you're getting technical, uh, he, he said that AI, and it's, it's turning into hands as well, but he said AI is an, is a utility, but it's intelligence. It, you, instead of electricity, instead of, instead of gas or, or these other things, it's intelligence. And you turn on the spigot, and companies that can spend a lot like they would on electricity can spend a lot on intelligence, and companies that can't will get a little intelligence. And I think it, it actually bodes to your point of the fact that it, it's going to be ubiquitous, and there's, there's a, a, a time that we can protect it, but it is gonna go away. It's just something to be aware of as we go through. When we actually talk about the practical application of using AI in ticket data, I think that there's still a lot of focus on personal productivity, and we haven't really s- got into the infrastructure phase. That's where we're seeing the, the labor, the FTE, and the P&L impacts is when we start to get into the actual infrastructure phase of installing an agent, of installing, uh, the data layer, the ontology layer, if you will, uh, within an o- an organization so that we can delegate and dispatch to, to the, to the agent to get it done. I actually think that human in the loop is becoming a more and more antiquated term, and it's almost a Luddite type term. It, it, it… Human in the loop, humans make mistakes as much or more. Uh, there's, there's a cool company in our space, uh, that got me turned on to, uh, AI, uh, before it was cool. I was still on the machine learning and data as well. I was at, at MSP-Bots, and we were trying to figure out AI and machine learning. But, uh, Mark Elaev, the fou- one of the founders with Matt over at, at, at Thread, uh, called me in March of 2022, and he's like, "We've got this thing that we got into the beta of called OpenAI, and we're, we're using it to, to, to guess and accurately put in time entry against a ticket." And I was like, "Oh, cool. How accurate are you?" Like, "84 or 85% accuracy." I was like, "That's really high." And people didn't adopt it because it was 84 or 85% accuracy attacks, uh, remember these tickets had zero, Todd Kane: Yeah, exactly. Callen Sapien CEO Synthreo: entry Todd Kane: an 84% improvement, yeah. Callen Sapien CEO Synthreo: It's, yeah, and, and the ones that did are 37% accurate. Todd Kane: Wow Callen Sapien CEO Synthreo: you know, the, the, the level of which I, I get into debate a lot and people argue with me on, and I understand accountability is the hard part, but, you know, with like self-driving cars, they are 100% more safe than human-driven cars. But that isn't good enough for a lot of people because can't hold the car accountable. But you can hold the company and the machine and all these other, just like if a seatbelt fails. Um, I think the, the, the answer should be it should be better than the human in a measurable way, and that's what I think when we're doing these things we should be looking at within the, within the instances Todd Kane: Yeah. Tired of fighting the MSP fires alone? The Ops Leader Pro group connects service delivery professionals who understand your daily challenges. From KPIs and workflows to career planning and team management, Ops Leader Pro has systems for you to use. Join operations leaders from successful MSPs who are sharing real solutions for managing client expectations, optimizing service delivery, and making your service delivery team as effective as possible. Ops Leader Pro, 'cause your service desk deserves more than just survival mode. Visit opsleader.co. That's O-P-S leader.co to apply to join the public community. Todd Kane: All right. So you mentioned, um, uh, sort of like the where the information is resident, and this I-- this is another layer that I think is really fascinating about this is, okay, so yes, you could use a cloud model for some of these things. Um, you could use a hosted like Azure container model for, for these things. Callen Sapien CEO Synthreo: Mhm. Todd Kane: your guys' approach is different. It's sort of like, uh, maybe a hybrid between the two, if I understand that correctly. Yeah, can sort of private containers, uh, uh, type approach. Um, the other one that I was thinking is like, well, maybe are we just gonna start doing a lot more, uh, ho- self-hosted and private models, right? And, uh, like that kind of makes some sense because I feel like, like despite the fact that the frontier keeps on pushing forward, like the frontier models from Anthropic, OpenAI, uh, you know, Kimi, they're, they're advancing and their capabilities are, are, uh, advancing incredibly. But they're also like some of, some of this is diminishing returns on where it is right now is perfectly fit for all the things that I continue to need, right? And I am like this close to moving to a private model, right? Probably the next, uh, the next Mac upgrade that I get, I'll go, I'll be fine, right? Like I won't necessarily need a frontier model for most of the things that I do. But, um, so I was thinking to myself, so like maybe most of these companies should just maybe look at self-hosted models and, and for, for some of this information. Won't be as fast, but you know, if you're just doing some data analysis and let it churn for an hour, then great, okay. It spits out something relevant. I was having a conversation with, uh, uh, John Dobbin. I'll give him a shout-out for, for the, the insight on this. Callen Sapien CEO Synthreo: Yeah Todd Kane: he scared the crap out of me on this. He's like, "So here's the thing. Most of the self-hosted models, like the open source models, there's, there's very little guardrails and containers built around them," right? Like the frontier models have a lot of, uh, capabilities around prompt injection that they will not do things that are, that are gonna be nefarious, right? But like, uh, what, what the, the sort of the, the idea that he, that he suggested is, say you have like a website where you can have clients submit tickets, right? And someone goes in and submits a malicious tickets that gets processed by a local AI that doesn't have good prompt injection, uh, management on it, and now all of a sudden it starts exfilling data based on that, that, that ticket submission. I was like, "Oh, crap. Okay. Private hosted models may not be the best play here," right? So it kind of scared me of like, there's, there's so many risk and reward trade-offs between the different approaches here, right? Callen Sapien CEO Synthreo: Mm-hmm. Um, no, it's spot on. There's, there's, there's really two challenges currently with the… Outside of the fact that you gotta probably spend truly to, to, to get something really meaningful, you probably spend eight to $10,000 just with component costs right now to, to, to get something that, that can run. And then, and then you've got energy costs and the, the other pieces because GPUs are heat. Uh, we, we, we, we went through this with the mining era of Todd Kane: Right Callen Sapien CEO Synthreo: right? Uh, and, and so we, we know that, you know, there's probably gonna be a lot more solar panels popping up, uh, for, for people in a little while, or geothermal. Uh, but the other piece to the local models, I, I do agree with John that, that, that there are… But you can, you could put the same type of guardrails, right? They do have lower context windows, right, which is a problem, uh, because you fill up pretty quick and you don't, you don't know what you can attack. Uh, but I do think that is, this is absolutely w- going to be part of the solution in the future. Uh, but one of the big challenges with these models right now is the hallucination rate. Uh, we see with locally hosted models, uh, in the 40s, 50% hallucination rate over the course of a conversation. And so you, you really need a lot of human intervention, both from the security side that John was talking about, but also from the business outcome side. The, the models are more prone to hallucination because the math isn't as strong, and the context is worse, and the memories are not as powerful. Todd Kane: So that it, Callen Sapien CEO Synthreo: really where Todd Kane: the, the, uh, let me dig on this 'cause like that, that is a function of smaller context window by nature because of limited infrastructure and something to do with the models themselves or sort of the parameters that are built around it, or just the context window? Callen Sapien CEO Synthreo: It's, it's, it's a combination of, of kind of all the pieces. The smaller models with the lower parameters have Todd Kane: Yep. Callen Sapien CEO Synthreo: capacity, Todd Kane: Yep Callen Sapien CEO Synthreo: what you said. Uh, you know, to put another way, it's like, you know, using, uh, less of your brain, right? A lot of those models were designed to have, for example, like 32 billion parameters that they can take on at a given time. We neuter them down to seven to eight billion parameters. So you've got a prediction engine that is trying to predict something with a quarter of the context and the memory capabilities that it has. So it does work great for, for certain repetitive tasks, QA, processing of, of, of almost a deterministic behavior. Uh, but where it, where it really starts to, to fall down is when we, when we start getting into, uh, you know, adding in multi-step processes, right? Uh, multi-step is really the big part that, that gets hit, and that's where we can add in different workflows and loops and, and these other things, but it, it definitely… It's actually one of the main reasons why John's security breakdown happens as well, because how you overwhelm and, and overcome the guardrails largely with these models is either by injecting a, a, a, a, a mistruth and calling it on it, and then, and then now it's like, "Oh, I was wrong on this. I lied, so now I'm gonna relax my other things because, because this is here." Or then context bombing. So I'm gonna drop in 50% of my context window because now it's the guardrails were maybe 30% of every turn in the conversation for the first half of this conversation, now they're 18%. So as that math is, is, is expanding, the, those, those guardrails just have less weight in the conversation. There's still more weight than the, than the, than the attack, but they have less weight overall, and then it's just about consistency in attacking it. Todd Kane: Especially given this… Callen Sapien CEO Synthreo: bel- Todd Kane: Go ahead. Callen Sapien CEO Synthreo: No, no, go ahead Todd Kane: Well, I, I think it's, that's relevant specifically for sort of the use case that I'm advocating for, in, in particular is ticket analysis. So you, Callen Sapien CEO Synthreo: Yes Todd Kane: say, even s- three months of data, that's still a lot of data to dump into, uh, like, that's certainly not gonna fit in a 200,000K, uh, window, right? Or 200K window. Uh, e- even a million, I think, like, it might fill up a fair bit, but, you know, you gotta be sensitive to that, right? Callen Sapien CEO Synthreo: We, we, um, we're handling this in a, in a way where we're tr- what we're trying to do is, is not offload the, the model itself, but offload the compute, because that will reduce some of the, the, the cost. And so what we do is we try and keep the thinking and try and keep the actions local, because that's, that's more controllable and definable. Uh, and, and then batching. I, I think batching is highly underutilized by practitioners, and this is where the Ticketdata side can come in very well. If I can, if I can run through a recognition pattern, whether that's machine learning or whether that's a SLM or whether that's just a model, and say, and categorize these things, then I can send them through in batches, and then I can reconnect. Uh, Gary Vee just, uh, said something a couple days ago that I really, really liked. Um, and, uh, I've followed him for a long time. I didn't expect him to become an AI guru, but, you know, the guy's a guru in so many other areas. But he calls it the 15/85 rule, and he, and he started it with his, with his, like, leadership team. He wants to be involved in the first 15% of the planning and strategy, let them go off and do 80%, and then back there for the 5%. And I think that when we apply it to AI, it's very similar. Let me be part of the planning and the strategy, go do your work, uh, and then, and then come back. And we could actually probably localize those much better than, than otherwise. I think Zofik was on, was on that path. I d- you know, I haven't talked as much to Lee since their acquisition, but, but I think that was really what he was trying to solve for was because we had people running these local models, and so he, he took that on th- uh, upon theirselves to do it. Todd Kane: It's funny 'cause like this morning actually in my group coaching session, I, I, I talk about this model a lot, and it's 10 / 80 / 10. So, you know, he's got the, Callen Sapien CEO Synthreo: Yeah. Todd Kane: the 50 and 80 and five. Uh, so yeah, my model is, is 10 / 80 / 10. I don't remember originally where I got this from, but it's, it's an industry, uh, sort of leverage productivity model, especially for delegation. This is where like I originally started using this in, uh, uh, avoiding executive swoop and poop, right? Because the, the executive would sort of get in there and mess around with stuff, and they didn't set up the parameters for what the delegation was or what they wanted, so they would make a mess and then sort of show up at random times and not pr-produce any value. So I would tell people like, "Okay, use the 10 / 80 / 10 model for delegation, 10% l- front loading expectations, delivery, what you wanna protect against, then let them go and do 80% of the work and then 10% cleanup on the end." So exactly the same, same model, slightly different, different numbers. But where I think this is particularly useful is delegation. Like that's where I talk about this framework a lot, right? And what-- I think what you're pointing to is we need to start thinking more about agents as independent, sort of autonomous employees doing particular work. And it's weird because like it's not necessarily contiguous, but I think we still wanna think about it in a 10 / 80 / 10, uh, delegation exercise of like, "Here's the parameters. This is what I want. This is the expectations. Don't do this, do this," all of those things. Let it churn for 80% and then inspect it on the back end, right? And like I'm sure you've heard this term meat proxy, um, which is, uh, uh, people that just take whatever comes from AI and turns around and, and spits it out. Uh, and this is where like work slop is coming from, right? The, the whole idea of, uh, you know, I, I, I was gonna send an email. I had ChatG- ChatGPT punch it up to like a three-page or a three-paragraph summary with lots of information and data. Uh, I emailed that over to somebody else, and then that person uses their AI agent to digest it into like the f- the, the much smaller version of this, right? So it, it's just meat proxy to meat proxy, and we're kinda uselessly using AI in, in between. Uh, so now I'm just ranting. But the, I think the, the, the, the, the way of, uh, using delegation frameworks for AI and the similarities that we should be applying in general, uh, sort of collaboration and management is very on par, right? Callen Sapien CEO Synthreo: I, I, it, it actually uncovers a very human problem that we're dealing with right now, and, and, uh, it's-- there, there, and there's two pieces to it. We have looked at leadership as a… And I mean leadership and managerial, because you don't have to have reports to be a leader. Todd Kane: Mhm Callen Sapien CEO Synthreo: looked at that as there's a class of people that have earned that right, and we're going to invest in those people because we think that they would be good leaders of humans. Uh, in order to be a powerful AI user and actually use it in a way that is going to impact in a positive way your, your clients, yourself or, or your mission, you have to be a good delegator. And we've gatekept these things to the, to the, to the upper class of this. And including, we've told people like, "Well, I don't think you're cut out for leadership." Well, now you have to be a leader. And I think it's going to-- And we-- I've actually seen a rise in more like HR consultants going into leadership management, teaching these things. It's also, uh, been a challenge for people that have, uh, globalized talent because we ma- we, we globalize the talent and we, we, we offload, workflows and intellect to individuals to follow a process with no deviations. And, and so, you know, comp- countries like the Philippines, uh, certain areas of India, uh, Vietnam, Indonesia, uh, Pa- Pakistan, these areas have, have built, you know, farms and, and, and, and cities mul- you know, um, uh, of, of people that, that are going to follow these things, and if they don't follow it every step, they're fired. And we're asking them to adopt the management capabilities and the creativity that we've trained out of them for 45 years as we've outsourced. And it's, it, it-- and a lot of MSPs use globalized talent, and it's something that, that we're going to really have to address in a very short amount of time. Uh, otherwise, the token cost is not gonna be worth it, in my opinion. Todd Kane: Yeah, and that's the other issue that I think we're facing here. Maybe, uh, slightly off track, but, you know, tangen-tangentially related is, um, the over-indexing on sort of the capabilities of AI and where they can actually fit and, you know, the token cost for those. People are now going, uh, you know, uh, it was sort of the last six months, eight months was token maxing, token maxing, we're gonna do amazing things, 20% layoffs, and then they're like, "Holy crap, why are our costs up 70%," right? And so they were token maxing. Now it costs more than the people that they had sort of doing jobs before, and maybe not achieving the same ends that they thought coming into it, right? So I think this will be an interesting trend to watch, but I, I think what you're, you're alluding to is more of sort of a business process thing of like, how do we, how do we compartmentalize and really understand the work that we're doing, and how do we intelligently hand it off to workers or digital workers in a way that actually produces the results that we want? I think that's gonna be a really fascinating exercise going forward, right? Callen Sapien CEO Synthreo: Yeah, I mean, that's i- in the-- We're, we're, we're coming up on a year officially really in market, uh, in, in November. And in that year, we've had really two pretty dramatic shifts onto my company's identity and, and the problems that we solve. There, it's the same mission, but, uh, technology and the landscape has shifted so much that, that we've realized that we need to be, uh, the optim- the, like, the, the greatest optimizer of delivering those outcomes for the business class. So what I, how I would, how I would liken that is basically like Cursor for delivering results with AI, Uh, and, and Cursor does deliver results for AI. It's just specifically around coding. And so we've, we've built it, and it's to the point where I think you mentioned the diminishing… No, uh, I don't think. You did mention diminishing returns. And we can get out of a, a, an open weight model like Inkling similar to Fable 5.0 results, uh, with, with our Wingtip engine, right? And, and the cost of that is a 10th or, or a 40th, or if you're willing to use a GLM, maybe a 100th to get, to get 90, 95% of the output. The other piece is that infrastructure piece, and I think it's a, the, probably the biggest opportunity with AI, though, for the MSP market, is we gave up infrastructure when we, when we went to 365. You've been in the industry long enough. I-- fortunately, cybersecurity came quick enough, but I remember talks of layoffs, panic in 2016, 2017, as we moved from, "We own all the infrastructure. We're building data centers," to, "We're getting 20%, 10%, 8% of what Microsoft will give us, and, and now we've got to survive on this." Then security came in, it saved us. Uh, and but we're now back in the spot where we get to manage and maintain the infrastructure again, and we need the right people that can figure out how to do those tasks. But then we need to figure out, like, where does that live? What does this look like? And how does that scale? And it's a huge opportunity, though, that, that exists, uh, in, in the space because I think we can get back to the 60, 70% margins and have partners that are absolutely willing to pay it because they're getting so much value. But it requires exactly what you said, the right delegation, the right harnesses, and the right, uh, infrastructure to, to do that, Todd Kane: Yeah, and that's the biggest thing here is, is Callen Sapien CEO Synthreo: at. Todd Kane: yeah, there's always where there's complexity, there's margin, right? And we're now getting to the space where like y- you can't just sort of, "Oh, just go get ChatGPT," right? I think that there's sort of like layers of where MSPs are going wrong in their AI strategy, and I've been ranting about this as like that's why I sort of frame it as like pulling their clients to cloud, pulling their clients s- to security. Like, where are you guys on AI? You're not leading, right? Like there's so much shadow AI going on, it's insane. So first there's just the governance aspect of that I think is a massive opportunity. Um, but I think like the, the work that you guys are doing is a huge opportunity of actually stepping into, you know, the industry term Pax8 is, is sort of, uh, advocating for is, um, is, uh, managed intelligence provider, right? And right now, a lot of people are just like, "Here, let me set you up with Copilot." And it's like, well, okay, like that's easy, but like that's not gonna produce a ton of value for people. I honest- I kind of firmly believe this is why we're seeing so many damning stats in the industry of AI initiatives failing in enterprise businesses, 'cause they're just like, "Here you go. Here's AI. You're gonna do great," right? Like, "Here you go, Tommy." And like nothing comes out of that 'cause they don't know what to do with it, right? Like, no one's trained them. They don't know how to use those models beyond just asking for recipes and stuff, right? So the, of course, that's not gonna produce anything. The second layer would be, you know, "Hey, let me do consulting with you. We'll s- we'll set up, uh, like so a Claude account or an enterprise GPT account and get everyone sort of rolled out on a maybe a arguably a better model," right? And maybe this is the way to go about it. And now it's just consulting with reselling kind of a SaaS product. But I think what you guys are doing is, is much more the hybrid of like, "I'm gonna give you a console that allows you to do like open weight routing on to different models and give you the privacy protection and, and the portability to multiple models." And I think this is massive 'cause of your point, like when we were, we were talking before, maybe you can d- you can dialogue on this story a little bit of like you had a client, like they were paying 50,000 a month in self-hosted Claude plus Azure, and you guys slashed that to like just over 10,000 a month with, uh, with, uh, the, the platform that you guys are doing. So I think this is the value that you can bring to people is like, not only am I giving you the capabilities for a fraction of the cost, but I'm actually able to manage it for you rather than just sort of like, "Okay, hang on, let me try and log in as an admin to your, to y- to your, your, uh, your profile," right? Maybe you can expand on sort of how you guys are doing that. Callen Sapien CEO Synthreo: Yeah, I, I think, you know, the most forward-looking, that we end up encountering are dealing with actually some of the initial 365 problems, right? The federated access into all these portals. We-- There was a partner of ours that, uh, they were wor- they were using N8N, and they, they were around with my co-founder, and they're like, "You know, the, my first thing I do every day is log into 38 N8N portals and see if everything's okay." Todd Kane: Ugh Callen Sapien CEO Synthreo: It's like, that is, that is not… I mean, like none of us got into… I mean, maybe somebody got into it to do that type Todd Kane: Some, some automation nut, yeah Callen Sapien CEO Synthreo: Yeah, yeah. There, there are some you know, that, that exist in the world. Uh, but, uh, and I, I don't want to yuck anybody's yum. Uh, but, uh, but at the same time, that isn't sustainable, right? And, and you're gonna miss something, and something's gonna crash, and, and, and we're, you're gonna miss it. And, and so for us, we, we do look to, to, to bring that You said it, manageability, guidability, governance over, over the piece. W- w- it's interesting 'cause we also had a, a recent, uh, partner that is a Claude network partner. They're, they're reselling… Not really reselling Claude, but they're reselling a service around Claude. And they had a, they had a partner that said, "You know, I'm, I'm, I'm over." You know, we started this conversation by saying, "Hey, a new model might have dropped," uh, and, and because it all sucks now. And, uh, and they, they said, "You know, I'm tired of this, this rat race. I need to go to something that's agnostic because if I pick a winner right now, I'm gonna be jumping every six months to try and get the most out of this stuff." And they said, "Okay, so what do we do with the last eight months that we've been in Claude? Do, how do I export this work? Where do I put my artifacts?" And, you know, the MSP's like, "I've, I don't know. We'll, we'll export your chat history that we can, and we'll, you know, see what we can do about your, your, uh, your projects." But the portability issue and the, the, the other pieces that people are just starting to understand. You know, I- we call it internally the Claude wall, uh, but it could be my chat tool wall. It could be Copilot. It could be ChatGPT. It could be, you know, uh, devs.ai. It could be HATS. It could be anybody. When we hit that personal productivity wall that you alluded to, it, you just, you can't get anything else out of it, and now your two options are, do I go the old way and build an app, and I manage that app and the infrastructure, or do I figure out how to use the modern stuff? And that's, that's where we come in and some of the others come in to deliver agents, uh, without having to host the infrastructure. W- build MCPs without having to spin something up. We had a partner the other day that said that one of their clients connected their banking software through our Excel app, uh, using a, a, an MCP they found on GitHub because Claude told them to, to, to go do that. And it had two stars and no comments, Todd Kane: This sounds scary as hell. Callen Sapien CEO Synthreo: banking through Todd Kane: Yeah Callen Sapien CEO Synthreo: And so, like, those are areas where governance makes sense, and they still need to do that. So build the MCP. Uh, offer it. Even if you don't use 3.0, that's, that's… For me, when we build an MCP, it's universal, right? If someone is, is dedicated, if they're a maxer and they're using Ultra and they're going, you know, in Groq, and they're, you know, they're taking their toket subs- token subsidies to the next level, don't necessarily want them running on my, my, my, on, on that area, but we still need to give MSPs a way of managing that. So that's when we, we build MCPs, governance policies, observability. And I think all of the things that we've, we managed before that I hope people don't take this pejoratively or, or, or as a wagging finger, but that we've kind of went away from over the last five or six years as we've resold a lot of SaaS and a lot of security products without wrapping services around it, uh, is, uh, is it needs to be relearned. And one thing that's a little scary to me is we've had so much M&A in the last 10 years that I wonder how many, like, experienced service delivery people exist out there. You mentioned John Da- Dobbin. I, like, that guy's worth his weight in gold. He has 24 years of delivering service. If I was an MSP, I would be grabbing onto him and saying, "Okay, should we sell AI, and how should we build that offering?" And find that service manager that is so passionate and had been in that thing, and have them reteach the entire company on how to deliver services at scale, and then base your AI policy on that. That's, it's get back to the basics, which sounds silly, but it's, it, it's, it, uh, the, the idioms are, are there for a reason, you Todd Kane: Yeah, there's so much truth in that. Like, I find everything in business and somewhat, uh, I guess a large degree in life is you, you getting back to basics. It's just we've gotten so far away from the simple things. Like, I always tie it back to, uh, you know, the John Wooden a- approach, very famous basketball coaches. We're gonna start with tying our shoes, and everyone's like, "What the hell are we doing?" It's like, eh, you know, tie your shoes correctly, you don't get blisters, and then you're not out of the game, and then a higher point scorer s- is in the game, we win, right? Like, all of this stuff from, starts from absolute basics. So I think there's a lot of Callen Sapien CEO Synthreo: Угу Todd Kane: truth in that. Uh, and I am right behind you on, uh, you know, let's get passionate about this. Let's start building an AI practice because, as I said, your clients are doing this without you, and they're doing it poorly. So they absolutely need your help. Whether or not they've explicitly asked you or not, they need your help. So no, this has been great. Uh, really appreciate you coming on, Callan. Um, if, uh, people wanna reach out to you and know a bit more about what you guys are building in the platform, where should they find you? Callen Sapien CEO Synthreo: Uh, either Synthrio, uh, .ai or you can email me directly at callans@synthrio.ai, and I'm happy to, happy to jump on. I've got a, I've got im- I've got-- My link is public, so I love this industry, and thank you so much for having me because this is a mission for us, right? We, we-- When we were building this company, we, we saw this problem coming, and we actually thought, "What if we were to build…" Because my co-founder built a successful MSP and sold it and exited. He's like, "What if we built an AI-native MSP and just went and sold?" And w- both of us realized how much that this community has given us. I don't care. When I say community, I mean all of it, ConnectWise, Autotask, Halo, all the, the individual ones. And we, we just want to help get back to those basics. And, and so yeah, please reach out to me, even if you're not a partner or interested in being a partner right now. W-w-we want to help Todd Kane: Cool. So I'll link to, uh, everything in the show notes, but, uh, it's been awesome. Thanks, Calum Callen Sapien CEO Synthreo: Thank you very much