[00:00:00] Speaker A: Foreign.
[00:00:11] Speaker B: And now we're joined by Matt Olson from Crestron to continue the conversation around AI in av, but from a slightly different angle. See, we want to look at what happens as that intelligence becomes a bigger part of the room itself. How does AI change the way we design collaboration spaces, classrooms and AV systems? Where should that intelligence live and how much should happen automatically?
Plus, what does an AV system need today to support AI capabilities coming tomorrow?
Matt, thank you so much for being with us today.
[00:00:44] Speaker C: Happy to be here, thanks for having me.
[00:00:46] Speaker B: So Matt, when Crestron talks about an AV system being AI ready, what does that really mean from a system design perspective?
[00:00:55] Speaker C: That's actually a really big question and it kind of touches on, let's see, four points of AED we're looking at, from pre trained models with machine vision with our cameras, to thinking about telemetry reporting and all the possible AI triggered actions, transcription and summarization. That's really big. Obviously that's one of the big workflow enhancements that we're seeing in AI today is just having it transcribe a meeting, translate a meeting, close caption a meeting, and summarize a meeting after the fact. And of course those tools are only as good as the content that we feed them, so proper design up front really helps.
And then of course there's horsepower for the compute engine makes a big difference if we've got the horsepower to embrace the edge AI processing that the platforms are tending to shift back to the edge at this point. So. So having a robust compute with the horsepower to handle those things as they come down the road over the forthcoming years is helpful.
And finally, orgs want things to last. Organizations generally want things to last. So having a system that is ready for those changes as they come down the road, whether that means having a robust compute at the heart of the system as a foundation, and peripherals that can be either be upgraded via firmware or in a modular physical way is helpful down the road. So those are kind of the main things that we're thinking about in terms of being AI ready. There are certain things that we do AI today, but looking forward and how are we going to be best prepared for the things that are coming that we don't even know about yet?
[00:02:34] Speaker B: So as these AI systems and to some extent automation systems become more intelligent, where should the processing actually be located? Right. We've all heard about the OpenAI anthropic and maybe we should just run it with the cloud controlling everything. We've also heard of on prem deployments with edge computing and maybe hardware AI baked into the individual device and peripherals themselves. So where are you kind of approaching this from?
[00:03:02] Speaker C: Kind of all of the above. And again, I hate to use the answer of it depends, but so much in our industry depends on the application, on the use case, on the workflows, even down to the individual, what's going to best suit their needs.
What we are seeing though is some trends. For instance dsp, right, where we're kind of accustomed to this industry, to thinking of DSP as this monolithic thing that handles all of the audio.
As we can observe in the industry, things are changing a little bit. From processing in the cloud with teams and Zoom to having microphones to handle some of these tasks themselves without needing the centralized DSPs. So it's just kind of an example of ways that the processing is evolving and moving. I don't think there's a fixed answer today. It's on the move and our goal is to be ready for it as it moves and be a part of that evolution.
[00:03:57] Speaker B: Just to push back a little bit on that.
So there's not kind of a line in the sand that says, hey, your AI should be on prem versus it should be in the cloud. Is there kind of like one question you can ask that kind of says, well yeah, you should lean towards tokens in the cloud and LLMs in that respect. Versus, versus some amalgamation on premises where you can run things a little bit differently.
[00:04:22] Speaker C: It's kind of the age old question of cloud versus on prem, really. Right. We've had Fusion for quite some time. Starters really became Fusion, became Fusion Cloud as an offering. What should I choose? Should I choose on Prem or in the cloud?
The answer is do you want to manage servers? Do you want to handle all these things? And of course there's data lakes and security and things like that as well, which are hugely important.
So processing on prem can be helpful, Right. To avoid exposure to the Internet and having everything managed on prem, however, that's a pretty big lift, right? So what you save in tokens, you expend in energy and cost of maintaining this infrastructure on premise.
So again, I would say it depends on the priorities of the organization.
[00:05:10] Speaker B: Well, let's move towards kind of like the meeting platform because we find that a lot of decisions are made downwind of what the meeting platform du jour, what the organization is comfortable with, both from a financial spend and also because sweet seat folks know how to use it and don't want to change. So if we look at things like Microsoft Teams and Zoom, they're adding their own AI capabilities very quickly. So how should customers think about the intelligence coming from the meeting platform versus the intelligence built into the AV system itself?
[00:05:41] Speaker C: I'm not going to repeat myself and say it depends, but a little bit. Right. There's some variability here.
Um, yes. So teams and Zoom are adding a lot of great features. Like in the education space, for instance, it can be helpful to simulcast multiple cameras simultaneously so that the far end participant can pick what they want to look at while they're participating. So, you know, whereas in the classroom I can turn my head and look in different directions, if I'm participating in remote class synchronously, I can select which camera I'm looking at. That, of course, requires the cameras to be all feeding into the compute and being sent to Zoom and having Zoom manage some of this.
On the flip side, if we want to present a more curated experience to the far end, maybe it's a high stakes boardroom or something like that, where we want to have a little bit more deliberate control of what goes out and control our production style switch locally, I can gain some benefits there, right. I have more control over what the shots are and how they're framed and things like that.
B, the response time is going to be that much quicker because it doesn't have to go to the cloud and back and that latency gets saved. So again, we embrace both. We built our computes to ingest multiple cameras if we want to take advantage of some of those in platform features, or we offer systems like Automate VX to have a full production style automated switch happen before it even gets ingested into the compute. So really depends on the workflow.
[00:07:09] Speaker B: Do you foresee there being kind of a. I don't say collision course, but the too many cooks in the kitchen, where we have too many intelligence systems or quasi intelligence systems in the room trying to make decisions at the same time. And how is that? Do you foresee that being mitigated?
[00:07:26] Speaker C: I think we've already seen some of that in some of the more traditional processes that we've run into, like AEC for instance, Acoustic echo cancellation. You can have it at the dsp, you can have it at the micron, you can have it in the platform, but we don't want to have it in more than one place at once because that causes problems. We want to set one as the primary, let it take care of things. And I think the same is true with all of these other AI features as well. We're going to determine which one we want to use thankfully, most of these AI features as they roll out are an opt in sort of thing.
So they're not just foisting themselves upon the ad system that I've already built. It's something that I can take advantage of or not take advantage of and make an intentional choice about how we use those things. Could the possibility exist if we're not intentional about it? Sure. But if we remain intentional about it, then I don't see that as an impending risk today.
[00:08:20] Speaker B: So, Matt, before we get into the tech, and I know we both love to talk tech, why are we even talking about AV and AI? Why?
[00:08:30] Speaker C: It's a great question. And I think we need to understand what even is AV and what we've been doing with it for the past several decades at the end of the day is to connect people, humans together.
We started in a room and we needed great sound, we needed great image. And it got better and better over time. Great microphones got better and better over time to enhance that interim experience so that we could connect at large in spaces.
Today we're talking about connecting not only those folks in the space, those things are still very important, but connecting people afar and approximating that in person experience with a remote experience.
So that's, that's why we're doing what we're doing.
And what does AI bring to bear? It's just another set of tools, right? It's just another way to bring more of those great connected experiences to more people and more of the time. That's really the point of all of this.
[00:09:25] Speaker B: You mentioned cameras earlier, and I kind of want to jump back to that because we have a lot of inquiries about cameras. And instead of simply asking a camera to follow a person speaking, can AI begin making some more, shall we say, sophisticated production decisions about the remote participant, about what the remote participant should actually see?
[00:09:43] Speaker C: Yeah, and I think that's a really important facet of having a great far end experience is following the conversation, right? Following the natural action as if you're in the room. Because that's really all we're trying to do in AV is approximate that in person human connection.
And a good portion of that is already possible today with a combination of AI and some traditional processing. For instance, Automate VX system uses audio positional data. Right. We can understand where someone is speaking in a room and point a camera in that direction.
We can then leverage some visual AI into that mix. And if the person turns their head and faces a different direction, there's a camera in a more favorable position.
I can switch to that other camera. Using AI, right.
We can detect when a speaker is in a presenter zone and then have a presenter tracker camera follow them. So I've used a traditional processing to figure out that someone is in that new zone. But now I'm using pre trained models with AI to follow that person through their presentation zone and follow them like a human, like operator. So there's a little bit of combination of old and new there or current and new, but some of those features are available today.
[00:10:57] Speaker B: That brings up another good point. Is that how soon or how close do you think we are to having a virtual video director? Right. A virtual TD running a conference room or a classroom or even a town hall?
[00:11:09] Speaker C: You know, again, I think we're pretty good in a lot of respects today with something like Automate VX when you pair it with our positional data and following the action based on audio telemetry, where people are speaking and calling the appropriate camera for that shot and following the action that way. And if two people are speaking, I can have a conversation note automatically happen without intervention. If someone starts speaking from the presenter zone automatically take that a group of people follow the group, you know, frame the group as they're walking around in the presenter zone. So to that extent, I think that a lot of it is possible right now, today.
It does require some human element to set that up to make that successful. Maybe there's a future where you put cameras in favorable locations and microphones in favorable locations and the system just kind of figures it out.
That's a possibility.
Plausible.
[00:12:01] Speaker B: Well, heck, we're already doing in our living rooms, right? We set our surround speakers up and we hit a button and we say calibrate. So I can totally see a world where, where that would be done as well. And with given the variances of living rooms and the variances of rooms that are used for conferences or just meeting rooms, you know, that seems like a kind of an easy fit.
And speaking of that kind of auto calibration, we obviously would be remiss if we didn't talk more about audio and you mentioned aec.
But what other audio things should folks be taking into consideration when they're looking to put kind of smart or AI enhanced microphones and processing in their rooms?
[00:12:39] Speaker C: Yeah, this is, this is one of those areas where
[00:12:43] Speaker B: it depends,
[00:12:46] Speaker C: it depends on a level of tolerance for. Good enough, I would say.
[00:12:53] Speaker B: Interesting.
[00:12:54] Speaker C: Yeah, I mean we've seen the, you know, when coming out of the pandemic when we had to have every room, video and audio enabled, it was just get stuff in there, make it good enough so we can make a call from every single space. Now I think there's some reflection on that and wondering if maybe there are some better ways. You know, video bars have come a long way. There's some great solutions, but they're not the right fit for every single space.
Right. Sometimes we really need to design a system around what's good for the space, both the acoustics, whether they're favorable or challenging.
It matters where we put things, and microphones are no exception. And if I am 25ft away from a microphone, I can sound like I'm 25ft away from a microphone.
As I get that microphone closer to the individual, it sounds better.
Acoustic treatment in a space is huge and something that is frankly and unfortunately often overlook these days as we go for minimalist and less material and open ceilings and hard surfaces.
That is something that technology is working very hard to contend with. But you can't overcome physics. If the room just sounds good from the start, it just sounds good from the start.
[00:14:07] Speaker B: That. That's. I don't think we can underscore that enough. I mean, I'm looking at where you're speaking right now, and behind you you have glass, which obviously is going to reflect all those standing waves that are coming off you talking. And a lot of times we're looking at rooms that have been retrofitted to be conference rooms or offices, and they have dropped ceiling with drywall. And you're right, there are boxes. And unfortunately the acoustics there are not great. And have you found that there are ways, and I don't mean like the old let's hang up egg crates or rugs like you used to do in the garage, but have you found that there are ways to kind of improve sound quality without kind ruining the minimalist look that we're seeing in conference rooms?
[00:14:51] Speaker C: Well, I'm sure, you know, you're an audio guy, you understand, you know, the single to noise ratio. And the farther you get away from a microphone, the more ambient noise you're going to pick up along with that noise. And you're going to just, you're going to have to deal with it. And frankly, the system today are getting a lot better separating the wheat and the chaff, the noise from the signal.
So that's great.
But something as simple as some fabric treatment in a room, we use some enhanced acoustic tiles in some of our spaces here, some simple rockwool panels on the wall, some felt. Any. I mean, honestly, anything helps, right? And we encourage folks to think about that as they're designing their spaces. Because even with the best technology and the best processing, if you start with something great, you require less to fix.
[00:15:41] Speaker B: What is that line? Garbage in, garbage out.
[00:15:43] Speaker C: It is,
[00:15:43] Speaker B: right?
So let's move to kind of the control layer. And obviously Crestron has been involved in room control for an extremely long period of time. Does AI eventually change the control model from the user pushes a button or the user knows or understands visibly where to push a button to. The room understands what the user is trying to do.
[00:16:05] Speaker C: That is a huge frontier for us and the industry at large.
Realistically, we've done a really good job over the years of improving the user experience through improved user interfaces, making them more intuitive.
Cite the iPhone is a perfect example. Folks can pick that up and do some very complicated tasks without a lot of training.
And I think the same is true for our rooms.
But what something like AI brings to bear is no training required.
If I harken back to the pandemic again, there was a voice control movement, right? People wanted to avoid touching things.
So voice control came to fore. And there was a lot of attempts to make that work really well, but it didn't work really well. And the reason it didn't work really well is because the user had to say the very specific thing in order to get the system to respond the way they wanted it to. It's a very deterministic algorithm. Just like pushing a button. I have to push this button in order for that thing to happen. If I push another button, I get something else that happens. If I don't push any button at all, nothing happens. So with AI coming into the fore, and something we're leaning into heavily is called a model context protocol server, MCP server for short, that's going to live right on our control system.
And what that is in the simplest terms is between where that goes between the program logic running on the control system and an AI large language model. You know, if it's Claude or Chat, GPT or Copilot, whatever the case may be.
And what that allows is non deterministic human like conversation interaction with our, our systems, right? So instead of maybe I just want to say I want to watch, you know, Stranger Things, right? I never programmed anything on the control system to respond to that specific request to call up the Apple TV and to set it to Netflix and to play Stranger Things. However, if the model context protocol is set up correctly to expose that, hey, there's an Apple TV in this room, it's connected to a display and Claude is intelligent enough to understand that Apple TV runs Netflix. Netflix has Stranger Things.
Now you start opening this whole world, this infinite world of things you can do with these control systems that didn't have to be hard coded and programmed on the front end.
So that's something that's really exciting. We showed a little snippet at Infocom and we're working that to bring that as a firmware feature right to our processors to enable that kind of interaction with control systems.
[00:18:47] Speaker B: And that may sound a little daunting and I don't want to minimize what Cresthon is doing at all, but MCPs have become a huge topic of conversation because to your point, it allows someone to interact with an LLM in a language that they speak or they understand without having to know programmatically what I have to do to make this software do what I don't need to have a mouse and a menu and know what to click. Or to your point, say the exact phrase. I can just type it in a way that makes sense to me and the MCP will translate that into what the end software needs to work. And I think combining that along with then speech, because if it understands what you're typing, it can also understand what you're saying. And as you can see, that enables a whole world of difference in how things are controlled.
Something else to kind of look at is, while it may understand, excuse me, while the LLM will be able to translate what you say into the mcp, could a room also recognize that a presentation or a lecture or a video conference or a town hall is starting and maybe configure the cameras, the displays, the audio, the lighting and other systems automatically, not just what the user is asking, but be, I hate to say context aware, but environment aware to put those into play.
[00:20:06] Speaker C: I think that we can kind of see that coming down the road.
And it all starts with data points, right? The more data points that you feed into these systems, the more context they have, the more they can kind of understand what's going on, make recommendations or even if you've empowered them to take action, to take action.
Right now, I don't, I don't know that we're quite there yet, but it's absolutely plausible to see that sort of thing coming. That you know it. I can count that there's, you know, 13 people in the room. I see that they're seated, they must be ready to have a meeting or hey, the heat load is going to go up because there are 13 people in the room. So I better set the set Point of the, of the thermostat down a little bit. There's all kinds of infinite possibilities that you can do once you have those data points. And I think that going back to what designers need to think about is how am I going to feed these data points into some future platform? So I should pick some foundational products that have that capability down the road to feed those data points into the system so they can make those decisions.
[00:21:17] Speaker B: Matt, when you're designing a system, where do you kind of draw the line between helpful automation and a room making too many decisions for the end user?
[00:21:25] Speaker C: Yeah. And I think this is a classic principle. Right. Because we've had the ability to automate based on various triggers for 53 years. Right. As long as Crestron's been around.
And my philosophy, and I think the philosophy of most of us at Crestron is automation should support the workflow but not get in the way of it. And we've all had automation in our lives. We're like, no, I didn't want you to do that. Stop doing that. Stop capitalizing the thing. Stop changing the word. I didn't mean duck. Right. I mean, there's all kinds of that we can reference where automation kind of gets in our way and we're having to undo the automation, and then it's just not helpful. But there's certain things that we can be pretty confident about. For instance, if someone comes in and plugs in a USB C cable to their, their laptop, they probably want to show that on something.
So let's go ahead and turn on the display and route that input to the display. And that's like helpful automation. Right. You're kind of intuiting something that's pretty obvious.
So if we try to over predict or say, oh, everyone who comes in wants the shades down, that may not be the case. Right. So a lot of it is understanding the user, how they work, understanding the environment, and adapting it to those particular. Those workloads. I don't think there's a one size fits all here.
[00:22:41] Speaker B: Mm.
Let's move from kind of the. The user facing to more of the, shall we say, back office admin portion of this. So today we can monitor AV devices and identify when something goes offline or isn't behaving correctly. Does AI create an opportunity to move from monitoring problems to maybe even preventing them?
[00:23:02] Speaker C: It does, it does. And I think that's the real promise of, of where all this is going.
You know, One of the AI's great abilities is to scour troves of data and identify anomalies trends over time.
And you can use that paired with a knowledge of what this device does to, you know, come up with some preventative maintenance. Saying, I, I've seen failures in this type of device at this time in these conditions, you know, and again, how do we, how do we get that decision? We have to have the data to begin with. So having the ability to ingest that data from the, from the front gives us that ability to examine that data downstream to identify those trends. But, but yeah, I think absolutely preventative predictive maintenance, I should say, is something that we're going to be seeing in the not too distant future.
[00:23:56] Speaker B: So let's look into kind of the, the AI portion, but also the AV integrators. Right. Folks like Key Code media. So does AI mean for the people who actually design, program and commission and support AV systems? What do they have to, what do we have to do to answer that call?
[00:24:16] Speaker C: Yeah, it's really understanding that the technology is going to grow.
You know, most organizations that want to set a standard, whether it's a classroom type or a meeting room type, and they generally want to live with that standard for a period of time, a technology refresh cycle. We know technology is changing a lot faster than that, right.
So we want to be prepared for that foreseeable future.
Starting with a strong foundation, something that's going to last test time doesn't need to be forklifted out in two years because it just wasn't powerful enough for the next whiz bang feature, I think is important. Right. So starting with something that's powerful enough to last for the next three to five years as a foundation is important.
Thinking about what can be upgraded around it over time. You know, maybe we can change a camera or change an audio peripheral over time, either by firmware or, or some, you know, physical, physically modular thing that we can change. So product selection, ecosystem selection, thinking about what's going to be able to give those systems that data over time to make those more intelligent decisions down the road is really key.
[00:25:29] Speaker B: Here at KIE Code, we leverage AI a tremendous amount, mainly to validate initial assumptions, align expectations with what we're putting together, and always having a human in the loop. But I'm curious if, Crestron, are you currently utilizing AI to help generate configurations, maybe validate room designs, and even identify programming mistakes and even things like optimizing camera coverage or speed up commissioning? Is Crestron putting tools together for that? I
[00:26:01] Speaker C: feel like you just lobbed me a softball.
Yes, yes, yes and yes. A couple of different ways.
On the programming side, one of the things we use is certified drivers, which just helps ease programming, make sure things go very smoothly. They can be dropped into various systems without having to reprogram the system. If I change from projector X to projector Y, easy, I just change the certified driver with that on the fly without having to reprogram things, which is, which is great in terms of commissioning and validation. If you can see it right over my shoulder here, there's a kind of QR ish looking code, we call it an Aruko code, and that is a cube that we place on the microphones with our Automate VX systems light up the system. The cameras see those cubes. They can understand the spatial relationship between the microphone and the camera, detect skew, roll, positioning the camera and compensate automatically.
So for commissioning these systems, which was already a pretty not too involved process, just got a lot quicker and totally automated. So something that could take several hours now takes 15 to 20 minutes in an automated way, just letting the cameras do their thing.
[00:27:11] Speaker B: Matt, if we look five years ahead, what will a conference room or classroom understand about what is happening inside the space that it simply doesn't understand today?
[00:27:23] Speaker C: I think that goes back to data points.
And one of the great things about AV is we know more about these space than just about any other subsystem in the building. We've got more touch points, we understand what's getting plugged in, we understand what's getting routed, we understand temperatures, we understand a lot about how these spaces are being used. But we have to have a way to collect that data. The device does the thing, that's great, but if it can't report what it's doing, what the inputs are, and feed it to something like a large language model, then it's kind of moot. So we have to have a way to get that data into the AI in order for the AI to do something useful with it. For us, if you think of data like pixels in a picture, the more data points we have, the more pixels we have, the clearer that image becomes. And that's the same for us when we're viewing something. That's the same for AI when it's viewing something. The more points of reference that it has, the better it's going to be about understanding how that space is being used or not used. Right? So we can make more intelligence based planning decisions in the future. But again, it all comes back to the ability to harvest that data in the first place.
[00:28:37] Speaker B: To take that one step further, a Lot of the data we've talked about has been kind of objective, right? This many people yields this many BTUs, meaning we have to put the AC up to this temperature. But what about. About when we get into things where a room understands who is speaking and what the content that's being discussed and how people are using the space and, and responding to what's being said by the presenters, Obviously that is, and we've talked about this before in the podcast is almost like YouTube. Watch analytics, right? Find out what resonated, why it resonated when folks started dropping off. Do you find that that is. That's the next stage for what Crestron could do. And I'm not asking for product roadmaps, obviously, but is this something that you feel may be part of the responsibility of the AV system to give these analytics?
[00:29:28] Speaker C: I mean, I think in general, sentiment is going to be probably the next thing that can be sensed, right? So to speak. Sensors kind of drive so much of our world, but one thing we can't really sense right now with conventional sensors is sentiment, emotion, feeling, interest, engagement.
But we already have tools that can identify faces, bodies, position, where they're looking.
It's not too far of a leap to assume that they can identify facial gesturing and sentiments and report that as a metric, we're not there today, that's not where we are today. But it's certainly plausible to see that that's where things could be going in the future.
[00:30:15] Speaker B: Matt, thank you so much for your insight and joining us and talking about AI and Crestron today. Thank you.
[00:30:21] Speaker C: It was my pleasure. Thank you so much.
[00:30:24] Speaker B: Thanks for watching. Broadcast to post. Don't forget to follow KIE Code Media on social and contact us about your
[email protected].
