An Introduction to Edgio’s Beyond the Edge podcast Episode 3: Does Low Latency Matter? hosted by Kyle Faber, Senior Director Product Management – Edgio Media Platform.
Kyle Faber: Welcome to Beyond the Edge, a podcast dedicated to exploring the dynamic challenges faced by modern digital businesses. I’m your co-pilot. Kyle Faber, senior Director of Product Management for the Edgio Media Platform. And today we’re diving into the topic of low latency. And does it even really matter? As the speed of delivery gets faster and faster, the definition of low latency has changed over time. It used to be that low latency meant sub-30 seconds. Then it meant sub-10 seconds. Then it meant sub-5 seconds. Now ultra-low latency offerings offer near real-time delivery across the globe. The truth is, a slight delay is often not problematic. The majority of broadcasters surveyed in a 2021 video streaming latency report indicated that they were experiencing latency in the three to ten-second range, and 25% of them reported ten to 45 seconds. So does that latency matter? Let’s go beyond the edge with our guest, Jason Thibeault, executive director of the Streaming Video Alliance. JT, thanks for joining us.
Jason Thibeault: Well, thanks, Kyle. I’m sorry, I’m sorry. Let’s go a little more informal. That was a beautiful intro. I never thought of you as a pitch.
Kyle Faber: In full disclosure for the folks at home. JT and I have worked together for the better part of a decade in our various roles around this industry. So we will get a little informal.
Jason Thibeault: But by the way, it’s the Streaming Video Technology Alliance. We did change our name. Just putting that out there.
Kyle Faber: I didn’t say streaming video technology. Alliance. I did say that. It’s right here and I know it.
Jason Thibeault: I’m on the board. You said streaming video alliance.
Kyle Faber: It is the Streaming Video Technology Alliance.
Jason Thibeault: Well, thanks. And I’ll give a little background on myself.
Kyle Faber: Right.
Jason Thibeault: So as Kyle said. Thanks, Kyle. I’m the executive director, of the SVTA, as we’re commonly called. It’s a global technical association. We’re similar to other organizations like IETF and Simti and W3C and CTA Wave. It’s a standards definitions organization. So we bring together folks from across the industry, and we try to work together to develop specs, best practices, guidelines, potentially standards to deliver better high-quality video at scale. Right? A lot of technical challenges to solve across the workflow. There’s challenges with latency, challenges with network protocols, challenges with packaging, challenges with quality of experience, measurement, all those things. Our folks come together, to solve those problems. We publish docs. People use the docs, to make better streaming platforms. Boom. That’s my elevator pitch. It’s pretty good.
Does Low Latency Even Matter?
Kyle Faber: It’s a great elevator pitch. And Edgio is right there making our streaming platform better by our participation at the SVTA. Let’s jump right to it. We told marketing that we did a clickbait headline for this bad boy. Does low latency even matter? So let’s go right to your hot take on the topic. Does low latency matter?
Jason Thibeault: No. Let me qualify that. I know that’s very controversial for me to say that. Yeah. So obviously latency does matter. But the problem is, I think in the industry, we have a tendency to do these broad brushstrokes of we have live video. Okay? So it must all be ultra-low latency. No, whoa, wait a minute. And that’s the issue that we face. So when I say and the title says, does low latency even matter in a lot of cases, no, it does not, because it’s not the priority for the experience that you’re trying to deliver. But for some use cases. Right, we know. And I just moderated a panel at Streaming Media Connect just yesterday on ultra-low latency and micro-betting. There’s a use case where low latency is really needed, and if you don’t have your latency under control, people are going to miss their bets, and they’re going to get mad, and they’re going to leave, and you’re going to lose money. That’s really what it comes down to answering that question, does latency matter? It comes down to the use case. So, yes, it does matter. For certain use cases – no, it does not matter across the board. And I think as an industry, we really have to stop focusing on it.
Not All Video Content is Created Equal
Kyle Faber: I agree with you 100%. You make a very great point that not all video content is created equal. We got into this entire industry, the entire source of the idea that this is so beautiful. Is that we can expand the different formats so we don’t have to wait for the democratization of what people can bring out. So you have live betting, live gaming, as you mentioned, you have collaborative chat spaces where you have live streamers and the Twitch-style communities where there’s back and forth with the streamer. That interactivity requires low latency when the viewer is doing something. Of course, you need that back and forth. Like right now if the video that you and I had, if we weren’t in the same room, this would be useless. So video matters. Might be useless anyway.
Jason Thibeault: I was thinking in my head, I was like, should I say that? No. Oh, you said it.
Kyle Faber: But when you move to a broadcast quality or content that customers… that end users are looking to experience with maximum quality and not maximum interactivity, you move into a space where the latency stops mattering as much, and you end up making trade-offs that your users aren’t valuing. And you’re making trade-offs for things that they are valuing.
Jason Thibeault: Well, not only that, but those trade-offs cost money, right? So if you’re putting all of this time and effort and expense into a trade-off that you think your users think is important and they don’t, you have literally just wasted a ton of resources for very little gain. And I think that’s when we get into the latency issue, going from ten-second latency to seven-second latency, not that expensive. It’s really not. Going from seven-second latency to two seconds. Now you’re talking about stuff that maybe you’re having to replace network equipment and gear and you’re having to renegotiate transit costs and things like that. It’s expensive going from two seconds to sub one second. There are some massive infrastructure impacts there, right? Let’s talk about ultra-low latency. You need specialized servers. You really can’t deliver that VHDP servers. So back in the day, we remember right QuickTime, we remember RTSP, we remember these special servers we had to have. Infrastructure is a pain in the butt. It’s elastic now, which is great. So I can have Wowza as a fantastic server for WebRTC and spin up a bunch of Wowza servers.
But the point is they can only support so many people. They don’t have the scale for WebRTC that they have for HTTP. And so now I’ve got to have more of them to support more people watching ultra-low latency. What’s that going to cost me? And am I getting the return if I’m delivering WebRTC ultra-low latency streams to everybody and only 1% of the people are using it to bet, but it’s costing me ten times more to deliver that stream than I would at HTTP stream. Why would I deliver that low latency stream to all those extra people who aren’t contributing back to my bottom line? So it’s exactly what you said. I mean, it’s just these trade-offs, they happen and they have to be considered. But the problem is that, again, as the industry, we just make these brushstrokes of all live streams must be near 0 seconds and it’s like, no, you’re wasting money, stop doing that. And I’ve had some cool conversations with sports distributors about the way that they’re sort of parallelizing. Sorry, easy for you to say. Can you say that three? I can’t. They’re running in parallel, parallel lines, everybody.
They’re delivering HTP streams to the folks who aren’t bedding and they’re delivering WebRTC streams to the folks who are. And so then if you were to ask them, does low latency matter? They’d be like, well, yeah. To which users are you asking? To the ones who are watching http no, to the ones who are betting.
Kyle Faber: I was just going to say it is directly down to the use case. And some people are in this for free, but nobody’s in it for free. There’s an expectation that there’s going to be a turnaround. And so you said, if 1% of your audience is betting… if 1% of your audience is betting and they’re all terrible at betting, and that pays for your whole stream to stream it to 10,000 people, 100,000 people, a million people, then go right ahead.
Jason Thibeault: Yeah, exactly. Good for you.
Kyle Faber: If you feel that’s driving them into your market. But if your users are not generating new revenue for you because it’s lower latency, then it’s important to evaluate the choices that you’re making to drive that latency. And that’s the bottom line, right? So to answer the clickbait question and to continue it on, does low latency matter? I don’t know. Does it really matter to you? And that’s really a question that everyone hearing this podcast should be asking themselves for their content when they go into a workflow discussion. They should be asking does low latency matter?
Jason Thibeault: Well, and we know what the vendors say, right? So that the tech vendors who are selling to the streaming operators. We know what they say. Yes, it matters. We have this use case for you. Listen, imagine this scenario you’re watching sports at home and then you hear from through the wall next door, somebody go “gooaaaalll” and you then see the goal on your TV and you’re like, that’s ruined for me. We know that this actually does not happen.
Kyle Faber: I don’t believe it’s ever happened.
Jason Thibeault: Once I’ve been in bars, like watching TV, and I’ve noticed that TVs were not synced well, and that’s either satellite or cable. So ha. Not just a streaming problem, but anyway, occasionally somebody about to go, yeah. And then I’m like, why did they oh, goal wow, look. It didn’t ruin my experience, first of all. And second of all, it wasn’t streaming-related, it was actually broadcast feed-related. But regardless, this is not a valid use case, as far as I’m concerned, for making the case of we need low latency. Yes, there is a reason to have low latency for live sports. Generally, you do want people to have things that are closer to real-time than not. But there are other considerations for low latency. Right? Now you’re talking about buffer and chunk size and there are all these other considerations where you may actually be causing other problems by forcing something to be low latency when it doesn’t… when the benefit is not there and it doesn’t need to be. So we know the vendors, they’re going to tell the story that everything needs to be low latency, but the content owners, the streaming operators, they need to make careful decisions about and answer the questions for themselves.
Does low latency matter for this use case? So that was the part that was missing from the clickbait, because then it wouldn’t be clickbait, right? It would not be and it would actually be realistic. So it was good that you guys truncated that to just this low-latency matter. I love it. I feel so excited to be part of a clickbait title.
Kyle Faber: We finally arrived. It took us this long in our careers JT to make it to clickbait, but you’re exactly right. It depends on what your users value. Maybe I am the one unicorn that has thin walls and a neighbor who enjoys the same sports that I am.
Jason Thibeault: At the same time.
Kyle Faber: Yes, even then I think, and I am a live sports fan, I value the lack of interruption to my stream, the complete and total lack of a blue circle that comes up spinning on that player way more than the occasional time I might hear this mythical neighborhood screams.
Jason Thibeault: That’s the thing. And it’s interesting when you think about again, back to that you’re going to go low latent, your chunk sizes are going to change, your buffer algorithms are going to change, you’re going to have to do different things in your player to make sure that you can maintain a reliable connection. Because if you’re real-time but you continue to have interruptions which may not be your fault as a streaming operator, that may be the fault of the ISP, the last mile network, the user’s router, the user’s computer, all sorts of issues could cause the little blue circle and interrupt your stream. And so then the question again that streaming operators have to ask themselves is not just does low latency matter, but what matters more than low latency in this specific situation? And that could be reliability. Oh, my users and viewers want an uninterrupted experience. Fancy that. If that’s ever a question that you no, no, my viewers want to be constantly interrupted. Of course, they want that. I remember, hey, back in the day when I was at Limelight, I remember the BBC one time saying to Limelight, listen, you got all these doohickeys and watch-them-a-doozles and I don’t know what the UK words are for those.
So I apologize to anybody from the UK because nobody at BBC said those actual words, but you get the gist. They were like, you guys got all this stuff regardless. It just has to work. People have to push play, and it shows and there’s no buffering and it’s good quality, and we’re done. And it was like, that’s what I’m reminded of in terms of these decisions about latency.
Kyle Faber: Spoiler alert, The BBC is not weird. Every single customer has said that to us every single time. We need it to be perfect. Now tell us about features, which is exactly my point. And so you said no one believes that their end users will ask about interruptions.
The Holy Grail – Personalized Advertising
Kyle Faber: But that does remind me about, like, what you were talking about, the entirety of the workflow, the interruptions that the users don’t ask for. But we’re still going to give them advertising. Sorry. That’s how most of the content gets here. Personalized advertising – that’s the Holy Grail.
Jason Thibeault: Right?
Kyle Faber: We’re going to start seeing it’s going to be more targeted. You’re going to start seeing ads that you care about more. You’re going to start seeing ads that are more relatable to you. And so there’s an element of product discovery that comes with advertising, and you want viewers to have a good experience with that advertising. The lower you reduce your latency, when you’re making those personalized ad requests, if you’re not providing enough time for that marketplace to provide the advertisement that you would like to show to that user, you’re forced to either show nothing. So you’ve lost ad revenue. You’ve carefully crafted your content to attract these viewers, and then, oh, just kidding, we’re going to let this ad slot rot. Or you go and you get an evergreen ad, like a Coca-Cola, right? You get a million backslides from Coca-Cola. Well, then if you’re not bringing in enough personalization, then your users just see Coca-Cola on repeat. And I think we’ve all seen that.
Jason Thibeault: I just had this experience yesterday on Hulu, and it was a repeat of this energy ad, and it was like, I will never use them as an electricity provider, even if they’re the last ones. I will light candles in my house and have squirrels running on treadmills. no to them, because I’ve seen your ad 57 times in the past hour. Done.
Kyle Faber: But your reaction to that is exactly what I’m talking about. You didn’t blame Hulu, and you know this industry.
Jason Thibeault: I know exactly. I had to explain to my wife. I’m like, well, she’s like, Why do we keep seeing this ad? Okay, here’s the reason I’m still frustrated. It’s very annoying.
Kyle Faber: So you end up with advertisers who will view your product because of that negative association with their brand. And so as the content producer, you have to understand that just like CDNs we completely understand that the blue circle is our fault and reflects on your brand. Having a low ad inventory and having that come in on ad repetition impacts that brand. And so when you’re designing these workflows that are going to potentially rely on advertising or potentially rely on breaks like that, it’s very important for you to remember the knock-on downstream effects of shaving 3 seconds out of the latency. Are you getting more revenue from your users because they’re betting? Okay, cool, do it. If it’s not leading to anything. If low latency doesn’t matter to you, then maybe you’re causing knock-on effects by striving for something that the low latency vendors have advertised their way into having you chase.
Latency in Workflows
Jason Thibeault: Well, you bring up some really good points, right? So let’s circle this back to the title. Does low latency matter? So we tend to think that that just refers to the content itself. But there is so much latency in the workflow, right? So the SVTA live-streaming working group actually put a paper out on this, right? They examined all of the points in the workflow to see where latency happens. And obviously, the majority of latency happens at the player, but there is so much more latency upstream, and you just identified parts of it with ad fulfillment. Okay, if we’re doing server-side ad fulfillment, right, there’s latency with stitching the ads into the stream. We’re doing client-side ad insertion. There’s latency with a new request to go out and fetch that ad pod. There’s other latency in the workflow that could actually be more beneficial to solve than ensuring that your stream, your live stream, is near real-time. So it’s like, again, the title will go back to that whole thing about payoff. You’re making choices. What’s the payoff? So instead of focusing on trying to squeeze every last millisecond out of the delivery time of my content, maybe I should focus my engineering resources, time and money, to latency upstream, which will solve some of these problems and produce a better experience all around for my viewers.
AKA don’t have as many ad latency, so we get the actual ads that are personalized instead of the repeat ad every single time. And obviously, ad fatigue is more than just latency reflected in the workflow. It has to do with lots of things. But regardless, that’s one of the ways to help mitigate that problem is to solve the latency in that part of the workflow. So I think it’s really you brought up a really good point. Yes. If we’re going to personalize ads and do that kind of addressable advertising, that’s going to have to happen at the edge, going to have to have a lot of resources out there to do that kind of heuristics analysis and crunching and being able to shift on the fly and redo manifests and all that stuff. And that’s happening, but it’s out at the edge. So the round trip time is lower. So again, solving some of those latency problems upstream, not just downstream, it’s really happening now. The latency is being addressed by folks upstream. It’s just, again, I think we have to if we’re coming back to that title, we just have to explode it out a little bit, say like, okay, latency does matter, but it’s not always what you think it is.
It’s not always just is it real-time streaming. It’s more, oh, how much other latency can I get out of my systems by going upstream? Yeah.
Kyle Faber: And so when you look at that entire workflow, anecdotally the Edgio Uplynk platform is launching a new latency target of 15 seconds. You mentioned that’s the entire workflow, from signal acquisition all the way down to that player. We made little slices all through each one of those workflow improvements. And we’ve been able to bring that down and deliver that 15-second marker. And so by standardizing on that, by allowing you to expect the responsiveness that you’re going to get from each piece of the workflow, because we haven’t touched on applying DRM, applying personalized watermarking. Each one of these transactions is important to some people, not important to other people, and takes up time when you need it. And if you don’t need it, then it rolls through it. But I did want to touch back on all of the different ways in which, as you shave down to low latency HLS or low latency dash, and you change that directive, but if you deliver even tighter latencies, you end up with the specialized servers, the specialized hardware that you were talking about moving to a WebRTC style delivery. Those methodologies of delivery are not nearly as scalable on a per capita basis.
Streaming Sustainably and Securely
Kyle Faber: And so then we’re potentially talking about sustainability, right? That’s important to other people. Sustainability drives a lot of ESG-style reporting that companies are very interested in showing we’re delivering to a million people. And so that is being driven by this much infrastructure. But in addition to that, another way to back into sustainability is efficiency. And efficiency is what did you get for what you spend. And so again, are you spending on twice as many servers and twice as much heat to get to the same audience?
Jason Thibeault: Yeah, to your point, I really think that it’s all about kilowatt per hour. Right. So you are measuring every choice against sustainability. So we used to do this with security, right? So before you’d build something, like, you’d build an app, put it together, a streaming service, you get to the end of it and you’re like, we did it. Okay, how can we make it secure?
Kyle Faber: Now, we secure it, we put a shell on the outside.
Jason Thibeault: Wait a minute. We should have had those discussions way back when, and we do now. Now, every step of an agile process or development methodology, it includes questions about ‘okay, is this secure?’ Is it secure to this connection? Okay, is this component secure as part of this overall architecture? We’re going to get there to sustainability. Right? So right now we’re at the end of it and we go like, okay, how do we make it sustainable? I know – we have less of them. Okay, thanks. Yeah, that’s really brilliant. Have less greenhouse emissions. We take cars off the road, obviously. But when you’re building apps to better use resources on the box, memory, you know, Nic, CPU, GPU, you’ve got to make those considerations at the time of build. They’re in your code. And so if every decision is measured against kilowatt per hours, you start to have measurements going throughout your entire lifecycle. When you get to the end, you have a clear understanding of what that unit of “thing” that is part of your workflow costs in terms of sustainability. And I think that’s really important because if you just take the approach now of, oh, we’re doing WebRTC. Oh, we have a million people to serve. Oh, each box only does 10,000 people. Oh, we need a million boxes. Oh, okay, well let’s just spin them all up.
Kyle Faber: But that’s what we need.
Jason Thibeault: That’s a lot of money to do that. But it’s also really impactful to the environment. Stop being an a-hole. Sorry, there’s no censor.
Kyle Faber: There might be a censor. I don’t know. No one gave me the rules, JT. I have no idea.
Jason Thibeault: The editing process is going to be like a blip in the sound and this black box is going to go over my mouth. That’s exactly it. I’m glad you brought that up. Sustainability is really a concern when you talk about scalability of streaming infrastructure because we’ve just had this opinion that you just spin up more resources and as most operators know, you can’t actually do that. I mean, to get really big in certain instances in the cloud providers, you have to reserve them weeks in advance. But anyway, it’s this like, just throw more at it. And thankfully the sustainability conversation and I’m going to give Dom Robinson of Greening of Streaming a lot of props for taking and running with this. We’re starting to have the conversation of stop trying to address scalable problems with that solution of just throwing more gear at it that doesn’t work long term. And so if we’re talking about specialized boxes needed for ultra-low latency and for low latency streaming, then we’ve really got to keep in mind the sustainability impact.
Kyle Faber: I couldn’t agree with you more, JT.
Jason Thibeault: I know.
Wrapping Up
Kyle Faber: I think that’s a great note to end on. The fact that you’re in vibrant agreement with me is the reason why I invited you on my first and probably last episode of Edgio.
Jason Thibeault: I don’t think they’re going to bring either one of us back. Well, we’ll see what happens.
Kyle Faber: Thank you to our guest, Jason Thibeault, the Executive Director of the Streaming Video Technology Alliance, and thank you to you as well as our audience, for joining us beyond the edge. We will see you in the next episode. Thank you very much.
Jason Thibeault: Thanks, everybody.
