AI Infrastructure Connectivity with Zayo's Bill Long

Full Transcript

Don Kellogg 0m10s

Hello, and welcome to the three hundred and twelfth episode of the week with Roger, a conversation between analysts about all things telecom, media, and technology by Recon Analytics. I'm Don Kellogg, and with me as always is Roger Etner. Roger, how are doing? I'm great. So, Roger, this week, we're joined by Bill Long.

Don Kellogg 0m26s

Bill is the chief product and strategy officer at Zayo. And, actually, I believe, Bill, you've been on the podcast once before, so welcome back. How are doing? Great.

Roger Entner 0m34s

Great to talk to you guys.

Guest Speaker 0m36s

Yeah. Great having you. You just got another friend of the show, Sam Perth, as your CEO. So pretty exciting times. You are at the epicenter of where telecom and AI meet.

Guest Speaker 0m51s

Right? Absolutely. There's this whole discussion. Is this a bubble or is this serious and will stick around? How does this look like from your perspective?

Roger Entner 1m3s

Yeah. Mean, if just to get wonky with you for a minute, you take a step fully back, we think there's long term durable demand. One is because if you look at TSMC chip fabs are reserved five years out, you can see that the chip fabs are reserved out. Those chips have to go somewhere. They go in the data centers.

Roger Entner 1m19s

The data centers need to be connected. On a five year horizon, we feel very good about the demand. But even if you take a further step out and you think about the economic value of a token that that token provides versus the cost of generating that token, there's this huge discrepancy where a pull request with a developer doing the work would have an economic value or an economic cost of $2,000 When AI can do it, it can be $2 So there's this huge spread between economic value created and the economic cost required. And the only way that you bring those back into balance is with more infrastructure. So we think that both when you look at sort of the TSMC chip fabs reservations out five years, it gives us confidence.

Roger Entner 1m59s

But larger, the economic value of the token versus the cost of the token that gets solved with more infrastructure. That means long term demand is in a really good spot.

Guest Speaker 2m7s

Yeah. And Jim Patterson, who has a terrific weekly or biweekly newsletter, The Sunday Brief, he's now the CEO of a company that builds enclosures. They're like booked out for three years. Yeah. He sold everything for three years.

Guest Speaker 2m23s

And so we see all the same thing, you as like the connectivity provider for long haul, big data center things. And you're not only working in lit fiber, but also in dark fiber.

Roger Entner 2m37s

Yep.

Guest Speaker 2m38s

What do you see there?

Roger Entner 2m39s

Yeah. Mean, we've seen a huge increase in demand. I think if you look at the training runs are requiring bigger and bigger power. It's very hard to find multi gigawatt pods of power that are available. So doing a single training run over larger geographies, those training pods need to be able to talk to each other.

Roger Entner 2m56s

And we've seen a dramatic increase in long haul fiber demand because of that. You go back eighteen to twenty four months and a large fiber order for us was $810.12 fibers. We're now seeing regular orders for four thirty two and even eight sixty four fiber count. So just order of magnitude increase in demand for long haul fiber.

Guest Speaker 3m15s

Wow. And since it's dark, there's no really that's planning for the future. Right? That's not even what they're seeing now.

Roger Entner 3m23s

Yeah. That's right.

Guest Speaker 3m24s

Everybody, you know, from Jensen Huang to Sam Altman to Amo Day from Anthropic are talking about the shift basically from training to inference. How does that materialize on the connectivity side?

Roger Entner 3m39s

Yeah. What's interesting is, of course, we're seeing a lot of demand for still training in the bulk fiber, long haul fiber for that. But when we're talking to a lot of the neo clouds and folks who are turning up data centers that we're providing connectivity to, they wanna make sure that the physical infrastructure that's being architected can work for training and for inference. So if they're deploying GPUs today that are gonna be used for a training run, they wanna make sure those same GPUs in the future can be used for inference. So what that means is you need to not only allow a huge, you know, gigawatt scale data center to talk to another gigawatt scale data center.

Roger Entner 4m12s

If you want to be doing inferencing, you need to be able to connect to where those inferencing requests are going to be coming from, where the data resides. So the things that matter for inferencing are, do you have the architecture to connect to the rest of the world? So make sure you're in an interconnection node that can connect to the rest of the world, that the scale can support the sort of multi 100 gig connectivity that's required, and that you can do it over private networking, not just over the open Internet. So it's people want the flexibility. And architecturally, it means you need to design when you're deploying these data centers, design them for the future, not just for a one training run and done kind of

Guest Speaker 4m49s

Yeah. No. And we're reading in the press an announcement from, like, both AT&T and from Charter touting that 400 gs is becoming their standard for metro fiber for them in as a response to AI. How does that materialize on your end? And does that mean that 400 gs is becoming the operational standard?

Guest Speaker 5m12s

I was speaking at a data center conference in Germany, and I said, we're much closer to 800 g's, if not to terabits. Right? That's no longer a g. It's a t.

Roger Entner 5m24s

Right. Yeah. We're seeing demand for waves go through the roof. We're standing up whole systems for customers now. Even though the the span speed is still at 400 gigs, it's 400 gig times many times.

Roger Entner 5m36s

So we've turned up over the past quarter multiple 20 terabit systems. And what that looks like, though, is that in order to meet the demand that you need for that, it needs to be fiber rich. So if you're planning for ten, twenty, 3,400 gig kind of circuits, that will meet some of the demand. But really what you need is you need to be fiber rich such that you can meet the multi tens, hundreds terabit type of connectivity requests that are coming in. And and it's really only dense fiber rich infrastructure focused providers who can do that.

Roger Entner 6m9s

It's not the onesie twosies type of over the top waves providers who can meet where the big demand is these days.

Guest Speaker 6m16s

Yeah. And then it's, you know, US, one of the largest, if not the largest provider here, are at the forefront of it. So when we look at the data center side, we have like 20 Ts now. How does this filter down from the data center to long haul, to metro, to then individual enterprises? Do they need already dark fiber to be AI ready?

Guest Speaker 6m43s

Or how does this then percolate to the entire system?

Roger Entner 6m47s

Yeah. Think the way that we think about this ecosystem is kind of like a two by two matrix, where you have training and inference for consumer and enterprise, and there are different pinch points that are arriving there. So we've talked a lot about training and sort of the requirements for long haul dark fiber on training. And so when you get to enterprise inference, as an example, the way that architecture is changing there is where your data resides versus where the intelligence that you need to apply to use that data on are in two different places. So your data may reside in the cloud, it may reside on premise versus the intelligence you want to use can sit somewhere else.

Roger Entner 7m21s

So one of the demands for enterprise inference is you really need to have an architecture that can allow the data to connect with the intelligence in a way that's not going over the public Internet, that can scale and is highly performing. So it basically looks like cloud on steroids, where when you're connecting to cloud, you had to have those same things. But the volume of traffic, the security profile of that traffic, the performance it requires, and the ecosystem that's enabling it are sort of put another zero behind each of those elements. And that's what you get with

Guest Speaker 7m50s

Yeah. And you're making me think through our architecture. We do a lot of things with frontier models in the cloud, but we're also working now with moving the inference locally, like building clusters with several RTX 6,000 machines. We're building, you know, an m five ultra cluster with multiple m five ultras, simply to have a diversity of models. And then a lot of our data sits on the cloud, without doubt.

Guest Speaker 8m22s

A, we have our data in the cloud, and B, our data redundancy is like, Don has a NAS in his basement, I have a NAS here, and then we have it on Dropbox, we have it on Google. So four things would have to go down simultaneously for us to be really knocked off. And so the connectivity demands are massive, we're a relatively small company. So with that, you know, we see already that the problems with not having enough chips, not having enough memory, not having enough storage. Does that mean that limited bandwidth will become the next major constraint?

Roger Entner 9m5s

Well, the answer was it was going to be. You know, we talked a lot about this long haul fiber, and and I mentioned where in the old days, ten, twelve fibers was a big fiber order for us, but we now are seeing orders for 432 fibers. When we did our long term planning, the capacity we'd planned on taking ten years to use up was gone in eighteen months. So we recently announced a large complete overpull of our full nationwide backbone deal we did with NVIDIA, where we basically sat down and looked at our fiber map and showed the whole ecosystem. We showed all hyperscalers.

Roger Entner 9m36s

We showed NVIDIA. We showed others that if we don't solve long haul fiber connectivity, we're going to be out of fiber. It was going to be a constraint. We have now, with the announced launch that we have that we're doing multiple 24 different routes connecting the major hubs, long haul is taken care of. But that also had pile on effects into we needed to have the fiber.

Roger Entner 9m56s

So we had to do a deal with Corning to make sure that we had the fibers. It takes lit equipment. So we've done deals with the equipment providers to make sure we have the WAVES equipment that we're gonna need to do it. So it affects the whole supply chain. And then you have the pile on effects where there are spot use cases.

Roger Entner 10m10s

One interesting use case that we're talking about now is that the whole Internet was basically built to replicate and distribute the same bits, meaning if we're all streaming the same Stranger Things episode, that content is cached deeply in the network. As you have more bespoke or personalized AI content, the metro network starts to break. So we've sold for aggregate demand in the long haul network, but as each of these spot by spot use cases comes up, it's creating pinch points at different points in the network.

Guest Speaker 10m40s

And the traffic is becoming it is not symmetric, but it's becoming more symmetric.

Roger Entner 10m46s

That's right. It's both upstream, downstream as you're uploading, you know, your Ring camera videos into the cloud to monitor things is you know, we have a great use case where there's a major retail company that has distribution centers, and they have over $100,000,000 worth of robotics in each of those distribution centers. And they're streaming sort of the live feeds from those robotics up into the cloud. Where they used to have a 10 meg connection into their warehouse, they now need a 100 diverse 100 gig. So it's just a order of magnitude increase.

Guest Speaker 11m17s

It's insane. And even, like, on an individual level, AI has become a wonderful troubleshooting and tech support alternative. Like, it took me eight hours to increase the RAM from one of our machines from one twenty eight gigs to two fifty six gigs, took hours for that memory to be trained by the system, all of that. And I used ChaCept as my tech support, took pictures of the motherboard and the different colors of that to walk through it. And I'm not the only one who does this.

Guest Speaker 11m55s

And so you have like then videos, you upload, you have all of these things. So yeah, it's just like multiplying. So what are you doing differently than anybody else in the market? People look at you, and I don't think they properly, fully appreciate what Zayo is doing. On one hand, I want to apologize that I didn't say, why don't you introduce Zayo?

Guest Speaker 12m18s

But I felt like with our listener base, you know, we're inside telecom, you know, name. But I think most people don't appreciate you the way you should be appreciated. You know, how should they look at you?

Roger Entner 12m32s

Yeah. I think setting the larger context, we are a digital infrastructure provider, not a network service provider. So if you look at the legacy of where most telcos came from, it was a legacy of providing voice service or video service or something, and then they had the infrastructure in service to that providing voice or video or something like that. Zayo came at it from the completely opposite direction. We were an infrastructure first company, and we offer services as just a mechanism to sort of access that underlying infrastructure.

Roger Entner 13m1s

So what we're doing differently specific for AI, because we are so infrastructure focused, we view this as a two sided ecosystem where the intelligence is the supply. So if you think about where that intelligence sits in large data centers and cornfields in Iowa and distributed everywhere, job one is to provide the infrastructure, so the network, the fiber to connect to the intelligence supply. On the flip side, the demand for that intelligence is with enterprises and consumers. And we are connecting the enterprise locations, whether that's headquarters locations, infrastructure they have in the clouds, branch offices, etcetera. And then consumers, whether that is through cell tower connectivity, connecting to cable providers or or telecom providers, that's the demand.

Roger Entner 13m48s

And then our job is to remove the friction from the demand to be able to get to the supply. So the way that we're approaching the market is infrastructure first, two sided ecosystem where we connect to the supply, we connect to the demand, and systematically make it easy such that you can access infrastructure and software speed. So that's very different than someone who was coming from offering wireless services first, like a Verizon or AT&T or even a cable provider first, like Charter and Comcast. We are infrastructure first and bespoke by connecting to know, we connect to over 2,000 data centers, and basically all the neo clouds and hyperscalers and AI labs are using us to connect. So that's that's where we're just a very different sort of provider out there.

Guest Speaker 14m33s

Yeah. You have a different focus. Yes. You don't have retail customers. Exactly.

Guest Speaker 14m37s

You live and die by these large organizations and how to service them. So maybe we can close our conversation by how should they do things differently and get prepared or what we're gonna see in the next twelve to twenty four months? Because I don't think anybody can look out much further than that.

Roger Entner 14m58s

Clearly. I mean, even twelve to twenty four months feels like decades in the old world, the amount action that's happening. But it's really plan for agility. I talked about earlier how AI looks like cloud on steroids. I mean, starts with make sure your architecture has the agility to connect who you're going to need to connect to, not just today, but in the future.

Roger Entner 15m18s

That you do it at the scale you need it with the security you require at the performance that's needed. So it's it's really plan for the unknown instead of being surprised by it.

Guest Speaker 15m26s

Awesome, Bill. Thank you for coming. This was fascinating, and I'm sure we will positively, this will be with us for a long time. And, you know, I'm looking forward to having you or Sam Perth coming on the show and to give us an update.

Roger Entner 15m43s

Great. Great catching up.

Guest Speaker 15m44s

Alright. Thank you.

Don Kellogg 15m46s

Alright. Thanks, gentlemen. Raj, we'll talk next week.

Guest Speaker 15m48s

Talk to you next week.