Find the episode and subscribe to Uptime with DCK on Apple Podcasts and Spotify.
In the latest episode of Uptime with Data Center Knowledge, we look at the evolution of bare metal servers.
To find out more about the subject, we chat to bothers Jacob and Zachary Smith – co-founders of Packet, a bare metal hosting service that was acquired by data center giant Equinix in 2020, in a deal worth $335 million. Packet became the foundation of the new Equinix Metal business, led by Zac as its managing director, and Jacob – as the VP of bare metal strategy and marketing
Correction: Soon after we recorded this episode, Zac was promoted to head of edge infrastructure services at Equinix, and Jacob – to interim lead of the digital services go-to-market.
According to the Smiths, the key attractions of bare metal are speed and performance: Equinix Metal can be set up in any supported facility in as little as 15 minutes, to run almost any workload on dedicated, physical servers. The process is considerably different from handling servers used to run public cloud applications, where the hardware is often shared between multiple users.
Jacob himself jokes that “no one really cares about servers” – but there are plenty of applications that benefit from bare metal, especially in organizations that value automation and are heavily invested in custom software stacks.
For such customers, bare metal represents choice – a dedicated server is a blank canvas, unburdened by multiple layers of complex software that enables typical cloud workloads. The customer alone will decide what the machine will do, and how it will do it.
We also discuss:
- Open Source software development at Equinix
- Why Equinix Metal doesn’t manage Kubernetes
- How to improve sustainability at the server level
Full transcript below:
Max Smolaks: Hello and welcome to Uptime with Data Center Knowledge, the podcast that brings you the news and views from the global data center industry. My name is Max Smolaks, editor at Data Center Knowledge, and in this episode we will discuss bare metal computing and its role in the modern digital infrastructure landscape.
To look at the subject in detail, I'm pleased to welcome brothers Jacob and Zac Smith, co-founders of Packet, a bare metal hosting specialist that was acquired by Equinix in 2020, and now serves as the foundation of the company's new Equinix Metal service. Today, Zac is managing director of Equinix Metal, and Jacob is VP of bare metal strategy and marketing at Equinix. Hello, Jacob, Zac.
Zac Smith: Hey, Max, thanks for having us.
MS: It's fantastic to have you here. Because I've been following Packet for several years, it always seemed like a very exciting business. I've met Jacob before on the show floor. So I was pleased to see that the world's largest data center operator was taking an interest in this, and putting their money where their mouth is. So I guess my first question is, how is life at Equinix? Because, you know, it's the world's largest data center operator, what are some of the challenges or changes that have happened at Packet as it transitioned from being a startup to a corporate division?
Jacob Smith: Cool, Zac, you want to take the startup to scale up question first?
ZS: Sure, you mentioned it well, Max. I mean, Equinix is a big company, I think we have over 10,000 employees, we're active in 240-plus data centers, and 65 markets. So it's big, right. And the one thing that Packet was, it was small. Startups, you know, prioritize agility, and I'm gonna call it quick decisioning, mainly because they have no other advantages, they have no scale, they have very few customers, you know, you don't have he kind of ‘oomph’ of the industry behind you.
And so the transition into being at a market leader, from what was a startup, which is naturally kind of a disrupter role, has been a transition for me. And certainly, it's exciting in certain ways, because you can see much more of the industry, your impact can be way greater. But culturally, you have to adopt different strategies. And so, you know, I've certainly had to adapt as a leader, from a founder and CEO of a small business, to an executive at a big global company. So that's certainly been a personal journey for me.
JS: Yeah, I'll add to it. I mean, the culture part is huge, but really an exciting one. Equinix has a great culture of being really grounded in some of the shared values, that we had at Packet around neutrality and staying foundational, serving the technology ecosystem, but doing it with a certain purpose in mind. And I think that that kind of founding value that Equinix has in its name, has really helped with the integration of two really different kinds of companies.
I mean, certainly, Packet is not the whole story, Metal is not the whole story, it's been part of a larger strategy, which is how do we look at digital infrastructure? How do we provide Equinix value in a new way? You know, when maybe you don't want to rack and stack your own servers in colo? How do you deliver that? And that requires thinking about new customers, building a lot of software. And those changes are big within the company. So I think it's actually been pretty exciting.
From that side, you learn a lot, as Zac said, but externally, it's been huge. I mean, we're able to have a much bigger reach, that just the sort of trust that sits with the Equinix brand in the market. You know, when you're saying, “hey, do you want to move your stuff over to my infrastructure?” Most people don't want to do that lightly. You know, it's really an important decision, foundational, so having trust and scale behind that, I think is just a requirement.
ZS: But let's talk about the actual hardest issue, which is we had to go from G Suite to office 365.
JS: Don’t bring it up, Zac. I mean, PowerPoint...
ZS: Google Slides to PowerPoint, it was a very big transition. We're still working through therapy.
MS: I can relate, that's a big one. So I’ve chatted with with Jacob, and just for our listeners: Equinix Metal is live in 18 sites today, and is going to be 27 before the end of the year. So if anything that you've heard about here sounds exciting, you can really go out, try it, this is a service from Equinix, officially launched last year, right?
JS: That's right. We kind of re-branded the Packet service; so one aspect was re-branding, the other was really to move it into Equinix data centers because what's special about Equinix Metal is the Equinix part, you know, otherwise, it's just bare metal servers. We can talk about that more in a minute. And then adding interconnection capability, integrating with other products like network edge and fabric, and tapping into that Equinix value. Have we launched it back in, I guess it was October 2021.
ZS: No, no, October 2020, Jacob.
JS: Gosh, years fly. So we've been in market 18 months, and it's been a wild ride, super, super fun ride from a from a go to market perspective.
ZS: And a little bit of a dream come true for me because I've been an Equinix customer since 2001, throughout my career. And now customers can access that global reach of Equinix, which you can get almost everywhere. And you can do so with the interconnection, that ecosystem access that people come to Equinix to enjoy. But you can do it by logging on to Equinix Metal, putting in your username, swiping a credit card, and 15 minutes later be active in 20 markets around the world with neutral infrastructure. That has not actually been possible, for at least my part of the career. So it's really exciting to see that happen. And for customers to be able to experience that global reach and neutral access of Equinix, but with a whole new methodology.
MS: It sounds like a very easy, convenient way to interact with this technology. But there was obviously a time when all servers were bare metal servers, before hypervisors, and all of the things that we kind of layered on top of it. But today, we live in an age of cloud computing and everything as a service. So who are the customers signing up for bare metal today? And why are they doing it?
JS: Max, let me start with this one. So first of all, I like to joke that no one really cares about servers or bare metal. And I mean that in a humble way. People care about what they can do with that.
ZS: You know, most people don't care about data centers either.
JS: I mean, there's a there's a select few of us that they really love data centers. But that's not the value per se, it's the transaction method, is the way that we monetize where we give access. But I think that the kind of thing that's special about bare metal is choice, it represents the ability to define, hey, do I want multi-tenancy? Where do I want it? How do I want it to work? 100% choice.
That comes with responsibility as well: you have to manage more than you would. And so, really finding that balance of what I would call cloud experience, which you can loosely call programmability, repeatability. Some people have a definition of cloud that includes a lot of stuff, like voice recognition as a service. We don't see it that way; we see cloud experience, at least for the part that we're working on with digital infrastructure, is about automation, programmability and scalability. And then there are things on top that the cloud is complementary to, right? Cloud is a major partner to us as a business, but where most of our customers want to interact is between cloud and edge. So, bare metal, interesting, great trend that's coming back again, it's going to be cool, you know, but in the end, it's a way to have opinion about your stack.
MS: Absolutely. But do you have a preferred customer type? What kind of organizations typically go for bare metal services? Is it more enthusiast market? Software developers? Maybe open source guys? Are large corporations also interested in this, and do they have applications for this?
ZS: Well, I mean, we think about customers in two ways. And Packet started with the GitHub data model where the user is first, right? Who is the actual practitioner, touching and accessing the infrastructure? And so in that regard, Equinix Metal really attracts a software-empowered, automation preferring customer, customer who wants to automate infrastructure. They're also interested in more fundamental opinions. So generally, you've got developers, DevOps leaders, etc, who have a strong amount of opinion, as Jacob was mentioning, around their stack. But they still want to have that gating option of automation, because they're doing a CI pipeline, they're using TerraForm, they want to automate the entire viewpoint, including automating the data center, and that's really what metal gives them. It gives them a way to instantiate, from the very time of a boot process of a machine, the physical data center. And so you can really have full control over your software stack. So I would say that's first and foremost is users who are developer-oriented, or DevOps-empowered, and appreciate and are attracted to Equinix.
Those customers work everywhere, of course, at all types of companies. But in terms of the actual core target user, these are digital leaders. These are customers who have a strong investment in infrastructure. They're looking for global reach. So that's service provider platforms, content platforms, media, other types of what I'm going to call it verticalized enterprises, that are already in technology. Atomotive or broadcasting, anybody who's got high amount of investment in technology around their business. But comma, they have to value automation. If they don't, they're already doing something else. They're okay, there are other models and ways to access infrastructure. But that seems to be the limiter, users who value automation, enterprises that are investing heavily in technology and need that global reach with that programmability.
JS: Let's talk about that, though. Because there's also the broader IT and enterprise market, which is a big part of Equinix customer base right now. And they benefit a lot from, like you said, closing out of legacy strategies and legacy infrastructure and moving to the next one, call it cloud or call it operated, whatever you want.
We did that Equinix with data centers a long time ago, people used to build their own data centers, now they've generally decided it's not worth doing that, most of them. And then there's this next transition around, well, “can you operate more at the infrastructure for me?” I mean, internally, we call our service hardware as a service or data center as a service. And that infrastructure as a service layer is really, [important] you know, the software partners and the cloud partners that we have. Say, people want to run VMware on top of metal, they want to run Kubernetes on top of metal, not directly consuming metal. And so I think that's a nuance that we're facing as an ecosystem, like Equinix always has. We see a lot of partners enabling those higher level use cases that are certainly finding their way to us. But usually, with a partner, it's kind of a one plus one equals three thing.
MS: Yeah. Okay, just bare metal alone is not gonna give you what you need, you also need a software partner. But one of the things you mentioned is that you you actually developed a lot of software at Packet and I guess, Equinix now. It's an unusual idea to think about, Equinix as a software developer. So what kind of tools are we talking about? Is management and optimization and uptime?
JS: All the software...
ZS: All the software, all the time. We have about 300 people working on Equinix Metal today, including a significant broader investment in our platform services and related, here at Equinix. Think programmable interconnection like fabrics, NFVs, network edge, etc. But in Metal, what we're focused on is…. My wife probably thinks “you're so boring.” But I'm boring about the same thing I do at work every day, which is: we're just focused on automating fundamental hardware. We've been doing that for seven years. And frankly, for me, it's still an exciting journey, there's so much to do.
When you think about making physical hardware, no matter what it is – we're talking Dell hardware, some new Nvidia thing, this card and that – there's so much to do to make the state and management of that repeatable, so that a software user on the other end can harness the power of that technology, but not have to understand all the systems integration part of it, right? Does this optic work with that cable, work with this switch, work with that NIC. work with that firmware?
What we're doing is we're writing constantly that fundamental automation framework, and we've open sourced as much of it as we can. So, putting it in the Linux Foundation through CNCF, our workflow management and state control related to bare metal hardware, it's called Tinker Bell. Putting out most of what we can related to firmware control, base management controllers, etc. And then the orchestration of the experience, so our customers can get a great portal, and a great API, that touch all of that.
Now, I would say that, beyond those base-level experiences, and constantly trying to move the ecosystem forward for making hardware just work, what we're also dealing with is just a scale problem. How do we get this into a few 100 data centers? How do we scale software, independent distributed systems, working across, you know, 40 markets, and 1000s of different customer deployments, both public and private? That is in and of itself a huge portion of what we do.
JS: Yeah, I would say that as a service operator, tooling is probably where we spend most of our time. I mean, people pay us for the value, but no one rewards you for like, “oh, my gosh, you provisioned a server for me.” So much of it is the expectation. I mean, it turned on, congratulations, right? The ability to be an operator at scale involves really boring but very important things like billing, metering, rating, price, booking invoice, at scale for a zillion things in a different ways. Things like asset management, knowing what all the things are and where they're at and what state they're in.
ZS: Which MAC address goes with which NIC and what hardware to what switch, everywhere.
JS: Certainly, you know, what most companies at scale have looked at [this], there's a lot of crusty old open source tooling around to prove that. But the hyperscalers, of course, have have invested in and really lead lead the pack there. I think where we differentiate, and what we're trying to do, first of all, what can we do in the open? This kind of question, because we want digital leaders to be excellent at digital infrastructure, make that market better, bigger, and we think Equinix is going to be the best place for them to do it.
But also just the idea that we're looking at a heterogeneous environment, people come to Equinix because they want an opinion about that. Like where it is: “oh, it's gotta be an NY4, next to NASDAQ.” That's not a generic ask, right? Or I really need it to scale in this way at this cost, or reach these users. Those are all very kind of specific differentiators. That makes the infrastructure that we're helping our customers operate very different. And so, it kind of pushes us to build automation that's much more foundational, and also much more flexible.
MS: Makes sense. And certainly, there are attractions to Equinix that you just can't find anywhere else. And you've touched upon this, but where does bare metal fit into hybrid cloud strategies? Because in January, for example, network specialist Aryaka reported on a survey of 1,600 IT professionals that found that more than half were planning to close all of their on-prem data centers in the next two years. Does this align with what you're seeing, this extinction event caused by the pandemic, for on-premises data centers? And how will this change the data center industry? Or do you think that like with all surveys, they might have made the problem seem bigger than it is?
JS: Well, this was happening before the pandemic, certainly Equinix had a front row seat to people closing their data centers and looking for a new way to do that. That's been a trend that we've been a part of, and a lot of that has been about finding their hybrid home. Like, where are they home rooming their network assets? Where are they home rooming their data? And then, obviously, how do they reach the places they're trying to reach – clouds, and users, connected vehicles, whatever. We've been a part of that story and the pandemic, just kind of pushed the gas on, as we all saw.
To me, and I'm fairly new to the data center industry – I came in for the classical music seven years ago – I don't have the same vantage point that Zac does, but I don't think the death of the on-prem data center is really a problem for Equinix, and even for customers. It's about what are you best in the world at. And are you going to be best in the world of operating your data center? You know, for some people, that's a yes. And for a lot of people, it's a no. And as we move forward, our job is to figure out well, where can we really fit? And where do we not go?
We made a very specific decision early in the journey with Equinix: customers are like, “could you make it easier doing VMs and Kubernetes for me,” and a lot of people are saying we should consider that. And we have partners who are called VMware and AWS, who are really good at those things. And let's be really good at what we're really good at. And so I think that's part of our view of how this can can go forward is that it's going to be what we call an ecosystem of providers. You're going to assemble the components that you need, and that's going to include a very diverse and evolving group of things called software and hardware and networks and whatever. And our job is to help to enable that to happen and to connect it all.
So, I gave you a little bit of our view, like where does bare metal go? I think it gets sexier. I mean, silicon is pretty cool right now.
ZS: And there's kind of an overarching trend there, where we do hear it from customers who are looking for hybrid cloud, why they want to move out of siloed on-premise data centers, where they're having to bring all sorts of connectivity and technology and ecosystem to them. That's very expensive, especially considering how many ecosystem partners most enterprises interact with. I need to talk to Salesforce and I have to have a VPN that goes into this, and I need private connectivity to AWS, and so there's just a general movement out of - I'm going to call it siloed on-premise data centers – into neutral, interconnected ecosystem driven data centers like Equinix.
But what we see is a movement where people are asking to remove the friction: global supply chain challenges, flight restrictions, how do I get my onsite team to Melbourne to rack and stack this, so I can get my two racks of things going? And then there are longer-term trends around building and operating that in the most efficient and sustainable manner. Things like, “help me track my assets in a secure way, help me dispose of my technology in a manner that's responsible, help me run it with increasingly improving PUEs and less energy. Oh, and could you help me measure that? So I can report on it to my shareholders or to my governance boards.”
And so there's a lot of movement for enterprises to say, help make it easier for me, while still giving me that choice. Because you know what, I'm invested in Dell Technologies, or I'm a strong user of Pure, or I really love my NetApp software that's tied in with all these things. And so they want that choice. But remove the friction along the way.
MS: Yes. And there was this expression, I'm not sure people still use it, “one neck to choke,” right?
JS: One nose to punch! It's much more peaceful.
MS: There is an actual company called One Neck IT and that was, you know...
JS: I'm surprised they're still alive. That's the question: is one throat to choke, one nose to punch, a reality? For infrastructure, probably not. But can you have primary relationships. And can you have things operated for you that you don't want to operate. I think that's where people are making decisions. And that's why we see multi-cloud and other things. Because there's a lot more riding on these decisions than in the past. And to say, well, that's just our app, you know, that runs our thing is not true for most businesses anymore. It's much more central to their business. And so they have a lot more invested in how it works. When it moves, how it evolves, all that stuff. And it' is moving beyond software.
When I jokingly said bare metal gets a little bit more sexy, it is because software is working its way down, and people are bringing more of their software opinion with them. And inevitably, where physical and digital, or hardware and software, kind of intersect is where a lot of the magic happens.
MS: Absolutely. Probably my last question: so you've mentioned sustainability, and sustainability is obviously a massive area of interest for Equinix, that's what most of their interaction with the market these days is about. If you get a press release from Equinix, half of it is going to be about things like, “this is how we made this facility green.” They're doing things like tapping into groundwater, that's the latest facility in Munich, will be using groundwater for cooling. And all of their new builds have greenery on the roof, and other things like that. So obviously, you're not involved in architecture, you're not going to design the green roof over the building, but what can you do on the bare metal side to improve sustainability? And so it is demonstratable, and people can take it to their boards, to show that they're doing doing the right thing?
ZS: This is an area of particular passion for myself, and also for the leadership at Equinix. It's not an accident, that it's a big part of what we do. Because internally, we're spending a ton of time on it. The place where it becomes interesting, beyond our overall design and construction, our own internal goals of scope two, and scope, three emission reporting and reduction by 2030, etc, is that when we're in the business of helping to operate the hardware. We can take the building systems that before, in a multi-tenant data center, we could only take it down to the cage. We could help you with cooling and power, we give you power whips, here's an A side and a B side, we've got these generators out back, and we've got cooling, which is general purpose.
But once we can help operate the computer, we can bring those building systems all the way down to the machine. And so, we've been focused on an open source initiative within Linux Foundation called Open19, where we work with OEMs, such as Cisco and Supermicro. and ASRock and Inspur, as well as supply chain, like Schneider Electric, and Vertiv, and Molex, and Amphenol, and cooling providers like CoolIT and ZutaCore, and the industry as it were. And working with partners to have open standards around pluggable liquid cooling, and power management.
What we feel is the opportunity here is if Equinix can help our customers to operate hardware within our own facilities, no matter what it is, whether it's from an OEM partner or it's something that you brought in. Can we give you our scale of investment around things like pluggable liquid cooling, where suddenly we can have closed loop cooling instead of evaporative cooling? And offset or remove the evaporation of millions of gallons of water for a facility? Could we reduce energy consumption by 30 or 40% per machine because we're not putting in fans? Could we move to 48 volt power and give you software to find power that dynamically switches, so that if you needed high availability, you got it? And if you didn't, we wouldn’t have to build N+1 for every power circuit?
So there's a tonne of innovation that we can do, and I think we have to do it as an industry together. There's no way any one company can solve these sustainability problems, and no verticalized proprietary solution is going to work because you can't access it everywhere. We're really interested as Equinix to work together, using our ecosystem-driven approach in the open to create these very disruptive ways where we can have sustainable or even net-zero data center and IT footprints.
JS: Yeah, and I'll just close it out, Max, because the difference, like Zac mentioned, now we're operating the computer, flows through a lot of our learnings. You know, bet a lot of people probably looked at Equinix when they acquired Packet, and went like, “risky, going into the cloud space, eh?” Actually, just think about what our customers do right after they get a cage – they put in servers, interact with the network, and that's what we're doing. And just by moving that one step up, we have so much more empathy, and insight, and responsibility for what our customers do every day, across tens of thousands of racks, which is put in computers and networks, and operate them, and cool them. And so, we have a front row seat to some of the customer pains.
ZS: Even as simple as getting servers into the data center and having to get rid of the packaging. Can we work with the supply chain to have reusable packaging? Can we do that? That is something that we can actually help our customers with, and also help ourselves with. So there's this like, nice virtuous cycle now that we're there, sitting next to our customers, doing this aspect of “down to the server” that we can really bring to bear, our global real estate footprint and supply chain access, to improve.
JS: It does optically look like we're getting into a new business, but it’s the same business, same foundational thing, really – enabling customers to reduce friction. And then these opportunities are really for innovation. Right? And I think that kind of open like we talked at the beginning of the interview, that sort of open neutral ecosystem ethos that was the founding of Equinix and persists today, that's what drives how we're going to do it, and why we're going to do it. Versus just like, “hey, there's a market there and people want to buy computers.” Of course they do, it’s a digital world. That's kind of the short story long, if you if you might call it, so thanks for letting us speak about it.
MS: It's a very ethical view on a data center business, so I'm rooting for you guys. Best of luck in your business. I'm afraid this is everything we'll have time for today. But we will absolutely will continue watching what happens to Equinix Metal and bare metal computing in general. Good luck with rolling out those additional locations. And hopefully, see you on the show floor, or at an event somewhere, and let us know if you're building something interesting.