Iodyne makes solid state storage devices for creative professionals in the entertainment and media area and sees an Edge 2.0 era arriving because of the AI-level performance of Mac Mini type systems twinned with large amounts of fast storage. You might think iodyne would welcome high-capacity SSDs in the 256 TB area but you would be wrong.
We had a discussion with Mike Shapiro, Co-Founder and Co-President, iodyne, about this, but first let's review iodyne's product line to set the scene.
The Pro Mini is a handheld, smart phone-sized device with 4TB and 8TB capacity options. It’s bus-powered and links up to a source device, such as cameras, or a host using USB-C or Thunderbolt and receives/transmits data at up to 3 GB/sec. It does not contain an off-the-shelf SSD as Iodyne builds its own drives The NAND chips inside are cooled using AirJet technology to prevent throttling due to heat buildup and the device has a customizable e-paper label.
The Pro Data is a laptop-sized drive with 12, 24, 48, 96 and 192 TB capacity points and 8 full speed Thunderbolt ports. It supports up to 4 simultaneous readers and daisy-chaining to multi-petabyte scale. The system supports RAID 6 or 0 and delivers up to 5 GB/sec sustained throughput.
These are high-end, studio-level systems for professionals with features tuned to their needs and not simple off-the shelf hand-held or portable SSDs. Why wouldn’t iodyne relish having 256 TB and up SSDs inside them?
Shapiro said: “I don't think those are of interest to us, particularly … as … the form factors that we're focused on are a little bit different in the following way. In the phone size [Pro MIni], you're really looking at, we have an 8 TB out today. We'll certainly be able to take that to 16 TB. That's obviously using actual just chips, sort of an all- in-one custom design, to do all the things we want to do in the Mini. And I think for phone size devices, 8 TB is a lot of storage. 16 TB? I certainly think people are going to need that over the next year or so for certain use cases, but that's really a lot for [a] portable.
“If you look in the other kind of form factor, our larger Pro Data product for edge … that's really where you benefit from having RAID and erasure coding to get more reliability. [But] individual drives that are that big [256 TB] create too much of a blast radius. And this is a problem that we've known about going back to the disc drive era. … You can always take the latest NAND technology and in particular right now, things like a 2 Tbit QLC die, which are now available. It's an incredible density. You can put a lot of those chips into one of our standard SSD form factors. But if you're losing that much data, if something goes wrong, what’s your rebuild time?
“To me as a storage designer, it's always an art of saying what is the customer trying to achieve from their usable capacity? And then what we're looking at is where is the most efficient place on the RAID protection curve to deliver that much capacity? Now, if you were in a data centre context, you might answer that question one way. If you're in a edge context, which is where iodine plays, you might answer that question a little bit differently, but it's always about that place in the curve.
“The place that I think those really large blast radius drives may have some value, those really high density ones, is in data centre and in specifically with AI workloads. And that's because, if you think about things like caching and training datasets, that's not your only copy of that information or it may be a cache, in which case at the end of the day, you don't necessarily need to rebuild it the way you would rebuild if that were a drive containing my Oracle database or something like that.
“Do people need 256 TB at the edge? Absolutely they do, but we're going to do that with slightly smaller pieces so we can have a great story around RAID protecting your data. [It’s] really, really important at the edge because you just captured something that's probably your only copy of it. So it's critically important. If you spent 24, 48 hours capturing a video, a network trace, an oil and gas field scan or whatever you're doing out there, that's critically important information. You've got to protect it with encryption, you got to protect it with RAID ,and you got to be sure above all else that we don't lose it.”
In other words, you just cannot lose that data because if you do, you've got to reshoot or re-scan what you just did and that may simply not be possible.
Shapiro said: “Exactly. It may literally not be possible because that may have been an event that happens once, or to your point, it may be something that's just simply cost-prohibitive to reproduce. It could be 10 or 100x the cost of the storage infrastructure. So we're not going to go that high right now, but we certainly have customers who need that much usable capacity.”
Are high memory prices affecting iodyne?
Shapiro said yes they are: “It's an issue for everyone. I think there's a couple things [to consider]. One is, it's very important to have extremely strong long-term supply chain relationships, critically important. So most NAND manufacturers right now are not accepting new customers. If you and I were starting a new startup today and we had a new product idea, you'll be politely told, "I'm sorry, we'd love to have you as a customer. We don't have any ability to have you as a customer.””
His company has long-term relationships with NAND suppliers: “Myself and my team, obviously we have decades of relationships going all the way back to Sun Microsystems days. It’s really important and, that way, we can at least as a smaller player still have top-tier relationship status. We are going to get allocation. We're a going business. We have existing contracts with them predating this sort of NAND apocalypse or whatever you want to call it.
“The second thing, which I think is really important right now in our space, is the value that comes with storage. So meaning value from things like RAID and encryption and other unique features. So a lot of the commodity drives where you're just buying a drive, if you think about it a year or two ago, those drives were way lower in price, but they were just commodity drives.
“Now, if you come to today, if you're a customer, that same drive is still a commodity. It doesn't do anything different for you. It's still just an SSD, but now it costs five times as much. So I think that's very obviously dissatisfying to the customers. What's important is for that extra price now, are you delivering more value?
“At the start of this year, customers did not understand that this was going to be a long-term phenomenon. They thought, well, prices have gone up, I'll wait a quarter and they'll come back down. And now the customers, when you see Tim Cook giving an interview in Wall Street Journal saying that this is a 100-year flood, it's starting to penetrate not just the tech … community, but also the broader community of customers. They're understanding now this is a long-term phenomenon.”
“We feel good about where we are, because every business is experiencing the same component price raises. Even Apple is paying more. Even the biggest folks are paying more. … It’s a rising tide for everyone."
He thinks the value-add of iodyne drives will be a big plus: “In our world of the edge space, security matters a lot, data protection. We introduced device pass keys for the first time in the industry with Pro Mini this year, which we're really excited about. Using pass keys to unlock data in the edge; we think that's a great innovation. These are the kinds of things we're trying to do where we think customers will have more resonance with the idea of, yes, I have to pay more, but I'm actually getting more.”
The iodyne market
The drives iodyne produces started out being attractive to video media professionals, the studio market segment, traditional studios, the Apple Studios, Netflix, and which now includes the top 10-20 percent of YouTube producers. Shapiro said: “They're almost running like movie studios. They're making many millions of dollars, like Mr. Beast, for example. He’s an iodine customer of our Pro Data. He's the top YouTube, at least in the US, he's the top YouTube creator by views. He's basically running a movie and television studio of his own.”
Live Sports is a key iodyne market. Shapiro said: “Live sports is a huge growing market. We have a lot of great sports customers, Dallas Cowboys, for example, in the US, San Francisco Giants. I've done a bunch of stuff with other NFL people and piloting with other leagues as well. … sports is a big one and then also corporate video. If you look at corporations and Fortune 500, especially coming out of COVID, how do you communicate with your employees? Well, they do it through video now. They've learned that video is an essential part of not only communicating with customers, but also communicating with the employees. The corporate video market is growing significantly as well.
“Where a lot of our growth is coming from now is taking what we've learned in M&E and expanding that to new data-intensive edge verticals. These are places like cybersecurity, energy, legal, manufacturing, and I think we'll ultimately include places like healthcare as well, because there's just a lot happening at the edge now that's incredibly interesting and we call that sort of edge 2.0.”
He noted: “I would say 80 percent of the data in healthcare is now or will soon be imaging data. The revolution in healthcare is imaging and other kinds of scans. Now, can I take those data associated with you, the patient, and can I feed that into new AI-driven software to help with diagnosis? And can I have my cancer specialist look at five scans by hand but also be supported by dozens of AI expert models that know how to look for tumours and guide that physician to make a better diagnosis?
“Or can I take a DNA sequencing data of my patients and use that to determine which of three different drugs might be the most effective ones? I think that healthcare is being revolutionised by data and a tremendous part of that is now going to involve capturing data at the edge, meaning in places like hospitals and clinics.
"That's a huge opportunity for new technology, new AI software. If we think about it from the storage perspective, where we immediately go to is; I have all these things at the edge I have to capture, but then I immediately I have to protect it. And then security is incredibly important, security and privacy. We think that's a huge opportunity for us as well.”
Edge 2.0
Shapiro said: “When I think of Edge 2.0, I [don’t] think of anything that is in a traditional enterprise data centre. I don't think a hospital is really. They do have a server room or something like that, but it's really not a traditional enterprise data centre, certainly not a hyperscale or cloud type of data centre. When we talk about edge 20, we're thinking about everything from movie set, hospital, oil platform, military base or embassy, airport or satellite.”
They'll have a scanning system hooked up to a workstation and ProData could be there sitting between the scanning system and the workstation?
Shapiro: “Absolutely. And obviously future things on our roadmap as well, but, absolutely, those are that type of a product that provides great reliability, security and can connect to multiple computers.
“I'm sitting here [at] a little coffee table [and] I could put a ProData and four Mac minis plugged into that Pro Data and it would fit right here in front of me on this couple square feet of space. And if I had that today with our top Pro Data, I would have 192 terabytes of RAID 6 encrypted storage and I would have hundreds of gigabytes of unified memory and that think of all the GPU and NPU capabilities of four latest M5 Mac minis. I mean, that's just incredible. That would literally fit on the table in front of me. So I look at that and I say, that's a real revolution and if I can capture data and feed it into those new software stacks, what can I do with that?
“We’re looking at, of course, on a movie set is a place where you do that, but also what could I do with that in an aeroplane or on a boat or in a hospital or all these place? I think this is where we at iodyne are focused, as this edge 2.0, because I think this new software, this ability to process data in real time and do things with it and take action in real time or eventually bring it back to my cloud where I can feed It to even more powerful AI is a real revolution that's coming and that's what we're focused, on the edge part of that.
“Why is that resonating? Well, it's really for two reasons. One is because, in M&E, you're not just capturing image data now, you're capturing 3D photogrammetry, you're capturing LiDAR information. Capturing information for a movie is not just about visual images and sound anymore, it's also about these other things. So you immediately need things like, "Hey, can I have enough sustained performance to have multiple things streaming into storage at once?”
“Then the other thing which they're critically concerned about now is IP protection. We just had the NAB show, which we have in Las Vegas in the spring and we'll be in the European version IBC later in the fall. And I would say the number one difference this year between last year and at this show was last year was; AI is coming, everyone's scared, what does this mean and so forth. This year [has seen], very different view of AI in the M&E industry. People were talking about how they had actually used it over the past year. They had specific things they were doing with it. They saw benefits and opportunities in the future, but the number one issue was how do we protect our IP in the context of this new world? We can't just take our images and our IP and all this stuff we capture and just feed it into somebody else's cloud AI engine because then they're going to train it on our IP.
“That's how we make money. We licence our IP. Protecting data, data governance, data sovereignty, understanding how this stuff is labelled, keeping control of your data, and then being very careful about where it can go as you take advantage of these new tools. That was the number one issue foremost in the mind of all the people, those studio segments, high-end folks in the M&E industry.
“Security, I think, is critical across all those pieces of edge 2.0 in this new world where your data really matters. And if you're a person who's capturing valuable data, whatever industry you're in, keeping control of that so you can control and monetize potentially how that feeds into all these models that people are using, it becomes critically important because, if you've done the work to capture data, it could be incredibly valuable and in ways that you don't necessarily even realise yet.
Power usage
Shapiro considers power usage to be a third important factor in his edge 2.0 world along with performance and security: “I cannot overstate how important power management is and power reduction is in edge context. And that should be, of course, familiar to customers from things like how much work is done on in say mobile phones and also things like Apple Silicon for laptops. So for PCI Express, from an edge context, in enterprise, they're always saying, okay, give me the fastest possible SerDes. Let's keep cranking up bandwidth, PCIe 5, PCIe 6, and so on. In edge, we look at it a little differently, ... If I bought a 96 lane PCIe Gen 5 switch, I have way more bandwidth than I could ever use in an edge device.
“The problem is I can't power it. So we look at it very differently, which is we have to look at how many milliwatts is it going to take to deliver a particular bandwidth level? … I could have four lanes of PCIe gen 3, that would be four gigabytes a second. I could have two lanes of PCIe gen 4. I could have one lane of PCIe gen 5. Now, those are all the same bandwidth because I'm doubling the SerDes speed, halving the number of lanes. What those now are not equal on though is, they're not necessarily in a given chip generation the same number of milliwatts. So those SerDes are burning different power levels to deliver that same bandwidth if you look across PCIe 3, 4 and 5.
“For us, it's less important what the generation is and more important what the power number is of a particular PCIe chip that is delivering that. That's critically important for us. We're always looking at balancing lane count and the SerDes power efficiency.
Controllers and CXL
Iodyne has its own SSD controller technology for the Pro Data and Pro Mini: “In the case of ProMini that obviously let us build a custom PCB that has all of an SOC security technology and e-paper display, secure Enclave and all these other, and NaND chips, and all build that onto something the size of a mobile phone. That kind of integration was necessary as opposed to just taking an M.2 [drive] and wrapping it up in a chassis as others have done.
“We think there's some real strategic advantages for that over the long term, and so we worked very hard to bring the first version of that to the Mini.”
What about CXL?
“The main problem that was hit with CXL over the past couple years is that CXL is based on the PCIe service and the PCIe service is just not fast enough and not capable enough to do what they needed from GPUs, which is why things like NVLink came out.
“ I think where CXL has a place potentially, and the part that I think will be really interesting for you to watch over the next couple years, is the memory area. So with the DRAM pricing that's going on right now, which is a major problem, there is another part of CXL, which is you can use it to create disaggregated pools of memory. Now that would not be necessarily unified memory near the GPU, but for example, could you use that as a caching tier, a memory-based caching tier?
“Marvell, in fact, just introduced a chip, Structera, that allows for CXL-based memory and they actually put compression, I think, into one of those so you can have memory compression. So I think that's a really shrewd move, very clever kind of idea. And I think that it would not surprise me if the DRAM pricing apocalypse, which is the worst pricing situation of all right now, creates a pressure in the market to do some interesting new things with CXL and memory.
“If memory becomes a thing that we want to disaggregate, I think that's a really interesting place where CXL systems can see some growth, because it is fast enough to do a cache or memory tier way below the SRAM that's feeding the GPU and so forth. And that may be very interesting for these large scale inference or training clusters.”
Bootnote
Meta has published a paper about a technology called Vistara. It’s designed an ASIC to enable CXL to use old memory and hook it up to fast memory and use it as a caching tier.