4 ways flash storage is reinventing your data center operations

public://pictures/Christopher-Null-CEO-Null-Media.png
Christopher Null, Freelance writer, Null Media

The storage market is in upheaval, and it’s all thanks to flash.

Sales of both hard disk drive (HDD) and hybrid flash/disk drive arrays from all the major vendors are either flat or in decline. In their place: all-flash arrays (AFAs), sales of which are growing at double-digit annualized rates, according to 451 Research analysts. With the need for high performance and availability commonplace in organizations of every size, HDDs are increasingly unable to keep up, and flash is supplanting it more often.

“The fundamentals of storage remain unchanged,” says Andrew Grimes, principal architect for flash at storage company NetApp. “You still need to serve data with reliability, to protect your data, and to deliver IT services across a range of requirements, regardless of the media under the data. But as flash technology matures, it lets us change the way we deliver those services: with greater efficiency and performance and with the potential for major savings for your organization.”

As flash takes over the data center, it’s a good time to look at four trends shaping that invasion.

The Forrester Wave: Enterprise Service Management 2018

1. Flash costs continue to plummet, making AFAs realistic for more types of businesses.

Why is flash overtaking HDD technology now? Flash prices are seemingly in free-fall. As of mid-2013, a consumer-grade, 240GB solid-state drive (SSD) cost about $165, or 69 cents per gigabyte. Today, that same drive would cost roughly $60, or 25 cents per gigabyte. For companies that have been sitting on the sidelines, waiting for flash prices to become sufficiently attractive for enterprise adoption, that time is here.

“Finally, the cost of solid-state drives has reduced to the point where many enterprises can now truly consider all-flash arrays,” says Ahmed Abdalla, solutions architect at security consultancy Adapture. “Advances in SSD reliability, performance, and capacity have led to this lower price point. While price parity for traditional spinning disk is not available yet, the price/performance trade-off is definitely a winning consideration for many enterprises.”

We’re not at the end of the line, either, thanks to another major trend.

[ Webinar: How Garanti Bank Delivers an Excellent End User Experience (12/6) ]

2. Hard drives have maxed out their performance, but flash has not.

The biggest news in flash storage today is 3D NAND flash, a technology being heralded as the biggest revolution in memory technology in more than 40 years. 3D NAND, as the name suggests, adds vertical layers of storage tiers—up to 64 at present—on top of the NAND die, which enables the chip to scale to much higher densities than have previously been possible with so-called planar NAND. That advance means that flash is now surpassing the storage density of HDDs. Intel and Micron are leading the charge here, and both are promising products that are vastly faster—these companies claim that their latest technology is roughly 1,000 times faster—than traditional SSD storage, with far greater capacity and lower prices to boot.

Meanwhile, although HDDs continue to improve capacity through advances such as heat-assisted magnetic recording (HAMR), HDD technology is near the point of hitting the performance wall. To further increase HDD performance would require faster and faster drive platters, and that isn’t likely to happen, due to the extreme amounts of power that would be required to spin media beyond 15,000 rpm, not to mention the resulting heat and reliability issues that would likely plague drives operating at such speeds.

In contrast, benchmarks show that even a cheap SSD is faster than the fastest HDD, virtually across the board. Thanks to advances such as 3D NAND, we’re going to see SSD performance continue to improve as these technologies become commercialized.

3. Flash arrays are improving their ability to meet reliability guarantees.

Flash has always been besieged by the known issue that data stored in flash has a finite lifespan—that sectors become unusable after a certain number of overwrites. Lifespan varies considerably from product to product. 3D NAND promises improvements in the drive's lifespan or endurance due to the use of better insulating materials and other basic technology improvements, but manufacturers have largely dealt with the issue of degradation through clever product design tactics, including overprovisioning to allow for dynamic reallocation of bad sectors.

What has emerged is a market with wildly variable pricing based on overall drive endurance. “The lower the endurance, the lower the cost,” says Gaurang Mehta, CEO of SANS Technology. “Endurance is measured in terms of drive writes per day (DWPD) or total bytes written (TBW). When a vendor offers an amazingly low price for an all-flash array, the potential buyer has to be completely certain of the quality and endurance of the drive media being used and should ask for this data from the vendor prior to buying.”  

Things are getting better as flash arrays adopt true enterprise data management features. “Shared multitenant flash arrays are going to be the standard,” says NetApp's Grimes, “and if a vendor can't innovate to that standard, its days are numbered. To support these changes, IT consumers of flash will need all the tools and features of traditional HDD arrays merged with all the benefits of flash. The performance is already in place. Now, it's a race to give the enterprise the rest of what it needs for enterprise-grade flash.”

4. Flash may already have a successor in place.

Ali Hodroj, vice president of products and strategy of high-end application server provider GigaSpaces, notes that a new solid-state storage technology is already on the rise: NVRAM, or non-volatile RAM, a RAM technology that maintains its state even when power is turned off. “NVRAM is faster than flash but slower than RAM and will assume the position that flash once held as high-speed specialty storage,” he says. “These advances matter because they reflect an advancement to high speed coupled with high efficiency and reliability, mainly to service ever-increasing Internet data demands, real- and near-real-time analytics, and forthcoming machine-to-machine or IoT applications.”

NVRAM is considerably faster than flash, and it doesn’t suffer from the same endurance problems. At present, no pure NVRAM enterprise products are on the market, but a few hybrid NVRAM/flash devices are. This includes Flashtec's NVRAM systems, which promise a 10x performance boost over a pure flash solution.

Ultimately, mass-market NVRAM is almost inevitable. “Continuing advancements will put it on the traditional Moore's Law curve," says Hodroj, "and it will gradually start taking on more functions previously delegated to flash. This process will be years in the making, however, and doesn't represent a rationale for deferring flash investments.”

Hard disk drives still matter

Reports of the death of the HDD appear to be greatly exaggerated. Despite the promises of the previous four trends that are serving to push flash to the fore, hard drives are likely going nowhere in the short term for one key reason: They are simply still too cheap to ignore as an option for enterprise storage.

As previously noted, SSD prices have plummeted in recent years, and seemingly every year, pundits predict that we’ll hit price parity. It hasn’t happened, and it isn’t likely to occur until at least the end of the decade, as a sizable price gap still remains between SSDs and traditional hard disk drives. It’s a significant one. On a dollars-per-gigabyte basis, flash storage still runs about four times higher than that of HDD storage.

Forward-looking reports have projected that SSDs will eventually supplant HDDs, primarily based on calculations of total cost of ownership, since operating a bank of HDDs requires more power, cooling, and physical infrastructure than SSDs. Despite this, acquisition costs remain far less for HDDs, and they keep dropping. Like it or not, acquisition costs remain the primary metric that IT buyers use as a source of comparison in evaluating storage. With a vast amount of non-essential information sitting in archives, bleeding-edge performance isn’t a major factor for every piece of data out there. Traditional drives will serve a vast section of the market relatively well and affordably for years to come.

Jay Kidd, retired NetApp chief technology officer, predicted in 2014 that, while flash use will run rampant in the enterprise, the idea of an all-flash enterprise was “utter nonsense.” Kidd predicted that 80 percent of corporate data would remain on HDDs for the foreseeable future. “Every storage architecture will incorporate flash to serve the ‘hot’ data,” he wrote, "those that provide only flash storage—without an ability to integrate with hybrid flash arrays and legacy disk systems—will be like hot rods in the garage. Fun to tinker with, but not versatile enough for the vast majority of IT workloads.”

Flash will eventually take over—but that switch may take more time than the industry thinks.

Whole lot of shaking going on

The storage story isn’t over, as the industry-shaking acquisition of SanDisk by Western Digital in late 2015 demonstrates. The market transition is creating new and interesting alliances. What will come of these or who the winners in the space will be remains unclear. Some industry observers think these mergers may actually slow down the rate of change and arrest the fall of flash memory prices.

The endgame, however, is inevitable, as flash prepares to enter that ugliest of tech marketplaces: the commodity market. At least it’ll get there in record speed.