Data storage has an odd position within the IT continuum. Although widely recognized for the critical roles they play in computing infrastructures of every kind and size, storage solutions tend to receive less public attention than microprocessor and memory technologies. That is a serious oversight, especially when one considers how important storage is in crucial new use cases and workloads, including artificial intelligence (AI), machine learning, big data, advanced analytics and the internet of things (IoT).
In the following interview, Andy Walls, the CTO and chief architect of IBM’s Flash Systems storage organization, provides insights into how these and other developments fit into the current state of enterprise storage and IBM’s efforts. Walls is particularly well-suited to that task since his remarkable 3+ decade-long IBM career spans key evolutions in modern enterprise storage, many of which were sparked or extended by his own achievements.
In 2014, IBM CEO Ginni Rometty named Walls an IBM Fellow, the company’s highest recognition for its technical leaders. In his statement, Walls noted that when hiring, “I look for people who don’t limit themselves. They have confidence in their abilities. They know they can do more than it appears. In my experience, there are the critics and there are the solvers. I don’t want to hire critics.” He couldn’t have described himself and his work any more clearly.
CK: Let’s begin at the beginning. Could you talk about your schooling and how you got into data storage?
AW: I considered going into law out of high school, but my mom talked me into going into engineering instead. Mom certainly knew best!! In 1981, I graduated with a bachelor’s degree in electrical and computer engineering from UC Santa Barbara. The job market was so hot then it was recommended we not go for master’s degrees right away. I considered getting one over the years but just never had time. Instead, I have had on-the-job education over my 37 years and been in storage almost the entire time.
CK: How did you begin working with IBM? Where has that journey taken you?
AW: In 1981, I started working with IBM in Rochester, Minn., writing a backup and diagnostics program for tape and disks attached to the System 38. Since then I have authored over 140 patents, achieved the prestigious honor of being named an IBM Fellow by CEO Ginni Rometty in 2014, and recently became the CTO for IBM Flash Storage.
CK: That’s quite a run, and you’re not done yet.
AW: I have been in development my entire career and have a passion for working with our clients and ensuring that our products meet their expectations. I have made a career out of doing what others considered impossible.
CK: Your work has spanned significant changes/evolutions in IT, particularly in storage. How are things today different than they were earlier in your career?
AW: My very first day on the job in 1981, I was shown the IBM System/38 that I would be working on. It had 2MB of main memory, a 65MB IBM Piccolo 8-inch hard drive and an IBM 3370 572MB hard drive. I thought I had died and gone to heaven. How in the world could anyone use so much storage? HDDs now fit into 10 percent of the volume and have 250,000 times the capacity. The 3370 DASD was the size of a small refrigerator. In that space, today, we could easily fit 2PB of effective storage with the FlashSystem 9150. That represents a 3,500,000X increase in capacity.
CK: I expect those radical differences extended to software and services, too.
AW: At that time, there was no redundancy. There was just backup. If a disk failed, it was accepted that you would be down until the drive got fixed. Boy, has that changed! Today, going down for even a minute is unacceptable! Over my career I have seen that criticality increase almost every year. Then, 20 years ago or so, RAID was introduced and completely changed enterprise storage. We continued to advance high availability offerings by introducing advanced copy services.
CK: It seems like attitudes toward information have changed, as well.
AW: The key then was storing the data and being able to retrieve it. Now, data is the life blood of an enterprise. Just storing it is not enough. Clients need to make sense out of it. They treat data as a differentiator. They analyze it, make decisions based on it and rely on it for everything.
CK: How about storage management processes and tools?
AW: It is amazing how the storage software infrastructure has changed. Twenty years ago, it was basically about backup and our solution was ADSM (ADDSTAR Distributed Storage Manager). Simplified management and control was also important. It was called Tivoli Productivity Center back then (IBM Spectrum Control now). These days, software-defined storage is everywhere. Storage virtualization is critical, backup has become “modern data protection” (with all sorts of valuable bells and whistles), and our storage software portfolio is way more than just backup: CDM (Copy Data Management), scale-out file systems, software-defined storage and more.
CK: How has cloud computing changed things?
AW: If you look back just 10 years ago, the cloud was really something confined to weather reports. Over that short time period, it is amazing how critical cloud has become. Enterprises cannot afford to do everything on premises, and they also rely on more than one cloud. So they need a multicloud strategy and the ability to have elasticity to respond quickly to business changes.
CK: What are some of the practical effects of cloud?
AW: A decade ago, an application owner would make a request for servers and storage for a specific application, and it would be weeks or months before it was stood up and tested and put into production. Now, an application owner can use a cloud service and provision a server and some storage in minutes and start testing. They can provision storage automatically from their storage department for a database application or expansion of an application and be in production quickly. Time is absolutely of the essence.
CK: It’s also remarkable to consider the changes that automation has made to simplify storage management.
AW: There is an expectation of simplicity and automation today. The ratio of capacity to storage admins is increasing tremendously every year and no longer can admins provision storage for users like they used to. So, REST APIs, provisioning scripts and AI-based storage automation are critical. Customers expect storage to be automated and as simple as using an iPhone or Mac. Simplicity! You hear the word all the time.
CK: Plus, substantial changes on the media side have fundamentally changed enterprise storage.
AW: Ten years ago, everything was magnetic. Flash in the form of SSDs just came in as an expensive fast tier about then. Over the past two to three years, adoption of flash has accelerated. Even three years ago, I often had to present customers the TCO benefits and reasons for going all flash. Today, primary data has gone all flash, even in small companies, and I never have to explain why customers should consider it anymore. In the last one to two years, the largest change has been using NVMe (Non-Volatile Memory Express) solutions end-to-end, and that will continue.
CK: What changes or trends have been the biggest surprises for you?
AW: Over a decade ago I was a bit surprised at how fast NAND Flash spread into SSDs from consumer devices. But IBM and I took advantage of that trend to quickly develop ways to differentiate our systems with flash.
CK: What accomplishments are you most proud of?
AW: I am most proud of being appointed an IBM Fellow. That is an accomplishment that few achieve, and I am very honored to be one. In addition to that, I am very proud of what we have done with IBM’s FlashSystem. From the acquisition of Texas Memory Systems to the FlashSystem 900, Hardware Compression and now the FlashSystem 9100. It has been a tremendous ride and very successful for IBM.
CK: Do you have any regrets concerning “roads not taken” or missed opportunities?
AW: We were a little slow adopting TLC (triple level cell NAND flash) technology. But when we did adopt it in the FlashSystem 900, we added hardware compression as well as our secret sauce to ensure there were no endurance problems. We will absolutely keep up with the technology curve from now on. In fact, FlashSystem 900 had 32-layer TLC, and we already introduced 64-layer with FlashSystem 9100.
CK: If you had to briefly describe the current state of enterprise data storage, what would it be?
AW: Access must never go down. It must be available all the time, period. Everything needs to be SIMPLE, automated and flexible. It also needs to have low and consistent latency all the time no matter what. And it is not about storage anymore, but about data. End users must be able to analyze it, make copies for DevOps and development seamlessly without any issues whatsoever. As important as the technology is the software-defined infrastructure to make it as simple as possible to deploy and use.
CK: What are the critical points that enterprises need to remember when they consider data storage?
AW: How simple and easy it is to use and deploy are critical points, but customers should also consider how flexible it is. The FlashSystem 9100, for example, can virtualize other storage behind it (up to 440 different heterogeneous storage systems are supported). TCO (total cost of ownership) is critical, too, meaning density, power and cooling, rackspace technologies, floorspace requirements and so on. The enterprise must also think and design for high availability (HA) and disaster recovery (DR) up front. Things happen. Sprinkler heads break, power generators go down, and admins make mistakes. It is critical to have a well designed and tested HA and DR strategy.
CK: How does IBM counsel customers regarding these issues?
AW: IBM has focused on these points throughout our portfolio of storage systems and storage software products, offering many different affordable solutions. I strongly recommend that clients think about storage as an investment, not a cost. Successful businesses these days are realizing that.
CK: How about investments in leading-edge performance? What are factors to consider there?
AW: Perhaps no one is complaining about the 2ms (millisecond) response time they are getting. But if investments were made to drop that to 250µs (microseconds) what would be the return on that investment? Could real-time analytics be enabled? Could the enterprise realize savings by using fewer servers? Could end clients be more satisfied by the reduced service time?
CK: It works the other way, too, right?
AW: Yes. Long latencies are amplified due to queuing theory and can end up making very impatient end users annoyed they have to wait a couple of seconds. Therefore, think of storage as one of the most important investments you can make these days enabling analytics, faster development, quicker deployments of applications, and ensuring their cloud and entire data infrastructure is always available 24x7x365, etc.
CK: What should they be looking for in new storage products? What questions should they be asking vendors?
AW: It really is about solutions today. That is why customers should be looking for full-service providers—experienced storage systems vendors that also have a full range of software-defined storage and cloud enablement capabilities. It is very important that the provider has a broad range of solutions that work well together. Simplicity and automation are both key attributes. Additionally, in order to be able to consolidate many applications on one platform, ask about data reduction capabilities. Can it be done with hardware compression, for example, providing little to no latency impact? Customers should not look just at headline numbers like how many read hits out of cache. Look at how the solution performs with real applications.
CK: IBM has contributed enormously to the evolution of data storage. Where does the FlashSystem 9100 fit in IBM’s storage portfolio and how does it contribute to the company’s storage strategy?
AW: It is the successor to our FlashSystem V9000 and a true end-to-end NVMe storage system. It is an integrated, dense, full-function, all-flash array using IBM Spectrum Virtualize and our FlashCore technology. But also includes backup, CDM, data replication to the public cloud and container support.
CK: What drove IBM to move in this direction?
AW: The FlashSystem 9100 family has taken the best from many parts of IBM’s portfolio to offer the densest, lowest latency, most flexible, fullest function all-flash storage systems on the planet. First of all, it builds on the larger success of IBM FlashSystems by following on and being successor to the FlashSystem V9000. As such, IBM-developed FlashCore modules with hardware compression are prominently offered. We used the latest 3DTLC flash from Micron and added several differentiated features—especially hardware compression.
CK: What are some of the practical benefits FlashSystem 9100 customers can expect?
AW: The offering continues to stress flexibility by also offering non-IBM SSDs that are especially well-suited for lower capacities. In order to most efficiently condense the FlashSystem V9000, we used NVMe inside to connect the controllers to the Flash. We are already NVMe-oF enabled with support for FC-NVMe coming soon with a software upgrade. So, it is an amazing end-to-end NVMe solution with hardware compression and, of course, all the other full features of the IBM Spectrum Virtualize family. But it does not stop there. This is not just an all-flash storage system product but part of a broader business solution. The NVMe FlashSystem 9100 also comes with five different software packages offering enablement for today’s multicloud enterprises.
CK: How do the FlashSystem 9100’s individual components/features—NVMe, Flash, SDS, analytics, cloud—contribute to its overall value? Is this a case where the sum is greater than the value of the individual parts?
AW: Each one has incredible innovation behind it. Take for example the NVMe. We completely re-engineered the back end to get rid of all locks, to increase parallelism as much as possible and to get the lowest latency we could. The result is a back-end NVMe attachment which takes advantage of flash giving very efficient utilization of all CPU resources and enabled for Storage Class Memory (SCM). The FlashCore Modules were also completely re-engineered to attach to NVMe in the industry standard 2.5-inch form factor. But we also fit in hardware compression that works hand-in-hand with the IBM Spectrum Virtualize software stack. When you then combine these innovations with the five world-class storage software packages I mentioned, you have a powerful solution that is ready to consolidate workloads, enable multicloud environments, and provide an automated and simple on-premises storage management, as well.
CK: How does the FlashSystem 9100 reflect larger trends you’re seeing in both the market and among enterprise customers?
AW: First of all, it is designed from the ground up around Flash and NVMe. It is also a solution which comes with IBM Spectrum Storage software for array management, data reuse, modern data protection, disaster recovery and containerization. The FlashSystem 9100 family has also stressed ease of use and setup and has performance characteristics that allow a user to just set it up and forget it.
CK: Why should enterprises consider IBM? What is IBM doing differently from other storage vendors?
AW: IBM has the broadest set of storage solutions and technologies in the industry. Our software-defined storage portfolio is remarkable in both its breadth and depth, including block, file and object solutions. We have not just accommodated flash in our storage products but truly designed for it and focused on maximizing its benefits. We design our own flash controllers to differentiate our storage systems but also make use of industry-standard SSDs. We are not just a point product seller but a true end-to-end solution provider. Of course, it goes deeper than just storage. IBM has solutions that range throughout business IT, demonstrating that we have both the expertise and experience needed to solve customers’ problems.
CK: Are any other storage vendors pursuing technologies or approaches that you find particularly interesting or intriguing?
AW: Storage-class memories are almost ready to truly be exploited. The FlashSystem 9100 is ready and prepared to take advantage of them as the technology matures. I also find it interesting that some providers are also doing their own flash controllers recognizing that doing so can create synergies and differentiation. That’s something IBM Storage had done for years.
CK: Looking ahead, what changes do you see occurring in storage in the short (2-3 years) and long (5+ years) terms?
AW: I think we are just beginning the cognitive and AI journey. This will continue to put the emphasis on data and not the processor. Therefore, being able to move the data quickly and efficiently will become ever more important to the success of those initiatives. Within five years, storage-class memories of various types will take hold as a solid tier between flash and DRAM and, in fact, augmenting DRAM. Customers already have multicloud strategies and will want their on-premises storage to have the same properties as the cloud. Automation and simplicity have rapidly become table stakes. Storage will essentially become self-managing with everything from provisioning, to code download, to problem resolution being automated, along with many other things. I think we will also see object storage becoming used for more than archival storage, especially as flash starts to enter that tier.
CK: What is IBM doing to prepare itself and its customers to meet those challenges and profit from those opportunities?
AW: IBM is uniquely suited to assist clients on their cognitive journeys. We have been a pioneer in consistent low latency storage and are already working on using SCMs to lower the latency further and to eliminate latency spikes and degradation. We were also pioneers in enabling storage automation and ease of use features via our purchase of XIV over a decade ago. We will use that experience plus our expertise in cognitive to assist clients with their storage management and support. IBM Storage Insights is our cloud-based platform for collecting information on storage systems around the world and detecting in advance issues which might affect our clients. This is just as critical to our clients as what technology we exploit. “Simple” is the name of the game today, and we will ensure that everything we do is designed with ease of use and simplicity in mind.
CK: Those sound like admirable, worthy goals that IBM Storage is well-positioned to achieve. Thank you for sharing your time and insights, Andy.
AW: You’re very welcome.
Charles King is a principal analyst at PUND-IT. © 2018 Pund-IT, Inc. All rights reserved.