Data Management

Enterprises Embrace Open Source To Tackle Growing Data Management Challenges

New research shows enterprises pivoting to cloud-based containerized environments and software-defined storage. This trend is revolutionizing how businesses are shifting data management strategies.

Linux Foundation Research and SODA (Strategic Options Development and Analysis) Foundation on Tuesday released study results on new data and storage trends for enterprise. The 2021 Data and Storage Trends Report reveals enterprise use of data and storage as it relates to cloud services and workloads in the era of cloud native, edge, IoT and 5G.

The research was announced as part of this week’s KubeCon event. The Linux Foundation and SODA Foundation are sponsors of the event. SODA Foundation is an open-source project under Linux Foundation that focuses on fostering an ecosystem of open-source data management and storage software for data autonomy.

Data has changed the nature of both enterprise computing and business operations, noted Steven Tan, chair, SODA Foundation, and VP and CTO of cloud storage solution at Futurewei.

“No longer can companies take a passive or reactive approach to market changes or customer behaviors. Understanding how to collect, manage, and consume data is a competitive advantage in today’s digital economy, and we believe the 2021 Data and Storage Trends Report can inform this journey,” he said.

Collaboration with SODA Foundation represents service to industry and community as a research center for understanding key technology trends that inform open-source development and resource allocation, added Hilary Carter, vice president of Linux Foundation Research.

“We believe the 2021 Storage and Data Trends Report can advance the work of its community and the broader open-source ecosystem and tech community,” she said.

Important Factors Revealed

One of the most significant or surprising results in the report is validating that environments are no longer dominated by a single platform. Virtual machines mixed with containers running on-premises, in hybrid cloud or in public cloud is becoming the norm, observed Tan.

“With data scattered everywhere, visibility of the entire environment together with automated controls is key to driving operational efficiency and enhancing business agility,” he told LinuxInsider.

The biggest takeaway about the impact on data storage for businesses is that enterprises facing growing data and storage challenges are embracing open source. This is consistent with organizations like Toyota, Vodafone, China Unicom, and others joining SODA Foundation to share their ideas and tackle challenges together, he added.

“With data breaches and ransomware attacks constantly in the headlines, enterprises are fixated on how to better protect and secure their data. In addition, the increasing adoption of open-source projects means more data and storage infrastructures are turning vendor neutral,” said Tan.

Report Basics

Storage requirements are moving from terabytes to petabytes, the survey of 247 seven respondents reported from April 15 to Aug. 19. Worldwide, 40 percent of the sample originated from North America, 25 percent from Europe, 17 percent from Latin America, 9 percent from India, 4 percent from Japan, 2 percent from China, and 3 percent from the rest of world (Middle East, Africa, Latin America, and Australia/Oceania).

The distribution of responses by organization type included end users who are primarily consumers of IT products and services, IT vendors and services providers, and members of standards organizations, open source organizations, and academia.

Mainstream annual data growth is between 1-100 TB per year as reported in the study. However, 9 percent of the sample is seeing annual data growth of 1 PB or more.

This is 10 to 100 times greater growth than the mainstream and is likely a harbinger of where many enterprises will find themselves within a few years, noted researchers.

Key Findings

Forty-eight percent of enterprises are using cloud VMs in production and development environments. A much larger number of companies (75 percent) use Kubernetes or a hybrid Kubernetes on-premises and public cloud combination.

Software-defined storage could revolutionize how enterprises manage data, the study suggests. Three technologies shared the leadership shift in enterprise storage infrastructure: file storage (65 percent), software-defined storage (60 percent), and public cloud storage (60 percent).

Containerized cloud native storage use comes with its share of pain points, researchers found. Performance was identified by 49 percent of end-user enterprises as the chief drawback when using containers and/or cloud native storage. Availability closely followed at 46 percent.

Open-source projects are shaping data and storage infrastructure use. Slightly more than one-third of enterprises engage with SODA Foundation, with GlusterFS and Alluxio coming in closely behind.

Enterprises fixated on data protection and security. Data protection and availability topped the response list followed by security and compliance at 61 percent and 57 percent.

Cloud Native, Open Source See Rising Activity

Storage is one of the most core components of cloud native infrastructure. But historically, persistent storage systems were run outside of cloud native and Kubernetes environments, according to Chris Aniszczyk, CTO of CNCF.

“This has changed in recent years as the cloud native storage space has exploded with new tools and industry efforts emerging to enable storage of persistent state in cloud native environments,” he said.

Open source is paving the way for innovation in the data and storage space. Now we know how companies are using open source to advance their cloud native and container strategies, observed Kei Kusunoki, infrastructure engineer at NTT Communications.

“The Data and Storage Trends report from SODA Foundation will be an invaluable resource for months to come,” he said.

The results of the SODA 2021 Data and Storage Trends Report show just how important open infrastructure is to large organizations managing data at petabyte scale, according to Allison Price, director of marketing and community for the Open Infrastructure Foundation.

“Technologies like containers and software-defined storage are going to lead the way, with open source and open communities being the primary drivers making it happen,” she said.

Jack M. Germain

Jack M. Germain has been an ECT News Network reporter since 2003. His main areas of focus are enterprise IT, Linux and open-source technologies. He is an esteemed reviewer of Linux distros and other open-source software. In addition, Jack extensively covers business technology and privacy issues, as well as developments in e-commerce and consumer electronics. Email Jack.

Leave a Comment

Please sign in to post or reply to a comment. New users create a free account.

More by Jack M. Germain
More in Data Management

Technewsworld Channels