Big data moving to Kubernetes with speed, complexities arising
Pepperdata announced the results of a new survey to gauge the pace at which enterprises are migrating big data applications to Kubernetes containers.
600 senior enterprise IT professionals in industries ranging from finance, healthcare, automotive, telecommunications, retail and other data-intensive businesses were polled. Companies ranged from 500 employees to more than 5,000.
Kubernetes is the preferred container orchestration technology for its agility, speed and efficiency for scaling and managing apps and infrastructure. Because of this, 77 percent of respondents signaled their intent to migrate 50 percent or more of their big data applications to containers with Kubernetes by the end of the year.
Applications that topped the list of those moving to Kubernetes include Spark, which was the most popular with 63 percent of the votes. Presto took 2nd with 45 percent, Kafka came in at 42 percent, and Trino scored 34 percent. Flink followed up with 27 percent.
The biggest challenges with moving big data to Kubernetes
- Allocation and reallocation of resources (30 percent)
- A lack of comprehensive visibility and monitoring tools (26 percent)
- Fragmentation in the Kubernetes marketplace (so many tools and opinions, 20 percent)
- Difficulty manually tuning Kubernetes (nine percent)
- Complexity and scaling (15 percent)
“The velocity of the shift to Kubernetes is surprising. It’s gratifying to see big data move to Kubernetes with this speed. It’s clear that this shift will create optimization challenges as customers understand the nuances of running within the Kubernetes environment,” said Ash Munshi, CEO, Pepperdata.
When asked what their goals were for migrating to containers, respondents were split. 45 percent want an increase in application performance and stability. 27 percent said flexibility and portability of workloads; and 18 percent want to decrease costs. 10 percent said they wanted a multi-cloud solution to keep from being locked into one cloud vendor.