CoreWeave and VAST Data: Rethinking Data for AI

1
00:00:00,040 --> 00:00:06,600
We are rapidly seeing growth of AI adoption and AI workflows that require performance, data

2
00:00:06,640 --> 00:00:13,000
streaming, and vast scale of data. VAST is building an AI operating system, basically serving the

3
00:00:13,000 --> 00:00:19,160
infrastructure software infrastructure that's needed to build AI applications. Partnerships are

4
00:00:19,200 --> 00:00:24,400
a blessing, as well as a little bit of a challenge to kind of navigate. When we met CoreWeave for the

5
00:00:24,400 --> 00:00:30,320
first time, both companies were much smaller. We basically came together and envisioned what would

6
00:00:30,320 --> 00:00:36,120
be the best, you know, infrastructure for AI that we could innovate and co engineer together. In the

7
00:00:36,120 --> 00:00:43,040
age of deployment of so much power and compute and data, we have no excuse for

8
00:00:43,040 --> 00:00:48,440
not being efficient. We're 1300 people pretty much spread across the globe. And I get to work with

9
00:00:48,440 --> 00:00:52,120
the likes of CoreWeave on building the future of AI infrastructure.

10
00:01:03,760 --> 00:01:09,720
So in today's hyper-connected tech landscape, no single platform can power the entire AI stack

11
00:01:09,720 --> 00:01:16,280
alone. Agentic AI workflows that require high performance, real-time data streaming, and vast

12
00:01:16,280 --> 00:01:22,280
amount of data at scale. So I'm super excited about talking about the unique partnership

13
00:01:22,280 --> 00:01:29,200
between CoreWeave and VAST Data. And I'm joined by Alon from Vast Data today. Welcome, Alon. Thank

14
00:01:29,200 --> 00:01:35,320
you. Happy to be here. Awesome. Could you please introduce yourself and VAST Data to the audience?

15
00:01:36,160 --> 00:01:43,080
Sure. Uh, nice to meet you. I'm Alon Horev, co-founder and CEO at VAST Data. VAST is building an

16
00:01:43,080 --> 00:01:48,280
AI operating system. Basically serving the infrastructure software infrastructure that's

17
00:01:48,280 --> 00:01:54,600
needed to build AI applications. We're mostly known for storage, which is what we started with.

18
00:01:54,600 --> 00:02:00,680
So we deliver storage services, storage software that's powering a lot of the model

19
00:02:00,680 --> 00:02:05,480
training and inference that's happening out there today. We are 1300 people pretty much spread

20
00:02:05,480 --> 00:02:11,800
across the globe, and I get to work with the likes of CoreWeave on building the future of

21
00:02:11,800 --> 00:02:18,160
AI infrastructure. That's exciting. So let's start the discussion. I'm sure our audience is going to

22
00:02:18,160 --> 00:02:24,400
love it. My first question to you is what was the business driver or the essential business

23
00:02:24,400 --> 00:02:30,240
requirement that brought this partnership together? So I think, you know, when we met CoreWeave for

24
00:02:30,240 --> 00:02:35,839
the first time, both companies were much smaller. We were about 30 people back then.

25
00:02:35,840 --> 00:02:42,720
But it was clear that the massive need for AI infrastructure

26
00:02:42,720 --> 00:02:48,919
is arising, right? And with that need for compute and networking, there's also a need for

27
00:02:48,920 --> 00:02:55,760
having fast access to data. And we basically came together and envisioned what

28
00:02:55,760 --> 00:03:00,840
would be the best infrastructure for AI that we could co innovate and co-engineer

29
00:03:00,840 --> 00:03:05,749
together. The folks at CoreWeave had lots of experience building and deploying systems,

30
00:03:05,750 --> 00:03:11,869
including storage in their past. So they already were able to reflect to us requirements and

31
00:03:11,869 --> 00:03:16,070
things that they felt would be innovative, but also appreciate some of the things that we've

32
00:03:16,070 --> 00:03:22,149
already done. And that's how we started. We knew that we wanted to apply first-principle thinking

33
00:03:22,149 --> 00:03:27,350
and build something that's truly innovative, that can beat other solutions at every metric, from

34
00:03:27,350 --> 00:03:33,509
latency to throughput to scalability. But also solve for some some unsolved problems like

35
00:03:34,229 --> 00:03:39,709
how do we reimagine multi-tenancy and isolation in the new world of infrastructure, doing bare

36
00:03:39,710 --> 00:03:45,229
metal hosting? So we had a bunch of ideas that each one of us brought together as we kicked off

37
00:03:45,230 --> 00:03:51,949
this partnership. Yeah, that's great to hear because in the era of AI, it is

38
00:03:51,950 --> 00:03:58,149
so data-hungry, and making sure that data is rightly governed and rightly served

39
00:03:58,149 --> 00:04:03,749
across different environments is so, so fundamental. So before we go further along, I was

40
00:04:03,749 --> 00:04:10,710
wondering, you know, partnerships are a blessing as well as a little bit of a

41
00:04:10,710 --> 00:04:16,230
challenge to kind of navigate. So my question to you is that in order to bring in operational

42
00:04:16,230 --> 00:04:22,989
fusion, how did you all integrate for velocity? Meaning how did you align the cultures, code bases

43
00:04:23,029 --> 00:04:29,069
and stats to remove friction for the end-user? That's a great question. You know, the way I

44
00:04:29,070 --> 00:04:34,750
look at partnerships is that they have to be strategic. They need to make sense now,

45
00:04:34,750 --> 00:04:39,389
but they also need to make sense in a year from now and into the future. And that requires

46
00:04:39,390 --> 00:04:46,309
alignment in various ways. First of all, it needs to be an alignment of DNA. How do the

47
00:04:46,309 --> 00:04:52,630
organizations operate? How do they innovate? How do they engineer? And we have our engineering team

48
00:04:52,869 --> 00:04:57,509
working very closely with the CoreWeave engineering team, where we really architect

49
00:04:57,510 --> 00:05:02,669
solutions together, right? And then we execute upon them and we measure ourselves upon them. And it

50
00:05:02,670 --> 00:05:08,029
starts with DNA, but it continues into an operational mindset of how do we deliver working

51
00:05:08,030 --> 00:05:14,469
solutions to our joint customers, and really being able to operate as an extension of CoreWeave's

52
00:05:14,469 --> 00:05:20,189
engineering power, right? CoreWeave would not have selected us if we weren't able to run at

53
00:05:20,190 --> 00:05:26,910
the pace that CoreWeave is running on its own and being able to only amplify CoreWeave's

54
00:05:26,760 --> 00:05:31,719
velocity and not pull it back. So I think it starts with the DNA and it continues

55
00:05:32,309 --> 00:05:38,229
with both an operational mindset, but also a good execution plan and an interlock between the

56
00:05:38,230 --> 00:05:44,350
engineering team. So do you have executive sponsorship, regular cadence of interactions and

57
00:05:44,350 --> 00:05:50,790
mitigating all the issues that came along? We do. We have a lot of that. But I would also

58
00:05:50,790 --> 00:05:57,589
say that we have alignment at various levels of the organizations that you actually don't need

59
00:05:57,589 --> 00:06:03,429
to have an escalation to a high top executive to get something resolved. This is where

60
00:06:03,429 --> 00:06:09,150
alignment happens, where we know how the infrastructure behaves together with CoreWeave.

61
00:06:09,310 --> 00:06:13,429
And we want to be proactive and we want to be understand how things are behaving and whether

62
00:06:13,429 --> 00:06:18,590
customers are happy, even without asking them or hearing them complain, even before something like

63
00:06:18,590 --> 00:06:24,469
that happens. Both organizations are very proactive in monitoring and engaging, and the same

64
00:06:24,469 --> 00:06:28,669
also applies to how we come up with new solutions to interesting challenges and come up with

65
00:06:28,670 --> 00:06:34,989
innovation. And I think that's part of the recipe for a good partnership. Did you ever run

66
00:06:34,989 --> 00:06:41,950
into some kind of code-stack discrepancies, or any kind of software mismatch that you had

67
00:06:41,950 --> 00:06:48,070
to kind of navigate? I think with any integration there are obstacles, right? And this is where being

68
00:06:48,070 --> 00:06:54,349
able to collaborate very deeply and move fast together towards the same goal, which is

69
00:06:54,350 --> 00:06:58,790
how do we build the best AI infrastructure that we can humanly

70
00:06:58,829 --> 00:07:03,029
imagine? It's understandable that we're going to bump into challenges, but it's also the

71
00:07:03,029 --> 00:07:07,150
opportunity because when we're building together, we have an opportunity to build something that's

72
00:07:07,150 --> 00:07:13,020
extremely differentiated for both sides. So I think I think challenges are good. And you meet

73
00:07:13,020 --> 00:07:18,499
them along the way. And as you integrate, you design and you finalize APIs for how data path

74
00:07:18,499 --> 00:07:24,099
and control plane and monitoring and know how all of those things work. But they eventually

75
00:07:24,099 --> 00:07:29,179
contribute, for example, to CoreWeave's user experience to get a single pane of glass that has all the

76
00:07:29,180 --> 00:07:35,139
information that the user needs to understand how the infrastructure is behaving. Got it. Got it. So

77
00:07:35,380 --> 00:07:41,179
my next question is that, as we all know, this is all being done for the interest and

78
00:07:41,180 --> 00:07:47,419
the common benefit of the end user. Right. So could you talk to me about how did the two

79
00:07:47,420 --> 00:07:53,059
technologies and this partnership create compounding value for the end-user? I think

80
00:07:53,059 --> 00:07:57,499
there are several angles here that I would say the first one is really about

81
00:07:57,899 --> 00:08:04,259
efficiency in the age of deployment of so much power and compute and data

82
00:08:04,660 --> 00:08:10,059
we have no excuse for not being efficient. We really need to be efficient in how we use

83
00:08:10,059 --> 00:08:14,859
hardware and how we use the power, because that translates to billions of dollars of savings. And

84
00:08:14,859 --> 00:08:19,858
this is why VAST has invested years and years into algorithms and data structures that allow us to

85
00:08:19,859 --> 00:08:26,780
compress data and allow us to use and leverage a very dense QLC Flash. These technologies did

86
00:08:26,780 --> 00:08:33,019
not exist before we started working on them, and I think that attributes to efficiency.

87
00:08:33,260 --> 00:08:40,259
Then there's scalability, because the workloads that we're seeing today are unprecedented in terms

88
00:08:40,259 --> 00:08:47,179
of their scale. You can see one job running on top of 200,000 GPUs, right? Imagine one process that's

89
00:08:47,179 --> 00:08:53,780
so distributed that it needs to run on so many machines at the same time and access vast amounts

90
00:08:53,780 --> 00:08:58,418
of information, hundreds of petabytes at a time. The innovation and engineering work that was

91
00:08:58,419 --> 00:09:02,378
required to to enable that has been massive, and that's what we've been working on for

92
00:09:02,379 --> 00:09:08,899
ten years. So I think these are new challenges that came because of of new AI

93
00:09:08,939 --> 00:09:13,979
workloads. This is how large scale training looks like. And inference is also disaggregated these

94
00:09:13,979 --> 00:09:20,019
days and scales across machines. And these are the things that we are delivering together that are

95
00:09:20,020 --> 00:09:26,299
very different in terms of how infrastructure was built in a general-purpose cloud or in a

96
00:09:26,300 --> 00:09:32,819
more legacy type environments that were focused on general purpose CPU workloads. Yeah. I think you

97
00:09:32,820 --> 00:09:37,460
very well said that, you know, it was primarily about efficiency. And, you know, as we know, because

98
00:09:37,460 --> 00:09:44,060
in the era of AI, cost has been a big inhibitor and people are struggling to realize ROI.

99
00:09:44,099 --> 00:09:49,379
So any sort of efficiency in terms of seamless experience, it would have definitely equated to

100
00:09:49,419 --> 00:09:55,699
some cost reduction, time to value, time to insight. Is there any specific metric beyond efficiency

101
00:09:55,700 --> 00:10:00,659
that comes to your mind, or do they all kind of correlate? So there are additional metrics.

102
00:10:00,659 --> 00:10:05,539
And I think one of them is not necessarily a metric but a trait. Fungibility. And fungibility is

103
00:10:05,539 --> 00:10:11,380
a very big deal in the cloud because it allows creating alignment between how customers want to

104
00:10:11,460 --> 00:10:18,179
consume and how cloud service providers sell. Customers want to pay on what they're using, right?

105
00:10:18,340 --> 00:10:23,579
The notion of spot instances or allocating and provisioning and releasing things like virtual

106
00:10:23,580 --> 00:10:29,699
machines or GPUs or even storage is one of the things that the big clouds have really

107
00:10:29,699 --> 00:10:35,859
innovated with. And now we see the same challenge and opportunity in this space that we're

108
00:10:35,859 --> 00:10:41,579
working on right now, which are clouds for AI. So the opportunity for CoreWeave, by

109
00:10:41,659 --> 00:10:47,178
deploying multi-tenant environments, is for those customers that need more flexibility and more

110
00:10:47,179 --> 00:10:54,139
flexible consumption models and also utilization models. That's a big opportunity, right? So CoreWeave

111
00:10:54,140 --> 00:10:59,339
has to solve this for the networking, to provide strong isolation. They need to provide that kind

112
00:10:59,340 --> 00:11:04,099
of multi-tenant experience to the GPU nodes. And we're doing that for the data side as well. So

113
00:11:04,099 --> 00:11:09,620
being able to provision capacity and provision performance unlocks flexibility and fungibility

114
00:11:09,659 --> 00:11:16,009
for the end customers. That's extremely important. And you will still have large

115
00:11:16,010 --> 00:11:21,809
customers, frontier labs, large enterprises that are going to be able to make large

116
00:11:21,810 --> 00:11:26,049
commitments in advance. But so many of the customers out there want to have the flexibility

117
00:11:26,049 --> 00:11:30,729
to not pay for everything in advance. And there's a big win-win situation for all three parties

118
00:11:30,729 --> 00:11:36,289
involved by enabling that kind of efficiency model. Great. Is there a

119
00:11:36,289 --> 00:11:42,928
specific industry or geography where you're seeing more traction versus the other?

120
00:11:43,329 --> 00:11:48,769
You know, so VAST started before AI was a big deal, right? We started the same month when OpenAI

121
00:11:48,769 --> 00:11:55,129
was incorporated. And we were initially focusing on high-performance computing and analytics on

122
00:11:55,130 --> 00:12:01,249
large enterprises, internet companies, because we knew that lots of data in one place with fast

123
00:12:01,249 --> 00:12:07,449
access has lots of value for big data analytics. Now, it's also very obvious that for AI, it's

124
00:12:07,450 --> 00:12:12,889
even more intense and it gets even bigger. So obviously AI is growing at an exponential pace,

125
00:12:12,890 --> 00:12:18,928
and we see that through our customers that grow with us and adopt the platform. But we also

126
00:12:18,929 --> 00:12:24,689
continue to see other sides of the market continuing to evolve and then apply the same

127
00:12:24,690 --> 00:12:30,689
principles of AI. Customers like, you know, CC Bio. The Chan Zuckerberg Foundation was a customer of

128
00:12:30,689 --> 00:12:35,409
ours, but also operates inside of CoreWeave. The same principles of going from classic machine

129
00:12:35,409 --> 00:12:42,329
learning to adopting AI and leveraging neural nets, leveraging LLMs to tackle other segments and

130
00:12:42,330 --> 00:12:47,968
other challenges in the market, beyond just classic text generation, is what we're now

131
00:12:47,969 --> 00:12:54,009
seeing with our customers in the farming industry, healthcare, autonomous driving, animation, so many

132
00:12:54,010 --> 00:13:00,010
different and fascinating initiatives, that we see classic machine learning and classic methods

133
00:13:00,210 --> 00:13:05,929
completely transformed and reimagined through this lens of AI. So we are seeing a big investment

134
00:13:05,929 --> 00:13:12,209
into AI, but not just through the frontier labs and the infrastructure providers. We also see that

135
00:13:12,210 --> 00:13:18,608
through the end customers that are investing deeply in individual initiatives. Great. So,

136
00:13:19,169 --> 00:13:25,969
this is a fascinating set of insights. And I'd like to kind of close the discussion because we

137
00:13:25,969 --> 00:13:31,169
want to know that you have learned a lot of lessons partnering with CoreWeave in

138
00:13:31,169 --> 00:13:38,090
the last couple of months, years. What is the journey forward? What does

139
00:13:38,090 --> 00:13:44,168
the future vision look like? What did you learn based on your best practices? What is your

140
00:13:44,210 --> 00:13:49,489
kind of call to action to anybody who's listening to us? Or where do you see the journey going ahead?

141
00:13:49,530 --> 00:13:56,048
So that's obviously my perspective. But I see two vectors. I think one of them is scaling.

142
00:13:56,049 --> 00:14:01,809
Every time you increase, you know, the size of a company or the size of an operation, you need

143
00:14:01,810 --> 00:14:08,010
to optimize, and you need to reimagine how it is that you architect this entire process. So this

144
00:14:08,010 --> 00:14:14,449
vector of growth for CoreWeave seems to be massive. And, you know, the revenue backlog speaks for itself.

145
00:14:14,450 --> 00:14:19,280
But at the same time, there is an opportunity to serve more and deliver more value to those end

146
00:14:19,280 --> 00:14:26,239
customers instead of just, you know, leasing out raw GPUs. A lot of those customers are trying

147
00:14:26,240 --> 00:14:30,639
to solve similar problems. There is an opportunity to deliver more value to them, to give them more

148
00:14:30,640 --> 00:14:36,399
managed services, to take more off their plate in terms of deploying inference, deploying frameworks

149
00:14:36,400 --> 00:14:41,999
they would use for training, fine tuning, reinforcement learning. CoreWeave is becoming an

150
00:14:41,999 --> 00:14:48,598
expert, and IS an expert, in all of those other aspects of of the AI development lifecycle. And

151
00:14:48,599 --> 00:14:55,159
the way we approach it is that data comes in different shapes and forms, right? We started with

152
00:14:55,160 --> 00:15:00,879
training and we stored checkpoints and data sets and home directories. But now when you look at

153
00:15:00,879 --> 00:15:04,639
inference, there are so many different other modalities of data. You store vectors to do

154
00:15:04,680 --> 00:15:10,079
similarity search. You store events for event streaming, because all of your users or

155
00:15:10,120 --> 00:15:16,200
agents are generating so much data. You start to accelerate how inference is running today. So we

156
00:15:16,280 --> 00:15:22,439
have those joint opportunities in delivering more value, delivering higher-order services that would

157
00:15:22,439 --> 00:15:26,880
enable CoreWeave to deliver more value to its customers, to increase its margins, and to be

158
00:15:26,880 --> 00:15:31,560
differentiated in the market. And this is a principle that also applies to us. The same

159
00:15:31,560 --> 00:15:36,919
platform, the same software running on the same flash can support more modalities of data. So we

160
00:15:36,919 --> 00:15:42,359
started with file, but now it's also object and block and event streaming at a database. And we

161
00:15:42,359 --> 00:15:49,039
are very aligned with CoreWeave, just providing more value to our customers over time. Great. Thank

162
00:15:49,040 --> 00:15:55,999
you Alon. This is fantastic. And I cannot agree more that, you know, in the era of agenetic AI, real

163
00:15:56,039 --> 00:16:01,879
time data availability, context retrieval, and all different types of data ingestion is very, very

164
00:16:01,879 --> 00:16:07,559
critical to autonomous agent architecture. This is going to be fascinating journey for the two of

165
00:16:07,559 --> 00:16:13,598
you all. And I look forward to seeing where we are heading now. Thank you. Thank you very much.

‍

‍