EP 008 - NVIDIA Blackwell Chips, Grok going open source, Apple starting to surface AI Initiatives, and more!

00:00:00: Hello and welcome to episode number eight of the AI Boardroom.

00:00:05: I oftentimes forget to say that's the name of our podcast, but by episode eight, you

00:00:11: know.

00:00:15: Svetlana, you're welcome.

00:00:17: How are you doing?

00:00:18: I don't welcome you.

00:00:19: It's like it's our part.

00:00:21: Yeah, doing awesome.

00:00:23: I mean, it's great.

00:00:25: Lots of news this week.

00:00:26: I'm really excited about what we're going to be talking about.

00:00:29: But yeah, just we recently came from vacation.

00:00:33: That's why I think we skipped a week from posting.

00:00:35: So I have a little bit more color to my skin than I usually do.

00:00:39: I don't glow in the dark.

00:00:41: Oh, now it's fine.

00:00:43: Like the...

00:00:43: The episode seven was that was colorful.

00:00:47: There's a lot of contrast.

00:00:48: Yeah.

00:00:49: But yeah, no, I'm well -enrested and came back with lots of things kind of happening

00:00:54: in the world.

00:00:54: I was very much fully disconnected just because I think was we talked about just

00:00:59: the Internet was very intermittent.

00:01:01: It was just not that great, but it was a good excuse to really disconnect and take

00:01:06: a break.

00:01:07: Sometimes it's nice.

00:01:09: I had one experience where we were in a club where you were not allowed to use a

00:01:17: phone.

00:01:19: A friend of mine was going to be married and we had his, what is it called in

00:01:23: English?

00:01:25: The bachelor's party?

00:01:26: Bachelor's party, right.

00:01:28: We had this bachelor's party and we were in the club and you weren't allowed to use

00:01:33: a cell phone.

00:01:34: We were in there for two or three hours and it was really nice to just like...

00:01:39: You weren't allowed and you could talk to the people there.

00:01:41: Like it was really nice.

00:01:43: Really cool.

00:01:43: Disconnect.

00:01:43: Yeah.

00:01:44: Disconnect from your device and then really pay attention.

00:01:46: Yeah.

00:01:47: So I think that's what we did.

00:01:48: Yeah.

00:01:48: And now I'm kind of back and well rested and connected back to the network of AI

00:01:56: news and all kinds of things that are happening.

00:02:00: So talking about AI news and

00:02:03: a bunch of parties and getting married like Nvidia seems to be married to the

00:02:06: evaluation, right?

00:02:07: And that's a good marriage.

00:02:08: We did talk about that.

00:02:09: That's it's it's really fun.

00:02:11: So before I think I went on vacation last time I did some research.

00:02:13: It was like, you know, Nvidia just broke the one trillion mark of, you know,

00:02:19: valuation.

00:02:20: That was last year.

00:02:21: Like, I think now they broke two trillion.

00:02:23: Was it?

00:02:23: OK, maybe I'm mixing up my numbers.

00:02:25: But like the way their growth is.

00:02:28: Yeah, astounding.

00:02:30: And now they're I think.

00:02:31: number three most valuable company like tech company with 2 .2 they definitely are

00:02:37: I think Apple's up there and the two trillion club and I think Microsoft's

00:02:41: there Microsoft even like reached two trillion at some points I think three

00:02:45: trillion which is completely ridiculous and of course it's over the aided like

00:02:52: that's that's and this is no financial advice we have to put the disclaimer here

00:02:58: But yeah, it's ridiculous.

00:02:59: I read stories about Nvidia employees that were already there for like 10 years or

00:03:06: so.

00:03:07: And most of the Silicon Valley companies, they give out stock to their employees as

00:03:13: part of the salary.

00:03:14: And I just looked it up.

00:03:16: 10 years ago, Nvidia stock was at 10 billion and now it's like 2 trillion.

00:03:20: And if you collect the stock for 10 years, you already got a good amount.

00:03:25: And people just quit their jobs or reduced to only doing half the hours because they

00:03:34: say, okay, I pay out now.

00:03:37: Yeah.

00:03:37: I can retire or I can do work for fun now.

00:03:41: Yeah, and that's actually an issue for Nvidia because they need more people and

00:03:48: not like losing the watch they have.

00:03:50: Nvidia, if you're hiring, I'm just kidding.

00:03:53: At the old stock prices and like if you can...

00:03:56: Yeah, the old stock price.

00:03:58: Calm down.

00:03:59: I don't want much, just to position the board.

00:04:02: Yes, please.

00:04:03: A yearly salary of like 500k and I'm fine.

00:04:06: Not asking much, you know.

00:04:10: But yeah, Nvidia, I think, I'm not sure how it's now, like a week or so after the

00:04:17: event, but it definitely helped the stock price, I bet, because yeah, Nvidia kind of

00:04:24: got a lot going, especially the new Blackwell architecture, which is more of a

00:04:30: platform now.

00:04:31: I think they talked a lot about a lot of stuff.

00:04:37: And most of that was...

00:04:39: Really impressive.

00:04:42: What was your most collectible moment from there?

00:04:46: I'm big in robotics, so I was really impressed by their lineup of robots and

00:04:52: experimentation and some of the things that they are doing.

00:04:56: But the fact that they're experimenting with so many different versions of robots,

00:05:03: you kind of see Tesla come up with one, OpenAI has their...

00:05:08: They're one figure.

00:05:11: And so each, everyone kind of has just like one best foot forward that they're

00:05:15: putting.

00:05:15: And I was really impressed by Nvidia just kind of exposing all of their lineup,

00:05:20: regardless of what kind of the capabilities there are.

00:05:23: Yeah, but they also don't do like their own robots.

00:05:26: They build like basically a foundation, foundation models.

00:05:30: Like Nvidia calls it AI Foundry.

00:05:32: So they basically say, okay, we deliver you all the tooling.

00:05:36: which you of course execute on our GPUs.

00:05:39: But we give you all the tooling, all the simulation stuff so that you can build the

00:05:43: robots.

00:05:43: So they basically say, okay, hey, that's your market.

00:05:46: There's your gold and we used to have the eggs and we still have a bit more.

00:05:54: So just buy the tooling from us and the rest will go by itself.

00:06:00: So that's how they market it at least.

00:06:02: But still pretty impressive.

00:06:03: But they do have their own researchers and so they're probably doing some proof of

00:06:08: concepts, I'm sure, with those robots.

00:06:10: They work also with universities a lot as far as I understand.

00:06:14: A lot of stuff that's coming out, no matter who it is, it's based on university

00:06:21: research.

00:06:23: Where's Boston Dynamics?

00:06:25: We've been hearing for the...

00:06:27: I had some updates from them.

00:06:31: As far as I understood, I've talked to someone who's really deep into the

00:06:35: robotics space, and Boston Dynamics got some updates.

00:06:39: And they also have all the government contracts and that stuff, so they are

00:06:43: really well suited.

00:06:45: And they still have...

00:06:47: Technology wise, like all the motors and all the everything like that's physically

00:06:52: working on that robot.

00:06:54: They're still upper class.

00:06:55: Like I think there is no one even close.

00:06:58: Did you know that Boston Dynamics was part of Google and Google just like got rid of

00:07:01: them?

00:07:02: I've heard about that.

00:07:04: Yeah.

00:07:04: Interesting.

00:07:05: It's like to say it like it's not the best decision Google ever made.

00:07:14: And that's also what the employees think.

00:07:20: But yeah, Blackwell Architecture, they have now, I think they said they reached

00:07:27: in eight years, a thousand X performance increase in AI calculations, which are

00:07:34: mostly vector calculations.

00:07:35: They cheated a bit on that number because they basically defined a format which they

00:07:41: call FP4, which is a four bit.

00:07:44: format which is just, yeah, you have like just a smaller amount of bits for defining

00:07:53: a weight.

00:07:54: So I think usually weights are FP16, so you have 16 bits.

00:08:00: And just the amount of numbers these bits can represent are a lot larger and so you

00:08:07: can dial in a lot more nuanced weights.

00:08:13: Whilst you can reduce the bit size to 4 -bit, it gets a lot faster.

00:08:18: It's also used a lot in the AI community called quantization.

00:08:24: For example, Facebook releases the 70 billion model of Lama 2.

00:08:31: The community goes ahead and quantize the stuff to get down to 6 -bit or 4 -bit to

00:08:37: be able to then run it on their local machines.

00:08:42: That's how Nvidia cheated a bit late.

00:08:43: They took this FP4 format and their increase in speed in that FP4 format was

00:08:51: like 200x to the previous generation.

00:08:54: If you look at the more used stuff and the actual models, it's not that huge, but

00:09:00: it's still a lot.

00:09:02: So it's still a big jump, especially for a year on year upgrade.

00:09:05: So Nvidia really has it going there.

00:09:08: So what's the difference between the...

00:09:12: I mean, computationally, I don't know, between the H level chips and the B, the

00:09:21: new blackwall chips.

00:09:23: I tried to find the actual numbers.

00:09:27: I think the FP4 performance was even like somewhere in the ballpark of 200x or

00:09:32: something.

00:09:34: I think the other stuff was a bit more like 7x.

00:09:38: Yeah, 7x.

00:09:42: Yeah, something around that.

00:09:45: Which, like, year over year is still really good.

00:09:49: And yeah, they have a larger chip.

00:09:53: So he even made a bit of fun of, like, he held up the old one and the new chip and

00:09:57: was like, Hopper is the previous generation, the new one is Blackwell, and

00:10:04: it's like a chip two times the size, basically.

00:10:09: And kind of like...

00:10:10: Size isn't all like a hopper.

00:10:12: You don't have to feel bad.

00:10:15: It was a bit cringy, but okay.

00:10:17: Can I ask, what is the impact really?

00:10:23: Will this difference be felt by smaller organizations, medium, big corporations?

00:10:30: Who is the target users of these chips?

00:10:32: That's actually a really good question because, and that's something I learned

00:10:36: now from the video, around the video conference.

00:10:40: We still, all the stuff we use, still on the 2022 architecture.

00:10:44: We haven't even had the performance of the H -chips in our everyday lives.

00:10:52: And so people are still building up the H -chips.

00:10:55: So until we get like the new Blackwell performance, might be next year or

00:10:59: something, because the data centers now have to be built, the chips have to be

00:11:03: produced.

00:11:03: There's still like a huge surge in requests for the nodes that only TSMC has.

00:11:10: So, yeah, it's still a limited supply and everyone's buying from Nvidia.

00:11:17: Like if you look at their statements, yeah, they basically quote only the

00:11:23: biggest CEOs of the industry and they all buy from Nvidia.

00:11:28: So you can imagine that's a war and Nvidia, I guess they can just dictate the

00:11:35: price as they want, unless they feel, because nothing comes even close.

00:11:39: From performance perspective, right?

00:11:41: Yeah.

00:11:41: And also not only performance, NVIDIA also delivers all the tooling, which is

00:11:45: basically just well used and well known in the community.

00:11:48: So all the CUDA cores, all the libraries, everything's just mainly NVIDIA focused.

00:11:54: So I tried to run some local speech to text and I couldn't run it because I

00:12:03: hadn't had CUDA because all the libraries were on NVIDIA's CUDA.

00:12:08: chips and optimized for that.

00:12:10: So there are sometimes you have translation layers and stuff like that,

00:12:13: but still like the de facto standard is all the video tooling they provide, which

00:12:19: makes it even clearer why they see themselves as a foundry and all of course

00:12:25: try to keep the people in their system, their ecosystem.

00:12:29: And the robotic stuff comes now on top of it.

00:12:32: They also have some.

00:12:35: metaverse stuff, but more like with production in mind and cat design and

00:12:40: stuff like that.

00:12:41: So we have it's they do a lot of things really right.

00:12:47: And playing with everyone is one of it.

00:12:49: So like I say, I think they dictate the price wherever they want.

00:12:54: So the H100 chips were bought out like hotcakes, right?

00:12:59: So I'm assuming that realistically speaking, as you mentioned, so.

00:13:04: the organization will be able to get their hands on these Blackwell chips no sooner

00:13:11: than probably next year, just because of probably there's going to be as high a

00:13:15: demand.

00:13:17: The stuff has also to be deployed.

00:13:19: I think when we see the GTC, I'm pretty sure they already have the orders dialed

00:13:25: in and are ready to go.

00:13:29: But before it gets to the mainstream where we...

00:13:32: feel it in our APIs, I think that's still taking some time.

00:13:36: I'm also not sure how much all the models and stuff like that, how much stuff has to

00:13:41: be optimized.

00:13:42: I mean, maybe like even, so I don't know how much it is plug and play for if you

00:13:47: really want to expose an API.

00:13:51: Yeah, but like if you think about a video has for AI compute, extra flop.

00:13:59: performance into one tower, like one like bunch of servers.

00:14:06: Just to give you some perspective on that, there are like two or three exaflop

00:14:12: supercomputers in the world.

00:14:14: And for AI, NVIDIA delivers one you can buy and like I can have it in my room

00:14:20: here.

00:14:21: But not from...

00:14:23: I wouldn't be able to deliver the power needed.

00:14:25: So you need really, really thick cables for that.

00:14:28: That's by the way also something that especially if you think about a company

00:14:36: and there's CO2 footprint and stuff like that, every generation getting it faster

00:14:42: is not the only thing.

00:14:45: It also gets more efficient most of the times, which needs less energy, which

00:14:52: makes it more and more practical for like...

00:14:55: right now we're still at the point where I inference stuff that's so expensive that

00:15:01: is like everything but not environmentally friendly to to do a I stuff so the further

00:15:08: we get the more sophisticated and maybe also specialist chips we get the less you

00:15:14: as a company have to to look at your CO2 footprint and

00:15:20: in the end, you maybe even have to pay less off of some certificates for reducing

00:15:25: or like countering a CO2 or something.

00:15:29: That's definitely something to be aware of.

00:15:32: But so the cost, one approach that even with the GTC conference, not even

00:15:37: Blackwell, well, I think it's related, but I saw a lot of influencers actually being

00:15:41: sent packages of these servers, including the Blackwell chips.

00:15:46: Did you see that as well?

00:15:47: And they were promoting...

00:15:48: No.

00:15:50: They were basically creating, I don't want to say, I don't want to call it

00:15:54: sweepstakes, but people would sign up and then basically one person would sign up

00:16:02: for the conference, but then you also get a chance to win one of those Black World

00:16:08: chips.

00:16:09: I've entered a few of them because I was like, even if I win, I'm sure Ed Gier will

00:16:13: be able to guide me on how to use it.

00:16:15: If anything else, I'll send it to him.

00:16:17: Yeah.

00:16:19: I'm also on the product side.

00:16:23: I'm not developing any AI myself, like training and stuff like that.

00:16:28: But there is a big part of me that's the nerd that would love to have one of these

00:16:35: at home and just put it to its paces.

00:16:39: But then on the other hand, I maybe even can't because I'm deep enough into all

00:16:44: this stuff.

00:16:45: You always put it on the black shelf, you know, or like on the glass shelf.

00:16:48: Yeah, like if you think how much even like they have one they showed one computer

00:16:55: which is basically like if they like it kind of is a small form factor in like

00:17:02: server speech at least which has already so much power and also there is one like

00:17:10: smaller thing that really is interesting is that Nvidia puts not all it's like

00:17:15: not one monolithic die, like it's not one big chip, but they combined two smaller

00:17:21: chips and also memory modules and connected them with a new interconnect,

00:17:26: which I thought it's two terabyte, but it's even 10 terabytes of per second of

00:17:32: data they can do, of course.

00:17:36: And that's important because then production of these chips is not as

00:17:40: expensive.

00:17:41: And the chips still function for the CPU, like for the operating system, they

00:17:46: function still as one because they have so much interconnect.

00:17:49: Which is basically the same that Apple did with their M2 Max chips and M2 Ultra chip.

00:17:58: Or like M1 even back in the day.

00:18:02: And they showed that it worked, so I'm pretty confident that Nvidia's approach

00:18:06: will also work because, just to keep it short, it was...

00:18:11: pain in the, you know, it was a real pain to program for systems with two separate

00:18:20: CPUs because there is so much allocation stuff and things you have to take into

00:18:25: consideration that it like bloats the complexity of your code like really to a

00:18:33: degree where it's less handled, less manageable and you make more mistakes and

00:18:37: it's not performant.

00:18:39: So...

00:18:41: having it working as one chip for the operating system is huge for developers.

00:18:46: Do you think, maybe I'll throw a question before we may want to move on.

00:18:54: Do you think, just at the pace that some of these chips are being democratized,

00:19:00: expanding, exponentially growing, performances increasing, that we'll see...

00:19:08: in the very near future, that level of performance in all of our devices, meaning

00:19:12: that computers, so would we have access to significantly higher computational powers?

00:19:20: Or would you still rely on kind of the data centers hugging all of them?

00:19:27: The interesting stuff is our mobile devices already are going that direction

00:19:33: for quite some time now.

00:19:35: And Qualcomm, for example,

00:19:37: last week presented their first notebook chips or like their new notebook chips and

00:19:42: they put a lot of emphasis on their MPU which is the Neural Processing Unit.

00:19:48: Apple does it for like I think since the iPhone 5 they have started building that

00:19:53: stuff in so right now the AI chip in my MacBook Pro is really really capable.

00:20:03: But then again, you have that AI chip, but you only have the Apple interfacing and

00:20:08: the Apple developer tooling.

00:20:10: And I can't execute models without bringing them into the whole ML network

00:20:20: stuff.

00:20:20: Otherwise, I could do inference on systems, which I couldn't do otherwise.

00:20:26: So you can't do like end -to -end development without access to it?

00:20:30: No, it's still kind of a...

00:20:32: fragmentation going on, I think there will be solutions on operating system level and

00:20:37: Apple is also open sourcing stuff right now, but even Apple's own open source

00:20:41: stuff does not work on their MPU.

00:20:46: So yeah, that's still to be solved, I guess, even by Apple themselves.

00:20:51: I'm pretty confident this year's Developer Conference will bring a big leap in that

00:20:56: direction because I'm pretty sure this year will be a lot of AI emphasis on

00:21:01: Apple.

00:21:02: Especially they now had two months ago they released Ferret, their first own

00:21:07: small model.

00:21:09: And like a week ago they released a paper how to train multi models, small models.

00:21:17: And I'm pretty sure our devices will soon be capable of running a lot of stuff

00:21:21: locally.

00:21:22: Speaking of that paper that I think you're kind of alluding to, was that the release

00:21:27: of the MM1 model?

00:21:29: The MM1 is just a paper, there is no model connected to that.

00:21:34: They have this favorite model, I searched it up.

00:21:38: It was more the paper of how they did it.

00:21:40: There might be a release of an MM1 model, I'm not sure about that, I haven't read

00:21:45: the paper, but for now it was more the scientific side of things.

00:21:50: I see.

00:21:50: And how to.

00:21:51: I think the MM1 is also a paper that they've talked about, but it is an actual

00:21:57: model.

00:21:57: So it is...

00:21:58: comparable to GPT multimodal type of version.

00:22:04: But that's what they also said for the ferret model when they released it.

00:22:08: And this is one you can just use right now.

00:22:10: Apple just open sourced it.

00:22:13: So I didn't try it, honestly, because I didn't even know about the claim.

00:22:18: They released it in November or so.

00:22:20: No, or was it no, February, January, January, like somewhere around that.

00:22:25: And they released it and I wasn't even aware that they want to bring the same

00:22:30: vision capabilities that GPT -4 Vision has because it's a pretty small model and if

00:22:35: it can do that, it's pretty impressive.

00:22:38: I think it's four, just to put it in perspective, I think it's like four times

00:22:41: less the size of GPT -V.

00:22:43: Yeah, yeah, yeah, yeah, minimum, yeah, yeah.

00:22:45: So it's a significantly smaller model, but I think the low quality that it can

00:22:50: produce is...

00:22:51: I might try it out for an extra next episode and...

00:22:55: give you some something.

00:22:57: Or maybe we do like separate video on the channel.

00:22:59: So what are your thoughts as to so Apple is coming on scene, right?

00:23:03: All of a sudden, I'm waiting for that.

00:23:05: I'm really waiting for that.

00:23:06: So but they're now starting to publish because for a really long time, you didn't

00:23:10: really hear much from them about AI because they go open source, which is even

00:23:15: more interesting.

00:23:16: Like when did Apple do that?

00:23:18: So yes, I'm just curious.

00:23:21: Why do you think they've been silent?

00:23:23: so long and now all of a sudden 2024 they're like here's here's this in AI that

00:23:28: we're doing here is this because it's how apple always did it like they were never

00:23:34: first to anything like period maybe the ipad but even that is debatable so because

00:23:43: microsoft did their first surface devices and like they they were tablet like

00:23:50: devices like

00:23:51: maybe not multi -touch, more of like stylus stuff, but yeah, still.

00:23:58: I think Apple is a sleeping giant on that, because they have all these devices,

00:24:05: billions of them, running with neural processing.

00:24:09: And the only thing that could bite Apple is they, in terms of memory on the device,

00:24:17: they lag behind Android.

00:24:19: big way, big times, because I think even my iPhone now, and I have the biggest

00:24:25: model, has like 8 gigabytes of RAM.

00:24:30: This is basically, I think more than a 7 billion model won't be even remotely

00:24:36: possible, and I think even smaller.

00:24:39: And that's even worse on older devices, because iOS was always so memory

00:24:44: efficient, they didn't need much memory.

00:24:47: and that could bite them now.

00:24:49: It's a bit different on the notebooks themselves.

00:24:53: So the MacBooks might get even more sophisticated stuff because they just have

00:24:57: more RAM to load stuff.

00:24:59: But even then, you're like loading a large language model on my MacBook is the only

00:25:05: thing that gets the thing heated up.

00:25:06: Like I don't hear the fans even when I export our podcast video.

00:25:12: But when I load an LLM, jeez, the fans are spinning.

00:25:17: And that's the thing, you have to keep it small and still capable to be running the

00:25:25: whole time on your device.

00:25:29: Otherwise, I think it won't be practical at all.

00:25:32: So that's insightful.

00:25:32: So you're basically saying that maybe one of the reasons why they're watching, just

00:25:40: kind of understanding where the market is going.

00:25:43: But...

00:25:43: It sounds like they're also like, oh crap, none of our devices can even support any

00:25:47: of these models.

00:25:48: Probably need to figure out how we can build our own.

00:25:51: Might be.

00:25:53: Efficient one.

00:25:54: But then also we need to optimize our operating system to accommodate whatever

00:25:59: that model ends up being.

00:26:01: And guess what?

00:26:01: That probably that model needs to be more time modal, which is why we kind of have

00:26:05: the MM1 kind of release and then probably enhancing Siri with it.

00:26:11: Right.

00:26:11: Cause you want.

00:26:13: them to do with I hope I hope they bury they bury Siri and then just come up with

00:26:18: something completely new.

00:26:20: Why?

00:26:21: They might also update Siri.

00:26:22: Why are you hating on Siri?

00:26:25: Because it's hateable.

00:26:27: I just asked her a question this morning.

00:26:30: I was like, what temperature is it outside?

00:26:32: And my daughter was like, I know her name now.

00:26:35: Yeah, so my friend of mine had his child in the car and he was asking.

00:26:43: He was asking Siri for directions and then child from behind was like, I like Siri,

00:26:49: she's nice.

00:26:51: See, why are you hating on Siri?

00:26:54: No, the thing is, they never updated it since the first release, at least it feels

00:27:00: like that.

00:27:02: But it was the first of its kind that you have to give them that.

00:27:05: So they were basically the first on doing AI on your smartphone.

00:27:11: Even if they didn't go big on AI on devices, I still cannot do shit with Siri

00:27:17: when I'm not online.

00:27:20: Google is definitely better.

00:27:21: Google can even translate without internet.

00:27:25: I can do actions.

00:27:27: In addition to large language models, hopefully they'll bring actions and then

00:27:31: being able to do things within your phone.

00:27:36: I see stuff like...

00:27:39: the Rabbit MQ device or the pin, I always forget how it's called, but the one which

00:27:49: you get on the...

00:27:50: I know what you're talking about.

00:27:51: I don't remember the name of it either.

00:27:52: Yeah.

00:27:52: Yeah, but like the whole new...

00:27:56: I think they have a direction where I was like, I wish I could just do it on my

00:28:00: phone.

00:28:01: I don't need an extra device for that because it won't replace my phone anyway,

00:28:04: so not even close.

00:28:06: So why...

00:28:07: Why bother bringing out a device?

00:28:09: Why not just release the ecosystem and integrate it into my device?

00:28:13: And in terms of integration, of course, Apple and Google and who else has its own

00:28:18: operating system are in the best spot to integrate it deep into the system.

00:28:24: And yeah, I'm pretty sure this will bring also the whole iOS experience into a new

00:28:34: era.

00:28:34: And I really look forward to that because...

00:28:37: This is like my main device.

00:28:39: I pay huge sums of money every two years for forgetting the new one.

00:28:43: Like the current one I paid 1 ,800 euros or something.

00:28:48: So you're one of those crazies that updates the phone every time the new

00:28:51: version comes out?

00:28:53: No, no, no, that's too much work.

00:28:55: Just every two to three years, kind of.

00:29:00: But yeah, of course, if Apple this year releases all the AI stuff, the next iPhone

00:29:06: comes with double the RAM to accommodate for that.

00:29:09: Of course, they will switch.

00:29:11: Yeah, likewise.

00:29:13: The pin, was that Humane AI?

00:29:17: Humane, yeah.

00:29:18: The Humane pin, yeah.

00:29:19: Yeah.

00:29:19: Just wanted to mention in case the listeners, like, what the heck were they

00:29:22: talking about?

00:29:25: It's basically a pin you can stick to your clothing and with a magnet and then you

00:29:31: have a laser display if you need a display otherwise you use only voice Which has a

00:29:37: lot of implications in and of itself.

00:29:39: So like I said for example the my my airports are pretty function really really

00:29:44: good with the haystree command and Sorry for everyone who's serious now activate

00:29:52: and just have the anti -AI integration on my phone and be able through the ACV

00:29:57: command to use proper AI would be game -changing in and of itself.

00:30:04: Like, do a research in the web for me or stuff like that, stuff you can do on the

00:30:08: go.

00:30:09: For me, it couldn't even go far enough.

00:30:11: I would love to have it coding while sitting on my dashboard.

00:30:17: An interesting perspective.

00:30:18: Do you think now, given that...

00:30:21: I want to say Apple owns 50 % of market share in the mobile phone market.

00:30:26: So it's a significant share.

00:30:27: So let's say they do come up with Elanbe Siri or this chatbot that lives now on

00:30:32: every device.

00:30:33: Do we need chat GPT?

00:30:40: The market is so dynamic and chat GPT is still so far ahead from from like an ad

00:30:46: perspective and usability I use co -pilot more often To to yeah to simple stuff and

00:30:56: even then like half the time I then open chat GPT to to do the same thing there

00:31:02: because just better and With Microsoft's it's like if you use co -pilot it

00:31:09: It's kind of iffy also, which model do they use?

00:31:13: And I think now they released the Turbo model for copilot chat.

00:31:18: I'm not talking about the M365 copilot, but the copilot chat experience.

00:31:24: So what should I use?

00:31:25: I also use Gemini sometimes, which is okay.

00:31:28: Especially I've now used Google Cloud a bit and asking Gemini for Google Cloud

00:31:34: related stuff was pretty useful.

00:31:36: And that's also the same for copilot.

00:31:39: If we use copilot chat for Microsoft Azure related stuff, it's cool.

00:31:43: It's really nice.

00:31:44: Someone just explains me how to use Azure finally and do it on my terms and also all

00:31:51: the CLI commands and stuff like that.

00:31:53: So like everything you need to use the respective cloud infrastructure is so well

00:32:01: defined and askable basically in this.

00:32:06: So yeah.

00:32:06: Well, I think maybe not in the near future as kind of these models.

00:32:10: It sounds like they still have some time to mature, to like really get to a level

00:32:14: of accuracy and maybe people building people trust and kind of like I want to

00:32:21: say like even relationship with them because right now the default chat bot

00:32:26: that people go to is chadgbt and everything else is typically in addition

00:32:31: to so.

00:32:33: Sometimes people do use perplexity as one, but still, tragedy is like a dyad.

00:32:38: But I do think that releasing Siri is going to be quite disruptive because Siri

00:32:45: already lives in that convenience of having that accessibility through the

00:32:50: phone.

00:32:51: Like, why do you even have to go to copilot?

00:32:54: And I think Samsung is probably going to follow suit.

00:32:57: And...

00:32:58: They already did.

00:32:59: They are they're bang.

00:33:01: Yeah.

00:33:01: Yeah, the latest the latest releases of the galaxy series They have galaxy AI.

00:33:06: I think it's how it is and that's Yeah, it works partly on device and it gives you a

00:33:15: really good preview of what's possible so they have also call screening and stuff

00:33:19: like that so that's really They really made a big leap for like considering that

00:33:25: big speed is something they did too.

00:33:26: So it's really

00:33:28: Galaxy AI is really something.

00:33:32: They really come out of the gates with a proper product there, which is actually

00:33:40: kind of useful.

00:33:41: Of course, it's still a lot of stuff is focusing on photos and I'm pretty sure

00:33:45: Apple will too.

00:33:48: So, but yeah, I think...

00:33:50: With Apple, it's always like they also will release new APIs.

00:33:53: They also will release new, new toolkits and SDKs.

00:33:56: And also they have not only the iPhone, they also have the Vision Pro, which they

00:34:02: now need to support.

00:34:05: Which is even more interesting for all the AI stuff because it like, it generates

00:34:12: stuff into your field of view.

00:34:15: Yeah.

00:34:15: I think, I think that's really, really interesting.

00:34:18: They have, I mean,

00:34:20: I wasn't aware of Samsung because I'm not a Samsung user.

00:34:23: I don't think I even own the Samsung device.

00:34:26: So, yeah, but that's interesting.

00:34:28: So one other thing that you've mentioned is that, you know, with moving these large

00:34:33: language models, vision, like multimodal to the phones, really the phones now come

00:34:38: with also like really excellent cameras.

00:34:40: I think mine come with like three lenses.

00:34:44: So the quality of the images of what you're able to capture videos or whatever

00:34:48: you're submitting to it.

00:34:49: is really high quality.

00:34:52: So yeah, I think that it's going to be game changing.

00:34:55: I mean, like some of these use cases, I'm already like mind blown where you could

00:34:58: show it a room and say like, Hey, can you suggest how to furnish this?

00:35:01: And it's like, yeah, why don't you even have to go to a different room?

00:35:04: And maybe, I don't know, it would be even more game changing if this kind of came to

00:35:10: the device as a standard feature rather than you have to go to ChadGPT and pay

00:35:15: subscription.

00:35:17: So yeah.

00:35:17: I think again, this is why I think it is going to be quite, I anticipate it to be

00:35:22: quite disruptive if this is basically the approach that they're going to take.

00:35:27: And I think one thing they have to take into consideration, which is really,

00:35:32: really important is stuff at scale.

00:35:35: And that's why Google always has a problem that they cannot just throw things out

00:35:40: because they have just another scale than someone like OpenAI has.

00:35:46: And...

00:35:46: That's why you really have to be sure that what you release works.

00:35:52: Like Apple building devices, hello.

00:35:54: Like if they put a server behind it and everything goes to that server, the server

00:36:00: is pretty much screwed.

00:36:01: They need to partner with Nvidia.

00:36:03: That's what they need to do.

00:36:04: Yeah, but still, there is a lack of chips.

00:36:07: So getting it to a point where that can run mostly locally on device is just

00:36:12: beneficial for the whole experience in and of itself.

00:36:15: So...

00:36:16: Yeah, I'm pretty sure this is something you If you're able to take this into

00:36:23: consideration Yeah, Apple in case you're listening Staying on the topic of it's

00:36:33: close to free in terms of your revenue For you but in related to large language

00:36:43: models so grok

00:36:45: Um, one, basically, I can, I guess that's the enumeration of the model.

00:36:49: This time we're talking about the XAI model GROK from Elon.

00:36:53: Correct.

00:36:54: Elon Musk's GROK, G -R -O -K model going open source.

00:37:01: Um, you were quite, when we talked about it earlier, you were quite excited and

00:37:04: excited about it and said that it was quite game changing.

00:37:08: It's a huge model though.

00:37:09: I looked at it.

00:37:09: It's like 340.

00:37:11: Was I?

00:37:11: I said I wasn't impressed.

00:37:13: Oh, did you?

00:37:14: Am I making stuff up?

00:37:16: But I thought it was...

00:37:17: You hallucinate.

00:37:18: I think you did mention that it's going open source.

00:37:22: I think it's game changing.

00:37:24: No, I think going open source and being uncensored, that's already interesting.

00:37:29: You have to want that.

00:37:33: And also...

00:37:36: It's completely raw.

00:37:39: So a model, when it's released, an open source model, you oftentimes read

00:37:44: something like instruct or something in the title, which means that it's already

00:37:50: trained on some data to answer in a specific way.

00:37:58: And Grok released without anything, like it's completely raw, basically.

00:38:05: Which...

00:38:06: was kind of interesting because they basically say...

00:38:11: When you say Rob, it means that there's no guardrails, no rules.

00:38:16: Yeah, no guardrails, no pre -trained behavior for being a chat model or

00:38:23: something.

00:38:24: Let's get completely...

00:38:26: And people then take it...

00:38:29: take some open source training data and train it to be an instruct model, for

00:38:33: example, so that you can have a chat -like experience with that.

00:38:37: They basically released the raw weights of that.

00:38:43: I'm not sure I think they trained it on Twitter, as far as I know.

00:38:48: I'd be surprised if they didn't, because it is where they lodged.

00:38:53: Or Twitter eggs.

00:38:56: But you get what we mean.

00:38:58: Yeah, but this...

00:39:01: Yeah, it's a huge model, yeah, right?

00:39:04: Like I said, people go ahead and quantize stuff.

00:39:07: Quantizing means reducing the bit size of the weights themselves, which makes the

00:39:12: model a lot smaller, while not losing that much performance.

00:39:17: And, yeah, I've seen some tests, basically tested on the X side itself.

00:39:29: Yeah, I'm not sure maybe with this with some more fine -tuning through the

00:39:35: community it will become or Will give our better results, but from what I've seen

00:39:42: It's no It's no model to to yeah get rid of GPT -4 But yeah, I'm open sourcing it's

00:39:53: kind of interesting move nonetheless.

00:39:55: Do you think more more companies will come to the

00:39:59: to the forefront and offer their models to convert them from closed to open

00:40:04: considering what seems to be the trend?

00:40:10: No, I'm not sure.

00:40:12: What do we have with it?

00:40:13: And even Grok, okay, they released the one.

00:40:16: Do we know if this is the basis for Grok 2 or is Grok 2 completely newly developed

00:40:22: and newly trained?

00:40:24: Then, yeah.

00:40:24: Then, okay, GPT -2 is also open source.

00:40:27: And even three, I think it's three, like normal GPT -3 is open source.

00:40:32: I'm not sure.

00:40:34: So yeah, how much it's worth, we will see.

00:40:39: But people often misunderstand open source.

00:40:43: Like open source is just, if you open source stuff, you get millions of

00:40:48: developers jumping on this.

00:40:51: Like depending on the topic, of course, but...

00:40:54: You have such a huge community, you wouldn't be able to do this in your own

00:41:01: company because you just don't have the resources.

00:41:03: And you open source the stuff and then the open source community jumps on it and does

00:41:08: stuff you weren't even thinking of.

00:41:12: And that's a huge opportunity for development and for further getting stuff

00:41:19: forward.

00:41:20: That's what Uncle Edon is looking for, I guess.

00:41:24: And that's why they outsourced it.

00:41:26: And also just to give the finger to open my eyes.

00:41:31: We're here too, breathing down your back.

00:41:35: But because I think, you know, I don't know, I just feel conflicted with all of

00:41:42: these models coming on scene and more and more models are being developed.

00:41:45: They're trained on very similar datasets.

00:41:48: We're running out of knowledge.

00:41:49: There's going to be a point where, you know, right now they, it does state that

00:41:54: they could grow and improve exponentially, but the critical piece that it needs is

00:42:00: data to be trained on.

00:42:01: So if we run out of data, then everyone kind of starts to level off.

00:42:08: And so what are we improving at that point?

00:42:10: So will models become commoditized because they're really kind of are right now we're

00:42:16: looking at.

00:42:16: what model is using what data, but ultimately, I mean, they're very similar.

00:42:22: And I think there's going to be a marginal difference between them.

00:42:25: I think models are already trained on like 70 % synthetic data.

00:42:28: Oh, they're very, yeah, they've switched that much.

00:42:32: And I think we have a, like, there are a lot of forefronts you could tackle besides

00:42:37: the raw data itself.

00:42:39: One of which is data quality.

00:42:41: I think Microsoft showed it with Orca, which is only trained on books.

00:42:45: or Phi2, which is a small language model, which also has only, I think, scientific

00:42:51: papers and really high value texts for useful training and is therefore really

00:42:58: good at reasoning compared to the sounds.

00:43:00: Which model did you say it was?

00:43:02: Phi2.

00:43:05: That's PHY2.

00:43:07: And yeah, if you look at these, so we have a lot of stuff we can tackle.

00:43:13: Also, as far as I understand, and also from some altments interviews he gave like

00:43:18: the last weeks, you kind of feel there will be another leap technologically.

00:43:26: I think we said it like two episodes ago, like system one, system one, last episode,

00:43:31: system one thinking, system two thinking, models being capable of really like taking

00:43:36: time to think something through.

00:43:39: That's something that will give us a big leap forward besides the data.

00:43:47: Yeah.

00:43:47: And yeah, but on the other side, you have like two different companies, Google and

00:43:51: OpenAI, and they both have a vision model and they use the same prompt on both

00:43:56: models and they had the same quirks I had to get rid of, like exactly the same

00:44:03: issues of not giving me the answer like I wanted, so I had to rework it.

00:44:09: But it was the same issue in both models, even if they're completely different

00:44:13: models, which I found really interesting because I know there has to be some

00:44:19: similarity in the data set, otherwise, why should it do that?

00:44:21: Yeah, and that's why I basically think so, unless you and maybe they're accommodating

00:44:26: some of the stuff with synthetic data or as you mentioned that there is a

00:44:30: technological change that leads to or some change in the approach and how we train

00:44:35: these systems or how these systems infer.

00:44:38: information from the data, like the inference of the system really changes

00:44:43: that whole process.

00:44:45: They're just going to go and kind of level off at the same performance and they're

00:44:49: going to start to look similar as what you basically you're experiencing and then the

00:44:54: difference between them is going to be very marginal.

00:44:56: That's basically what I'm trying to say.

00:44:59: So, yeah, yeah, just an observation.

00:45:02: So, okay.

00:45:03: I think we leave it with a completely news episode today.

00:45:08: After we skipped the news on the last episode.

00:45:13: We planned to talk about Devon, the AI engineer too, which was also all over the

00:45:20: news.

00:45:21: But I think we will schedule this to the next episode.

00:45:24: Let's do it.

00:45:26: And then, yeah, talk also a bit more about how you can use AI today for your

00:45:34: development purposes.

00:45:36: And yeah, I think I can finally give some practical insight into what that means.

00:45:45: And yeah, Lana can ask a lot of non -programmer questions about that.

00:45:50: Yeah.

00:45:50: And I think, I mean, that's one thing and I was just going to...

00:45:54: add my two cents of really inviting our listeners to provide us feedback because

00:45:58: we do sometimes get nerdy and you know we talk about quite technical stuff more on

00:46:04: the development implementation.

00:46:06: Do you want to hear us talk more about maybe the strategy side or do you benefit

00:46:10: from understanding kind of in the weeds of how these models work, the implications of

00:46:16: some of the news that are coming in your business and how it relates.

00:46:19: Just want you to understand like what's resonating with you and

00:46:23: that would also provide us feedback for what should we be doing less and more of,

00:46:27: or what should we be introducing as new topics.

00:46:30: So again, inviting you guys for feedback, truly appreciate that.

00:46:36: Yeah, definitely.

00:46:37: And also the development episode was out of feedback where someone wanted to know

00:46:43: something about AI and web development.

00:46:46: So we hear you, we try to consider you as soon as we can plan it in.

00:46:51: Yeah.

00:46:52: I hope that you liked this episode.

00:46:55: I think we had, it was a good chat.

00:46:57: Yeah.

00:46:57: Right.

00:46:57: Well, today was a good like, or at least I talked a lot so that's like, feels good

00:47:01: for me.

00:47:03: He's like, I can pat myself on the back and then go to sleep now.

00:47:07: So Edgar is in Germany and we have a significant time difference.

00:47:11: So it is towards the end of the day.

00:47:13: So I think he can definitely, you know, take a breather, drink some tea and go to

00:47:18: sleep.

00:47:18: I definitely had my share of words today, that's for sure.

00:47:23: Yeah, but I still hope I could give some valuable insights about what all this

00:47:31: stuff that is presented and shown on LinkedIn means and hope you get some

00:47:38: insights.

00:47:38: If you have any questions, of course, reach out to us.

00:47:41: down in the comment sections if you're on YouTube, write us on LinkedIn if you

00:47:47: listen to us on Spotify.

00:47:49: And yeah, really enjoy getting further along with our podcast and getting our

00:47:55: groove dialed in.

00:47:56: So I hope you feel the same.

00:47:59: Sorry, that was not expected.

00:48:01: And yeah, that's also part of the show.

00:48:05: So have a nice day.

00:48:07: And last words to Lana.

00:48:09: Thank you.

00:48:09: Yeah, again.

00:48:11: Thank you for tuning in.

00:48:12: This is episode, I think eight for us.

00:48:15: So appreciate your support.

00:48:17: If you've taken the time to listen to us and, you know, hit a like, subscribe, um,

00:48:22: you know, to show some support.

00:48:24: Leave a phone.

00:48:24: Yeah, I think this gives us also, we check in, um, on a weekly basis, just to see

00:48:30: what kind of feedback we're getting.

00:48:31: And again, it's, it's something that, um, we really value and it's a sign of

00:48:37: approval also for us.

00:48:39: Um, you know, as we continue to grow and that we're providing relevant topics.

00:48:44: So again, appreciate you taking the time.

00:48:46: Thank you for your support.

00:48:47: Um, and again, invite anyone to provide us feedback so we can do more of what is

00:48:52: impactful to you and what information that you find valuable would love to hear it

00:48:59: and we'll bring it, um, into the episodes sometime in the future.

00:49:02: So thank you again for listening.

00:49:05: Bye.