EP 005 - Groq, OpenAI Lawsuit, Figure AI, and Importance of Explainable AI for Businesses

00:00:00: Hello and welcome to episode number five of the AI Boardroom.

00:00:06: And of course, I'll call you Lana from that point.

00:00:15: Lana is really cool.

00:00:17: I think it's a cool short name.

00:00:18: I like it very much.

00:00:21: How are you doing?

00:00:22: What was your week?

00:00:23: busy week, I think for both of us.

00:00:25: We were just kind of chatting.

00:00:27: Yeah.

00:00:29: We just...

00:00:30: enjoy a chest beforehand as much as I do the actual podcast itself.

00:00:35: Well, we launched officially this week, so I think that's what that came a lot of

00:00:38: excitement and a lot of just kind of work behind the scenes, but really excited to

00:00:43: finally get some of our episodes out.

00:00:45: Yeah, and I think that the first one and the fourth one did the best.

00:00:52: And I'm pretty confident that the further we go, the better the episodes will get.

00:00:57: So let's stay with that.

00:01:01: Let's start with the news section.

00:01:03: That's what we're working towards by releasing all episodes at once so that we

00:01:08: can do them weekly and include the news.

00:01:12: And give you guys some context and what...

00:01:14: this means for you, of course.

00:01:17: Let's start with, I think one of the biggest or bigger news this week is like

00:01:23: Mr.

00:01:23: Large launched and at the same time they announced that they partner up with

00:01:27: Microsoft, Microsoft's investing money.

00:01:32: Yeah, that was the headline basically.

00:01:37: I even heard someone saying like.

00:01:41: I think they don't believe anymore in OpenAI what they're doing.

00:01:44: I was like, mate, they did this, everyone.

00:01:47: Like you can have basically every open source LLM on Microsoft's platform and now

00:01:54: you get the Mistrille like additionally.

00:01:56: Microsoft, they don't care about any of the AI companies, I guess.

00:02:00: They only care about getting the stuff into their products.

00:02:03: Well, I think, and it's also, it's an example of diversifying portfolio, right?

00:02:09: So you're kind of investing into multiple things, hoping that one will stick.

00:02:13: And I think, was it December where this whole kind of multi -day havoc that

00:02:20: happened at OpenAI?

00:02:22: I think that kind of also, you know, kind of left an aftertaste, I think, in

00:02:29: Microsoft's mouth, because they're like, oh, well.

00:02:32: honestly, it's their infrastructure that failed, so...

00:02:35: I'm not sure who's to blame on that.

00:02:38: No, I mean the board firing Sam Altman.

00:02:43: Yeah, because it's like if.

00:02:44: outage.

00:02:44: But yeah, you're right.

00:02:45: Yeah.

00:02:46: Yeah, so I think it's what could have happened.

00:02:48: I think they then hired Sam Altman into their organization and then Sam ended up

00:02:55: coming back.

00:02:56: And it's like, there's just so much happening.

00:02:58: It was such a life to the November.

00:03:01: It was really nice.

00:03:03: I really was going on Twitter.

00:03:05: I'm never on Twitter, honestly, but I was going on Twitter seeing if someone of them

00:03:09: was posting something new, like hourly or something.

00:03:15: Yeah.

00:03:17: Yeah.

00:03:19: Yeah, it was kind of that.

00:03:20: that's basically, I think maybe a big move why is because open AI doesn't want to, or

00:03:25: I'm sorry, Microsoft doesn't want to put all of their basket into open AI.

00:03:30: And they're like, okay, well, how do we, what alternatives can we have?

00:03:33: How do we diversify our portfolio foundational models that we offer to our

00:03:37: customers?

00:03:38: I think that may be one of the reasons I'm not saying that's the only one, but that's

00:03:43: just my take.

00:03:43: also always worked like that.

00:03:45: So I'm working in the Microsoft universe for my whole career.

00:03:50: And I'm pretty familiar with how they work and what they're doing.

00:03:54: And to be honest, they always try to spread all the lines out.

00:04:02: And I always was like, if you have this acronym FANG for Facebook and Apple and

00:04:09: Netflix and Amazon, and Microsoft's not even mentioned it.

00:04:13: And I always was wondering why, because of course they hadn't had the best years,

00:04:19: like in the beginning of the 2010s.

00:04:21: But to be completely honest, they always were so hugely spreaded everywhere that

00:04:28: they're not going anywhere.

00:04:30: And I was always wondering why Silicon Valley is looking at them as a bit of the

00:04:37: old Microsoft guys.

00:04:40: That's what you say.

00:04:42: and if Balmer would have stayed president, they most likely would have died.

00:04:48: But yeah, everything got a swing with the new CEO.

00:04:53: But yeah, what I'm excited about, I'm a really big fan of the Mistral models.

00:04:58: I think they are all like already at a level without Microsoft's funding.

00:05:05: And as we all know, compute is like the one bottleneck for everyone doing stuff

00:05:10: with AI and honestly for AGI most likely.

00:05:15: So if we would have had like infinite compute, then AGI would be like right

00:05:19: around the corner.

00:05:23: So do you have any experience working with Mistral's models?

00:05:27: Yeah, I use the Mixed Drill, the Mixed Drill of X -Bots model.

00:05:32: That's like the one I use when I use a local one.

00:05:36: Because it's honestly, it's basically on one level with GPT, with GPT 3 .5.

00:05:44: And it's some task that even performs on the level of GPT -4 and it's running

00:05:49: locally on my computer.

00:05:50: It's really cool.

00:05:51: I really like this model.

00:05:53: I'm still like on a side mission to figure out how to integrate small models into

00:06:02: production.

00:06:05: And performance is of course the main reason for that.

00:06:09: And I think we also said in the last episode that thinking about using a small

00:06:14: model for a specific task might be the better way to go than using a general one.

00:06:18: is it, Dino?

00:06:19: How much smaller is Miss Jill?

00:06:21: is not that tiny, it's I think 48 billion parameters, but it's 8 times 7, I think.

00:06:31: So there are 8 or 7 models, R7 or 8 billion parameters, I'm not sure which way

00:06:37: around.

00:06:38: And of course there has to be some kind of routing router, like picking the exports.

00:06:45: What I wasn't aware of is that...

00:06:48: the inference goes through the whole stuff.

00:06:51: So it's not like in the beginning, there's a router picking two models and then they

00:06:55: work on that, but like it's going around.

00:07:00: As far as I understood, I tried to get some grasp on how mixture of experts work,

00:07:08: but yeah, you have a really, really good model and being at a position where a year

00:07:14: after GPT launched,

00:07:17: having an open source model just being on par, I thought it's a big achievement.

00:07:26: Yeah.

00:07:27: take a stab at also explaining why having multiple of these models is really key.

00:07:34: I may be wrong, but please correct me.

00:07:36: But the idea of using just one huge model is that typically it's a one way street.

00:07:44: So you basically ask the system and it's called basically fast thinking.

00:07:49: So for those who have read the book, Thinking Fast and Slow, so it basically

00:07:53: inferences the or

00:07:55: you prompt the system, it goes and creates an output very quickly.

00:07:59: And so you've seen probably that within a few seconds, you'll see an output.

00:08:05: It's basically just going one way street, it goes, picks the right kind of words,

00:08:10: and then it gives you the output.

00:08:12: The power between these different models is that now you have an opportunity to

00:08:17: think slower.

00:08:19: So instead of running that prompt through one system, it's basically...

00:08:24: throwing that across to another model and then it thinks again and it looks, okay,

00:08:31: am I equipped to basically answer that question in a much more detailed, deep

00:08:36: fashion?

00:08:37: No, and then it just continues to inference other models until it feels

00:08:40: confident in providing an output.

00:08:42: But I think the big difference, and I think we talked about this more, I'm just

00:08:45: curious on your take, is that because you're ultimately running that prompt

00:08:50: through multiple of these systems, that's where the slow speed comes in.

00:08:54: because instead of having you prompt one system, you're now prompting ultimately

00:09:00: seven or eight at the same time.

00:09:03: it depends how you classify speed because for example GPT -4 is also a mixture of

00:09:08: expert model and that's a bit of like...

00:09:15: Usually you might think or like I think it comes naturally to think about having

00:09:20: several small models.

00:09:22: How can they be better than each model itself?

00:09:27: Like how can be the sum of the parts bigger than the one?

00:09:30: But...

00:09:30: But it is, and it's really fascinating for me to see.

00:09:36: It's one of the most interesting technologies we have at the moment, is

00:09:41: this mixture of experts model, because you really have different small, fast

00:09:47: -inferencing models combined.

00:09:49: And the router always picks two and works with two of them.

00:09:52: So...

00:09:53: as far as I understood.

00:09:55: There will be different strategies and stuff like that.

00:09:57: I think that's something where then the provider of the model can fine tune stuff.

00:10:03: But in the end, it's faster than having the one big model because if you would

00:10:09: have to influence the 48 billion parameter model, the output would be worse and it

00:10:16: would be slower.

00:10:17: But influencing to seven billion is like, it's just a lot faster.

00:10:21: And that's basically, you have of course a bit of overhead for the routing and stuff

00:10:26: like that, but in general, if you have an index which takes into consideration

00:10:34: quality of the answer and speed, that makes sure of expert's model is faster if

00:10:42: you keep the quality the same.

00:10:45: That's interesting.

00:10:46: So do you think also compute, like the power of compute that you have available,

00:10:51: does that play a part as well?

00:10:53: Always.

00:10:54: I think in the end, everything comes down to math and you have to multiply vectors

00:10:59: one way or another.

00:11:00: There is no secret key.

00:11:04: The only reason why we can even do all the AI and vector stuff is because someone,

00:11:11: for like some six, seven, eight years ago, found a more efficient way to multiply

00:11:16: vectors.

00:11:17: And that's the only reason why we even can have...

00:11:20: billion parameter models, like trillion parameter models, because of course the

00:11:25: hardware gets better, but in the end you have to do math, basic math.

00:11:29: And so it's also, if the model has a certain size, certain parameters, you

00:11:36: basically are stuck with the same, with the same inference time on the same

00:11:42: hardware.

00:11:43: There is no secret sauce.

00:11:44: For example, Google, I think they just have really, really good hardware.

00:11:48: But the model itself, if it's like if GPT -4 and Gemini were just the exact same

00:11:55: size on the exact same hardware that would influence the exact same time, even if

00:12:00: there are two different models.

00:12:03: Yeah, that's kind of interesting.

00:12:06: In the end, it's all about probabilities.

00:12:10: But still, I think it's really interesting to...

00:12:18: to look at the stuff and to see how we can use small models.

00:12:23: Also, in use this week, there is a new API called Grok with a Q in the end.

00:12:31: And they have extremely fast inference.

00:12:33: And they also have this mixture of expert model for Mistral, like this Mistral

00:12:37: online.

00:12:39: And it's basically, it's close to instant that you get the answers.

00:12:46: which is kind of cool because like I said, it's on the level of GPT 3 .5 and with

00:12:51: grog API, you could influence it like in basically no time.

00:12:54: That's really, really cool.

00:12:58: I think you mentioned that it's also a library of different models too, right?

00:13:02: It's not a single...

00:13:04: have LAMA 2, 70 billion, and the mix flow model, I think that's the two that they

00:13:11: offer right now with more following.

00:13:13: And they're also in the beta of the whole stuff.

00:13:20: So what makes them different though?

00:13:21: So if they're using Laman established model, so what is GroG doing different

00:13:25: than if you were just to basically get that model from one of the cloud

00:13:32: providers?

00:13:33: just fast.

00:13:33: I don't know if they might have specialized CPUs for that.

00:13:38: Compute units.

00:13:40: Because there are chips, they even as one company, they do transformers in silicon.

00:13:47: They bake the transformer architecture into silicon and the chip is the only

00:13:52: thing that can do is do inference on models.

00:13:56: But that lets you...

00:13:58: do stuff in near real time.

00:14:00: And if you go to multimodality and have audio processing and stuff like that, and

00:14:05: live video feed processing, then it gets really interesting.

00:14:08: Of course, they are screwed if transformer models are outdated and we have something

00:14:12: new.

00:14:14: But I think it's worth the mention.

00:14:17: I'm not sure how Grok does it.

00:14:20: I only know that they're really, really fast.

00:14:23: And I can compare it to basically my...

00:14:28: experience with MixedRill.

00:14:30: And yeah, if they can keep it up even on scale, I think it's pretty interesting for

00:14:34: small stuff like summaries and things like that, because they're basically a lot more

00:14:40: usable.

00:14:40: Also, I've seen one demo using it for a live call script retrieval.

00:14:47: So they did the whisper stuff, they then caught the Grok API and they were just a

00:14:52: lot faster in retrieving the right documents.

00:14:57: But you suspect that it's hardware mostly?

00:15:00: You suspect that it's mostly improvement in hardware?

00:15:03: Not the...

00:15:04: I think the cloud is just focused on inferencing.

00:15:12: That makes sense.

00:15:14: Should we talk about Elon Musk's lawsuit?

00:15:17: Hahaha, yeah, that's honestly the other big elephant, the big Elon in the room.

00:15:25: so Elon Musk sues OpenAI for reaching the OpenAI's original kind of nonprofit

00:15:35: mission, which I think we talked about a little bit before, but ultimately, you

00:15:41: know, and I'm looking at readers, it basically says that, you know, Musk

00:15:45: lawsuit alleges a breach of contract, saying that Altman...

00:15:49: originally approached him to make an open source nonprofit company, but the startup

00:15:53: established in 2015 is now focused on making money.

00:15:58: So, and the big question I think we talked about is like, why is Elon Musk even suing

00:16:03: in the first place?

00:16:03: I think you had some kind of interesting ideas to offer there.

00:16:10: Yeah, I always think on what ground but to be honest from from like for me being a

00:16:16: German sitting in Germany and looking at US law is always funny Like well,

00:16:22: honestly, I've seen suits are basically an expert in American law To be honest Elon

00:16:34: is Was invested into the opens into the

00:16:40: like nonprofit part as far as I understood and was sitting there on the board.

00:16:46: And like, I'm not sure how things went down, but shouldn't he have been aware

00:16:52: back then that they are going to make a new entity out of that that is for profit?

00:17:00: But he just didn't invest and now he's like sad about that.

00:17:03: I don't know.

00:17:04: It's like.

00:17:06: I think the other big question I think we both asked, it's like, ask.

00:17:11: You have so many different other initiatives to focus on.

00:17:14: Why the heck are you even spending time suing OpenAI?

00:17:19: And I mean, I mean, open, I mean.

00:17:22: the same thing why he thinks about buying Disney.

00:17:24: He's a man with a lot of ideas, with a high IQ and a lot of money to spend.

00:17:32: I think he's not even caring about if he's broke tomorrow.

00:17:37: He loves doing an idea and his being so much that, yeah.

00:17:49: The only thing, like I liked Musk for a long time and I liked what he was standing

00:17:54: for in terms of pursuing things that no one thinks are possible.

00:18:01: I read the, or have had the audio book of the autobiography, which was not written

00:18:08: by him, but it was written by someone who just wanted to write about him.

00:18:12: And he was...

00:18:15: He was like into going and like uncovering Elon.

00:18:22: And like the further he went down the book, he just got to understand him more.

00:18:30: And I think that's also something that like sticks with me.

00:18:37: But the last years, I'm not sure if he's getting ahead of himself a bit.

00:18:45: And that's sad to see because he's brilliant and I think his methodology is

00:18:51: questionable.

00:18:54: If you ask ex -employees, they mostly broke under him, but they also achieved

00:19:02: things no one thought are possible.

00:19:05: So the big minds are always kind of broken themselves.

00:19:12: So...

00:19:14: I'm not sure.

00:19:15: I'm all for getting into the future and being like, pursuing the future.

00:19:26: But meanwhile, he's talking a lot and delivering not that much, so...

00:19:32: Are you talking about the book that you're talking about?

00:19:36: Is it by Walter Isaacson?

00:19:37: Yeah, I think it was the one out.

00:19:39: Yeah, I love his books.

00:19:42: So I think it's on my reading list.

00:19:45: So I think I've started actually reading the book.

00:19:49: I haven't finished it, but all of the books by Walter Isaacson are really

00:19:54: excellent.

00:19:54: So for those who are interested in what we're talking about.

00:19:58: Yeah, and I thought it's inspiring is maybe not the right word, but having such

00:20:07: a good deep look into someone like Elon Musk, who's a really special character,

00:20:13: who has a special background, it's kind of interesting.

00:20:17: But like I said, I think he got a bit ahead of himself and I'm a bit iffy on him

00:20:25: right now.

00:20:27: I hope that he finds some time to reset a bit and focus on what's actually in front

00:20:33: of him.

00:20:34: Do you think it's also this lawsuit kind of a part of, you know, his own models in

00:20:43: X competing with OpenAI, or do you think that has nothing to do with Y?

00:20:48: course, I think he has an agenda through everything he does.

00:20:51: He didn't buy Twitter out of a bad Sunday or something.

00:21:01: I think he has a lot more of a bigger picture and plan than we all think he

00:21:06: does.

00:21:08: But like I said, I think the death of every great mind is the hubris.

00:21:17: going along this.

00:21:19: I talk to so many really good people that are successful business owners and they

00:21:26: always have this notion of hubris, how is the name of the hubris right?

00:21:34: Pronounced.

00:21:35: And you always think like mate, come back to earth.

00:21:39: You're not the messiah at all.

00:21:44: And...

00:21:46: The further you get and the more successful you become, the more I think

00:21:50: you're inclined to get ahead of yourself.

00:21:56: And that's always, or like oftentimes it's a bad sign.

00:22:02: And you should take a step back.

00:22:05: Yeah, slow down.

00:22:06: But talking about Elon Musk, they also have a robot, a humanoid robot.

00:22:12: And the whole robotics space kind of got a lot of drive with LLMs because basically

00:22:22: the machines get new brains with all the transformer technology.

00:22:27: And that's kind of interesting.

00:22:29: And there was a new investment into a company called Figure AI.

00:22:34: Mm -hmm.

00:22:34: And they built humanoid robots.

00:22:36: I think the demo showed them making a coffee.

00:22:40: And they're like at the beginning, but they achieved to get like $675 million

00:22:47: this week.

00:22:49: And one of the investors was OpenAI.

00:22:51: Another investor was Microsoft.

00:22:54: And maybe it's also something that triggered Elon because Tesla is building

00:22:59: the humanoid robot themselves.

00:23:01: So.

00:23:02: doing it.

00:23:02: I think Imettas also have their own version of a robot too.

00:23:06: They posted a video that actually has a pretty great gait in comparison to future

00:23:12: AI.

00:23:12: I mean, I don't know as much as into the technology.

00:23:16: I haven't dove deep into it, but I mean, just how was assessing as to like, how is

00:23:22: it, you know, walking and actually performing on like the physical space?

00:23:26: Cause I mean, there's been a lot of improvements there as well.

00:23:31: Future AI is just, it still seems a little bit behind, but as you say, it's not only

00:23:36: about the physical, it's about its capabilities.

00:23:40: Figure A, I'm sorry.

00:23:41: Jeff Bezos is also invested as I just seen Like there's a lot of money going into

00:23:49: that like it's like like always I feel like it was it was even worse, but it's

00:23:54: still like if you have an idea There's some AI label somewhere in there But as

00:24:00: far as I understood Amazon also

00:24:03: is like, or Jeff Bezos especially is really interesting because Amazon builds

00:24:06: its own robots for their facilities too.

00:24:10: So maybe, maybe it has something to do with that.

00:24:13: But yeah, there's a lot going on.

00:24:15: I'm really interested.

00:24:16: I'm also there was, was this UC Berkeley or Stanford?

00:24:20: I'm not sure.

00:24:22: They published a source robot.

00:24:24: They published the software and the part list and you can, like the off the shelves

00:24:30: parts.

00:24:31: Mm -hmm.

00:24:33: And it's really, really interesting.

00:24:35: I think if you buy all the parts, it's like $32 ,000.

00:24:42: But then you have basically a robot and you can, and here's a learning mode, so

00:24:45: you can take the controls, do stuff with that.

00:24:48: And then the demos were really cool.

00:24:51: It makes you X and stuff like that.

00:24:53: So it's really able, it just has some robot arms just for grabbing.

00:25:00: And...

00:25:01: Yeah, it really was like a really interesting project, especially thinking

00:25:06: about it's open source and everyone can build this at home basically.

00:25:14: Yeah, kinda.

00:25:16: Yeah, for me it was cool to think about this because I was thinking, okay, maybe

00:25:21: I'll put up a 32K, buy the stuff, build this thing and see how far the bass model

00:25:28: goes.

00:25:29: And then you try to just improve on that, right?

00:25:32: Maybe you get another base which is not as bulky.

00:25:36: Maybe you get even feet to walk if it's even necessary.

00:25:41: Maybe you improve on the whole shell to make it more, less looking like a

00:25:51: prototype.

00:25:53: And then, yeah, it might be necessary for you to charge then up.

00:25:58: of 60, 70K for such a role.

00:26:02: But like, what is that if you have, compared to a salary of a year salary of a

00:26:08: human?

00:26:10: Yeah.

00:26:11: do you think that the big question that I think is pending in my mind, have they

00:26:16: built a vessel for AGI, but AGI is not really here?

00:26:20: So how effective would these robots be, even if you get them to, you know...

00:26:24: Do they need that?

00:26:26: And that's the question, yeah.

00:26:29: In order for them to operate, do you think that AGI needs to be reached or do they

00:26:35: need to be also task specific?

00:26:36: And that's what we would consider sufficient.

00:26:38: I think AGI might be necessary at some point to solve the underlying theoretical

00:26:45: problems, basically.

00:26:46: For example, having a less friction motor or like, I don't know the English word,

00:26:59: but like everything that moves breaks at some point.

00:27:05: To get like.

00:27:06: If you look at human hands, we move for 60, 70 years if you look without any

00:27:12: issues.

00:27:13: So like they're really, you really don't need a lot of service for human body,

00:27:18: especially considering how much it moves.

00:27:21: So biology has that solved, we did, we do not.

00:27:27: And that's something AGI I think might help like how to be able to recreate what

00:27:33: biology has already.

00:27:34: been able to do over evolution because it needs so much processing, so much time and

00:27:43: so much research and having AGI could speed that up.

00:27:46: It's the same for everything.

00:27:47: It's the same for climate change.

00:27:49: That's what AGI is really interesting for, to solve the problems we just don't have

00:27:52: the time to with our human minds.

00:27:56: And...

00:27:57: your perspective, you think that if we train these robotics, and I'm just kind of

00:28:02: thinking like, well, how widely spread do you think they will be in five years?

00:28:07: And what would be a fair expectation of them for them to, again, solve specific

00:28:12: tasks around the home?

00:28:14: Is it still going to be?

00:28:16: Yeah.

00:28:18: us in the home it's still like 20 years off.

00:28:21: Maybe 10.

00:28:23: But I think the rich people, like really rich people, they might try it earlier.

00:28:28: Musk might have one version of the robot already at home.

00:28:31: I don't know.

00:28:33: But for being like a car, which you just own, I think that's a bit off still.

00:28:42: That said...

00:28:44: for companies, that's a really interesting development because not only to get rid of

00:28:53: actual humans, but to really get dangerous stuff done by machines.

00:29:01: Maybe even that, even if you have to have someone controlling it remote, if that's

00:29:07: working properly,

00:29:09: to have it work remote with an intelligent core to avoid mistakes might be something

00:29:16: to think about because it's already more efficient and you might be able to be more

00:29:22: productive as a company.

00:29:25: So yeah, I'm honestly a bit irritated because why Boston Dynamics was because

00:29:33: like they were so far ahead in robotics and like in every way, shape or form.

00:29:38: Why aren't they putting out new stuff with integrated transformer models and stuff?

00:29:45: It's a bit weird.

00:29:48: I don't know.

00:29:51: But there is, you know, one other use case that I think you've mentioned.

00:29:55: This could be game changing for bigger companies.

00:29:57: So I think the hospitality industry, they've had some track record of using

00:30:04: robotics already.

00:30:05: So they're probably not as advanced as the ones we're talking about here, but, you

00:30:09: know, actually answering questions.

00:30:10: I don't know if you've seen them, but there's some versions of robotics.

00:30:14: I'm trying to look it up right now, but.

00:30:16: where I've seen them actually bring stuff to the rooms or clean.

00:30:23: There's other ones that greet people coming in through the doors and things

00:30:28: like that.

00:30:28: So this could be maybe a use case, as you kind of mentioned, maybe for companies to

00:30:33: leverage them for things.

00:30:36: But I'm just curious, how would we be able to feel it in the industry in the next

00:30:40: couple of years?

00:30:40: I think it's probably these...

00:30:43: customer experience, so like maybe the hospitalities who we would anticipate

00:30:48: adopting them first.

00:30:49: Maybe, but to be honest, I'm not sure.

00:30:53: At least we have like the town I lived before in Hamburg, we have like an Asian

00:30:59: restaurant and they have a robot bringing you stuff.

00:31:02: But it's more of like...

00:31:05: It's more of like a programmable wheel, programmable tray on wheels.

00:31:11: And then you have to take the stuff from the tray and then it also has like a

00:31:15: display with a smiley face and stuff.

00:31:18: And it's just uncomfortable to be honest.

00:31:21: It's just making me feel uncomfortable.

00:31:24: And that's, I always said all the technology in the world will not replace

00:31:30: human interaction.

00:31:32: and human hospitality because, or at least that's the last frontier that will be

00:31:38: replaced.

00:31:40: Or maybe everything, like I think we had an episode two or so, everything human

00:31:45: made will be just so expensive and like you have to go the artificial and

00:31:50: automated route.

00:31:53: Yeah, I don't know.

00:31:55: We'll see.

00:31:56: Yeah, no, I mean, amazing developments nonetheless.

00:31:59: I think, yeah, we'll just have to kind of watch.

00:32:01: But I think it's really interesting that how many of these companies are emerging,

00:32:05: as you mentioned, like Boston Dynamics is not kind of even competing in this space.

00:32:09: But now you have all of these other players like Meta, as you mentioned, the

00:32:15: Figure AI and others emerging out of thanks to large language models and then

00:32:21: them being smart enough, leveraging that technology to actually better.

00:32:27: their robotics, I think in the very short term.

00:32:29: So it's exciting to see.

00:32:30: Yeah, and I also think if you want to start a new company now or you want to

00:32:34: have some like real cutting -edge technology company, look into robotics.

00:32:40: I think that's honestly one of the best bets you have because in an economy where

00:32:46: AI is leading everything and everyone and AI is making for good robots, you are the

00:32:53: one, you want to be the one building the robots.

00:32:56: So I think...

00:32:59: from that point of view.

00:33:01: And you don't even have to start fresh, I think the just the building capacity we

00:33:05: will need for that stuff will be so high.

00:33:08: You might even focus on building motors and stuff like that that are more

00:33:14: efficient or come up with some efficient part that's allowing the robots for less

00:33:19: service or longer a lifetime in general.

00:33:24: Because that's my biggest complaint.

00:33:27: I was like, I was driving.

00:33:29: through the city after autumn.

00:33:34: All the leaves were on the streets and no one bothered getting rid of them.

00:33:40: And then I just, like, I was going to the train station, I was thinking about how

00:33:46: could a machine look like that does this?

00:33:49: And always, every time when I think about robotics and stuff like that and automate

00:33:53: stuff with robots, I always come back to the service cost.

00:33:57: and all the things that can go wrong in a certain operation where the robot is not

00:34:05: flexible enough to get rid of it and you have to step in as a human.

00:34:10: And I think we're still at an age where robotics are not as...

00:34:17: or need so much service that they get impractical for a lot of stuff.

00:34:22: Mm -hmm.

00:34:25: Well, we'll see.

00:34:25: I could imagine the best ideas I have for automating stuff with machines is always

00:34:30: connected to drones.

00:34:32: For example, like drones that are able to paint a wall or just set up four drones in

00:34:43: a room for every wall.

00:34:44: They fly up and down, room is ready, it's painted.

00:34:48: So that's stuff that I can imagine that also from a mechanical standpoint can

00:34:53: work.

00:34:53: But that's what you have to think about.

00:34:55: Like look at Tesla.

00:34:56: They built cars for now close to 20 years soon, and they still don't get their doors

00:35:02: right.

00:35:08: Building stuff is hard.

00:35:10: And I think, you know, as long as, even with all AI systems, is you kind of build

00:35:15: it, you first need a signal that it works.

00:35:17: And so maybe that's where they're kind of doing it in phases, right?

00:35:21: So you kind of have to, you can't solve for everything at once.

00:35:24: And so I think probably, as you kind of mentioned, yeah, and if there's promise...

00:35:29: the Vision Pro now, Apple released it.

00:35:31: It's a first -gen product.

00:35:32: Nobody knows what to do with it.

00:35:35: And maybe like five years down the line, we were like, of course this was going to

00:35:41: be fun and entertaining and the future.

00:35:46: Like after 10 years, everyone's smarter.

00:35:50: Yeah, but I think some of the things that you've kind of mentioned, maybe even the

00:35:55: technology isn't here.

00:35:57: But still, what we've built, something is better than nothing.

00:36:01: So.

00:36:02: And we have a lot of stuff that has to be done.

00:36:05: We don't have enough people to do it.

00:36:09: So robots might be one way to fix that.

00:36:15: Just saying.

00:36:17: We'll continue to watch and report back what we kind of understand and know, but

00:36:21: yeah, definitely some really interesting developments this week.

00:36:25: So should we dive into our next topic that we want you to kind of spend a little bit

00:36:32: more time discussing?

00:36:33: Or do we have other things to dive into?

00:36:36: yeah, that's your topic from Friday's post, I guess, right?

00:36:42: On LinkedIn.

00:36:43: So yeah, then who would be better suited to introduce us to the topic?

00:36:47: Where's the robot?

00:36:51: I'm really glad you're not a robot.

00:36:54: So I think I'll just kind of give you give folks a preview as to what I was talking

00:37:01: about.

00:37:01: So there's been, you know, a project that I was working on.

00:37:04: There's multiple.

00:37:05: I mean, this is an application for most products, and it's going to become more

00:37:10: and more important for companies to consider explainable AI as part of their

00:37:16: solutions right from the beginning.

00:37:18: For regulatory reasons right now, not everyone's kind of adopted those

00:37:22: regulations.

00:37:23: kind of cohesively across the world, especially in the United States.

00:37:27: We're a little bit behind other than California and maybe a few other states

00:37:30: that have some regulations, but it's just maybe not as widely spread.

00:37:35: But as you kind of build, and one of the things that I was working through is

00:37:40: there's just a gap in understanding and why we need explainable AI in the first

00:37:45: place.

00:37:45: So I just decided to write a post about it that I think as we build these solutions,

00:37:51: it is so important to...

00:37:53: bridge the gap between people considering AI as a black box and understanding what

00:37:59: its output is.

00:38:00: And so the way to do that is explainable AI.

00:38:03: And so what it is and how it works, but we'll kind of talk about more, but that's

00:38:09: one of the essential pieces is, you know, if no one really adapts AI in your

00:38:14: organization, you can't capture value from it.

00:38:17: And so if people don't trust AI and how it works, then...

00:38:22: How do you build that trust?

00:38:23: So you have to start pulling back the curtain and really explain how AI works.

00:38:28: There's multiple ways to do it.

00:38:29: So there's change management, communication practices that you could

00:38:34: implement or plan for.

00:38:36: But how do you actually surface evidence in your AI solutions, basically the front

00:38:44: end user interface or some interaction interface that your consumers are

00:38:49: basically

00:38:51: communicating with these AI models.

00:38:52: And I think that's what my post was about.

00:38:54: And it picked up a lot of conversations, I want to say.

00:38:58: And so I was like, yeah, I think this is something worth kind of talking about.

00:39:02: And just curious if other listeners have had some experience with this.

00:39:08: But I do think that there are industries that do this better than others.

00:39:13: And again, part of it is being regulated.

00:39:16: So you'll see law.

00:39:20: medicine, like healthcare, being one of those that, you know, there are evidence

00:39:25: -based industries.

00:39:27: So they rely on evidence to reach their conclusions.

00:39:29: And so they expect everything that is implemented to have evidence behind it.

00:39:34: So they're not just going to take a decision, just point blank.

00:39:39: Yeah, so they need to surface something behind how the heck did that, you know,

00:39:44: machine learning engine or larger language model surface that evidence?

00:39:46: And this is why I think initially,

00:39:50: we heard about, like, well, can we source sources?

00:39:52: Like, we can't trust that output because we can't even know, we don't even know

00:39:56: where it's sourcing that information.

00:39:57: We can't just take it point blank.

00:40:00: You know, because, and then there's also a risk because of hallucination.

00:40:03: So how can we trust this output if you're first not even surfacing the evidence and

00:40:09: telling us like how this, you know, model came with it, with this output?

00:40:14: And I know you more than anyone else, Edgar, are working on hallucinations

00:40:19: and...

00:40:19: One of the things that people want to know is, well, how the heck do we avoid them?

00:40:25: How do we build that trust that truly what the machine had output is true?

00:40:32: I think we also wanted to do an episode about prompting, but I might pick up some

00:40:42: strategies from there.

00:40:44: So basically, yeah, let's just think about a large language model and how it

00:40:52: generates information.

00:40:53: It gets text and it takes parts of the text and tries to add to that.

00:41:01: and create an answer step by step, token after token.

00:41:09: So it's not really possible to know what route has this token went.

00:41:19: And additionally to that, said route wouldn't even say as anything because we

00:41:24: don't know which parts of the brain of the LLM do which decision.

00:41:30: So one way to get this straight with large language models especially is to go ahead

00:41:39: and ask the model to not only generate an answer but also generate the reasoning for

00:41:43: that.

00:41:45: That in and of itself works partly but it's also tricky because if you then have

00:41:51: a trained large language model which knows like.

00:41:55: that you asked for the reasoning steps, it starts to hallucinate in the reasoning,

00:42:01: which doesn't make sense at all.

00:42:04: So yeah, that's why we often use rec and stuff like that and lower temperatures,

00:42:10: like lower creativity on the models to reduce the hallucination part, which yeah,

00:42:17: makes the whole topic really, really interesting because for like, just from a

00:42:21: conceptual standpoint, from a mathematical standpoint,

00:42:23: Hallucinations are basically not solvable as far as I understand them right now.

00:42:29: Because, yeah, it's just the way transformers work is making them just

00:42:37: generate the next token.

00:42:38: And again and again and again.

00:42:40: And that's all that we have now is just the result of the next token generation

00:42:45: being that good that we can actually use it.

00:42:49: And so, yeah, what you, I think you told about,

00:42:53: told me about a product which you can buy for explainable AI.

00:42:59: How does such a product look?

00:43:02: So I'm probably not by, but you could implement some of these tactics.

00:43:07: But going back to what you just kind of said, and I've been really impressed by,

00:43:13: you know, Chagy PT and like maybe Proplexity also, because they claim to

00:43:17: surface evidence.

00:43:18: And I sometimes even go into that system and I fact check another LLMs output that

00:43:24: I'll literally just take the output and then I'll premise or like prompt the fact

00:43:28: check and then I'll put the whole output in quotes.

00:43:32: But that's basically what you just kind of said, and I just wanted to emphasize that

00:43:35: again, is like these transformers, the way that they're built, they don't house these

00:43:41: libraries of text, like these billions and billions of data sources that they're

00:43:45: trained on.

00:43:46: All they do is they find patterns in language and they just store gestalts.

00:43:50: So it's just basically the essence of speech, like the pattern and language and

00:43:55: everything like that.

00:43:57: And all of those patterns come from like so many different...

00:44:01: sources that it's really hard for you to, as you mentioned, like, know, did this

00:44:06: combination of words come from this source or this source or like what idea came from

00:44:11: whatever.

00:44:11: So when I think Chad GPT, you know, for the longest time, they didn't have the

00:44:16: ability to source data because it's really hard exactly for that for that reason.

00:44:21: Like, how do you source where in the brain that information came from?

00:44:25: So I think I was.

00:44:27: And it comes from maybe misunderstanding how these systems work, because I do think

00:44:32: that some people do believe that all these large language models do is they're just

00:44:37: better data curators, but they're not.

00:44:40: They don't have libraries that they just look up information from.

00:44:44: But what's really impressive is how, I think it's perplexity.

00:44:51: They have a, you know.

00:44:52: a group of models that they use, but JGPT specifically, that they now can source

00:44:56: data, so they must be indexing something or somewhat to be able to even surface

00:45:02: some evidence behind not everything.

00:45:05: They don't do it in line with every single sentence, but just the fact that they're

00:45:10: able to do that, like, that's really impressive.

00:45:13: And I think that's one of the most powerful things LLM's brought to us

00:45:17: because thinking about a language processor which you don't have to pre

00:45:21: -program, but it's still able to classify text and that's the beauty of it and you

00:45:29: can't do so much with it.

00:45:30: And that's...

00:45:32: what we are working on too is to make the language processing part the genuine value

00:45:40: addition.

00:45:41: So because what are we humans doing different in a lot of circumstances?

00:45:47: We get in some text, which is basically a pattern containing information.

00:45:54: And we try to find the right pattern matching to the initial request by

00:46:00: answering in words.

00:46:02: So.

00:46:02: That's the interesting part.

00:46:05: You have to, like we oftentimes say on the podcast, you have to break with the old

00:46:12: thinking you have about how a computer works and how you program stuff and how

00:46:17: you develop stuff and even think about solutions.

00:46:20: Because in the end, you have to think about it more like how language...

00:46:26: works and how language processing works and that's how you can come by how to work

00:46:31: with LLMs.

00:46:32: And that's also where you better understand how hallucinations are going

00:46:36: because it has to generate the next token.

00:46:38: It will do this even if it hasn't have a proper answer because not having a proper

00:46:45: answer is just it gives an output which has a lower probability of being right,

00:46:49: but it's still an output.

00:46:50: And that's how the hallucination is going.

00:46:54: going to happen over and over again because it has it gets a pattern which is

00:47:00: either not able to correctly find the magic pattern to Because lack of data like

00:47:06: of training whatever Or it just takes like the best The best probability the best

00:47:15: probable solution and Gives it back

00:47:20: And this best probable solution, even if it's a matching pattern, might be off

00:47:23: slightly because the probability of it being right was only 67 % or something.

00:47:30: And that's basically how you have to look at it.

00:47:33: It always gives an answer, and it gives always the most probable answer.

00:47:40: But the probability might be too low.

00:47:44: And indexing stuff, pulling out from external sources for a proof.

00:47:49: helps a lot mitigating this because you say, okay, I have this answer, probability

00:47:54: is low and it doesn't match to any documents I have.

00:47:57: There comes the language processing part again.

00:47:59: I can match the answer with other texts and see if it correlates and give an

00:48:06: answer to that, which is a lot higher because the language models are really

00:48:12: good at understanding texts.

00:48:14: And I think you said something that's really key.

00:48:18: So this is one example that I think with letter language models that I think is so

00:48:21: widespread, but it's really important to understand that all of AI, not just larger

00:48:26: language models, are prediction machines.

00:48:30: So all they're doing is, as you're kind of mentioning, attempting or having all of

00:48:36: the probability, there's basically likelihood to be right.

00:48:43: And sometimes it's more confident in its output than it's not.

00:48:47: But the one thing that they weren't explicitly programmed to do is respond no.

00:48:53: They just basically take your prompt and they're like, OK, I'm expected to give you

00:48:56: an output.

00:48:57: Like, I wasn't told that I'm not allowed to answer yes or like, no, I don't know

00:49:03: this.

00:49:04: So they're going to come up with some explanation to your question, regardless

00:49:07: of whether it knows it or not.

00:49:09: But I'll book caveat.

00:49:11: to set it up so that it says I don't know, but then you have to argument what it

00:49:16: actually works with.

00:49:18: Exactly.

00:49:19: You have to put in those.

00:49:20: So, and I think you'll notice this with even OpenAI where you can, if you asked

00:49:25: about even the current events, it's going to say, hey, I don't have that in my

00:49:29: training data set.

00:49:30: If that was an additional module probably, or some additional code that OpenAI had to

00:49:35: write to detect, like when a user asks about a current event, do not respond with

00:49:42: anything, do not make this up, because that's an easier hallucination to control.

00:49:46: But if they didn't have that, then it would totally make stuff up.

00:49:50: But you could do that.

00:49:51: And this is where I think it's key to understand that you can explicitly program

00:49:57: or condition the system to respond and not respond when the output is less confident.

00:50:03: And then you could probably even set benchmarks for, OK, when you have a

00:50:07: confidence or this probability of being right of 60%.

00:50:13: That's cool, give the answer that's better than nothing, but if it's slower than

00:50:16: that, just say I don't know.

00:50:17: So you can actually do that.

00:50:19: think Google even built it into the first versions of Bart.

00:50:23: I'm sure if it's still in Gemini, but they had a probability score, like a confidence

00:50:29: score on the output.

00:50:31: And that's basically the best thing I've seen for getting.

00:50:35: getting rid of hallucinations.

00:50:36: And it's also something you can just dial into your preferences.

00:50:39: For example, for us, I can't have the system hallucinate because it's like

00:50:46: working with support requests.

00:50:49: But I still left like a small margin of, I think, 0 .2.

00:50:56: If one is like the most creative version, I put it to 0 .2 in its creativity.

00:51:03: so that it can, if the user really asks for what might I do, it might take the

00:51:11: context and suggest something.

00:51:14: Usually the button for lock -in is named lock -in.

00:51:17: Something like that, right?

00:51:19: So just to not remove the whole common sense because then it becomes less of a

00:51:25: traditional chatbot and more of an intelligent thing because it just can

00:51:29: think in the context of what it's given.

00:51:32: And that's...

00:51:33: Yeah, that's how I think you can approach a dial in and also make it explainable.

00:51:40: But, and that's a big but.

00:51:43: I mean, it's still earlier on, so these are younger systems.

00:51:48: So we don't know what we don't know.

00:51:49: And I think the way that some of the, you know, even OpenAI is approaching it and

00:51:54: other until you kind of stumble upon a specific scenario where it's like, heck,

00:51:59: you can't anticipate every possible edge case.

00:52:02: So some of it you can address kind of prior to production, but some of it you

00:52:07: will not be able to.

00:52:08: And we're still kind of speaking about hallucination, but.

00:52:10: I think one way to again build that trust with users and then build confidence in

00:52:15: the output that it is more confident in, but also surface the evidence.

00:52:21: I think inline references, especially for models like large language models, that's

00:52:28: basically the best that you can do because there's just so many parameters behind

00:52:31: those models that you can't, it's too much for basically our human brain to fathom,

00:52:35: like the logic that's really truly behind it.

00:52:38: So.

00:52:39: you're kind of looking at the best next thing is like.

00:52:41: even really know what part of the LM is doing or containing what knowledge because

00:52:49: it's just a complete system adjusted for stuff.

00:52:54: I think there is some technology for optimizing separate areas of a language

00:53:05: model to work more efficient.

00:53:08: So you can have a bit of...

00:53:10: insight into what the model is doing when.

00:53:14: But yeah, it's far off from explaining what it does.

00:53:19: And even if you ask it to explain its steps, it might hallucinate in that too.

00:53:24: So yeah, don't, sometimes you even see, like you have a mathematic equation, you

00:53:30: say like, give me the answer and give me the reasoning.

00:53:32: And it gives you the complete reasoning, completely right, and then gives you the

00:53:36: wrong answer.

00:53:38: That's...

00:53:39: you.

00:53:40: sometimes it's the other way around.

00:53:41: So yeah, we're still in the phase where this is something you have to think about

00:53:46: when designing solutions or implementing solutions in your company.

00:53:51: Because yeah, in the end, you really have to think about what this uncertainty means

00:54:03: for your application.

00:54:04: And then you just have to quote unquote just...

00:54:09: have to figure out what this means and how you handle the uncertainties.

00:54:15: But I'm pretty sure, and that's something that's like, I would love to shout it out

00:54:20: to everyone listening, there are ways to handle the uncertainty and to work with

00:54:25: uncertainty.

00:54:25: We do it all the time when we work with other humans.

00:54:28: The outcome of their work is also uncertain in some way, shape, or form.

00:54:33: And we also learned how to deal with that.

00:54:35: So...

00:54:36: There is also a way of dealing with that in an automated way so that it still

00:54:41: improves the quality of your work, the efficiency.

00:54:46: Honestly, I didn't mention that in the news, but the company Klana, they

00:54:53: integrated CHPT into their customer support and they saved the equivalent of

00:55:01: 700 employees.

00:55:03: and two thirds of the support requests were handled.

00:55:09: So it works at scale in 35 languages, one system.

00:55:19: And so there is evidence that you can save a lot of money using AI because what was

00:55:27: it?

00:55:28: Like 700 police you can calculate yourself.

00:55:30: It's like, I think it was...

00:55:32: was really a lot of money they saved now with that.

00:55:36: So.

00:55:37: and I think a quick calculation, if you take 700 times hourly, average hourly

00:55:43: rate, and then times it by the number of hours these employees work in a year, I

00:55:46: mean, that's in like, what, hundreds of millions?

00:55:51: Yeah.

00:55:52: I think it was 70 million or something.

00:55:55: Yeah.

00:55:56: So let me, can I just maybe provide two examples just to round out like the

00:56:00: explainable AI and maybe other approaches.

00:56:03: So there's two, like one example, maybe take something as complex as larger

00:56:07: language models, what we talked about, then like, okay, maybe there's not as many

00:56:10: creative solutions right now because it's just a very complex architecture and it's

00:56:14: just like a very complex model for you to not explain.

00:56:17: So, and I've experienced working on the other spectrum where you have a machine

00:56:21: learning model that you have.

00:56:23: parameters that you kind of architected at a smaller scale that you can actually have

00:56:28: visibility into those parameters.

00:56:30: So I'm talking about, let's say, if you have 10, and they call them parameters or

00:56:36: markers or signals for markers, basically, for the machine learning that indicate or

00:56:47: that provides the reasoning behind.

00:56:50: kind of your data, so looking at your data and looking at patterns.

00:56:55: And those are the things that you can fine tune.

00:56:57: So depending on the weight, the importance of that data piece, or basically you have

00:57:04: visibility into how big of a role that particular marker plays in the output of

00:57:10: the recommendation.

00:57:12: So again, there's different terminologies, again, parameters, signals, markers.

00:57:18: is another word, but basically instead of playing with 90 billion parameters as is

00:57:25: in the case with GPT -4, you have tens.

00:57:30: And so it's much easier for you to actually work with development and your

00:57:34: data scientists and ML engineers to figure out, well, how do you explain the output?

00:57:40: And it's actually easier even like when you implement this in the beginning

00:57:43: because you can troubleshoot it output.

00:57:45: You can better the machine learning because you understand what its output is.

00:57:48: You involve users, they give you that feedback.

00:57:51: You take their feedback and you look at, okay, well, what's missing here?

00:57:56: And then it becomes much more actionable.

00:57:57: So the way we've implemented on these explainable AI is twofold.

00:58:03: One is behind the scenes.

00:58:05: So it helps our development side of the house basically to...

00:58:11: get visibility into the output without every time going behind the scenes into

00:58:15: each prompt or basically query and look at its output, evaluate and analyze it.

00:58:20: We've built like an explainable component that actually surfaces the different

00:58:27: weights, the importance of each signal.

00:58:30: And then that gives us visibility as to like, okay, well, now we understand why

00:58:33: that output was wrong and we have to do X, Y, and Z.

00:58:36: It becomes much more actionable.

00:58:38: But then the other lens to it is...

00:58:41: users.

00:58:42: So when you deploy this, so as you build the solution, that's important kind of

00:58:46: behind the scenes because the solution hasn't been deployed into the users' hands

00:58:49: yet.

00:58:50: But as you deploy this into the solution, because the biggest value add of

00:58:55: explainable AI is the fact that you're building trust with the users.

00:58:59: And so when you say like, hey, this is an AI tool, trust its recommendations, but

00:59:05: you're not giving an explanation for how it came up with it.

00:59:08: Mm.

00:59:08: initially we launched without the explainable AI solution.

00:59:13: Most of the questions that we received were like, how did, why did this result

00:59:18: come up?

00:59:18: And some of it, I think we bridged initially with our change management kind

00:59:22: of practices, because we explained, we did either quite a few road shows to explain

00:59:28: how the machine learning and like these systems really work.

00:59:32: So people had an understanding, but then the questions that came in were quite more

00:59:36: intricate.

00:59:37: It's like, why this?

00:59:38: exact recommendation, why not that?

00:59:41: And so I think that that becomes important as part of your results.

00:59:44: You have to explain this result came up because, you know, this data was

00:59:49: important, this piece of data was important, and this piece of data, and

00:59:52: this is why.

00:59:53: And then you can even, the user can go and compare two results, a recommendation, and

00:59:57: be like, oh, I see.

00:59:59: Now you're empowering the user with the same knowledge that kind of the machine

01:00:03: learning has behind the scenes, but you're converting it into more of a human.

01:00:07: language that they can interpret.

01:00:09: So you're not just saying like this data parameter and this one X, Y, and Z, you do

01:00:15: have to translate it into something that users find helpful in order to deciphering

01:00:19: that feedback.

01:00:20: And that's, again, there's different implications for when you use it kind of

01:00:23: behind the scenes from the development, but as you're building that trust with

01:00:26: users, that front end.

01:00:29: on the development side, it's also really, really useful because having kind of a

01:00:35: traceable and understandable trace of labels, whatever.

01:00:40: I think it might be, I think just thinking about it, like it might be addable into

01:00:45: inference if you say, okay, we are able to classify.

01:00:50: parts of our parameters and then at least have some labeling going on.

01:00:55: Like you can have thousands of labels just like if you enter this parameter group of

01:01:00: weights, then this is most likely to have that label and then you might have some

01:01:06: way to really figure out how the generation is going and which path it's

01:01:11: going.

01:01:12: It might be something.

01:01:13: I'm pretty sure someone's working on that already.

01:01:16: Yeah, you can even cluster feedback, right?

01:01:19: So if you're finding patterns even in the feedback or the types of queries that the

01:01:23: system runs and it just cannot handle well, but it does well on other things,

01:01:28: you could cluster that feedback and address that part of the solution

01:01:32: differently.

01:01:33: So there's just, again, it's empowering not only building trust with the users,

01:01:38: but empowering the team to find better solutions if you know exactly how it's

01:01:42: performing and you're surfacing that feedback to the users.

01:01:46: So one of the other things, and I know there were, we've already taken up a lot

01:01:51: of time talking about kind of the news and we're coming up on our one hour, but one

01:01:58: example that I'd love for people to explore, it isn't healthcare, so it's a

01:02:03: little bit nuanced, but it could give you inspiration for how explainability at

01:02:10: work.

01:02:10: So there's a company called A -Bridge, A -B -R -I -D -G -E.

01:02:16: And their healthcare company, so you know what they currently what they do is they

01:02:21: take a conversation that happens between a patient and provider they record it so

01:02:27: there's no typing and we know that that's like a huge pain point with with

01:02:30: physicians is the amount of time they spent face to face looking patients in the

01:02:34: eyes, so they removed the computer out of that picture and they've basically have

01:02:39: based on the conversation that voice interface converts it into.

01:02:45: a narrative and then it goes a little bit further and it organizes that into a

01:02:53: readable note.

01:02:54: So there's probably some system prompting something like some large language model

01:02:57: combination with with learning involved and they organize it into a basically a

01:03:03: chart which you know physicians that's ultimately what they want to get to they

01:03:08: want to organize that conversation into something that they can ultimately bill to

01:03:12: your insurance agency and so they prepare it.

01:03:15: And basically when you log into the app to see like, okay, well, what was my

01:03:19: appointment all about?

01:03:20: All you see is like this note, like well organized note.

01:03:24: But then you're like, how the heck did this machine come up with all of these and

01:03:29: like nicely organized this?

01:03:30: And what I love for people to explore is exactly how they surface that evidence.

01:03:34: And so I think they do it multifold and it's just amazing.

01:03:38: So anywhere in that note that you can click or hover over, you can click.

01:03:44: and there's a sidebar that opens and it takes you to that part of the

01:03:49: conversation.

01:03:50: So you could either read the transcript or you could play the sound.

01:03:54: So basically, you'll say like, oh, you know, this person was complaining about

01:03:58: something.

01:03:59: You click and you can actually hear the conversation between the provider and

01:04:04: patient where the patient was talking about that complaint.

01:04:07: And so you're literally taking the conversation, the chart and whatever.

01:04:11: So...

01:04:11: you're serving all of that evidence and storing it.

01:04:14: And so that can I really trust that?

01:04:16: Yeah.

01:04:17: And I think you are building trust and you if, if anything that you're still

01:04:21: questioning about, you can.

01:04:22: even over time improve on it, like when you say like, okay, I found several points

01:04:29: where, yeah, it wasn't accurate, even if it took the right piece of audio.

01:04:37: And then you can save that feedback, type in what would have been the better answer,

01:04:43: and then just train on that in the future.

01:04:47: Wow, that's interesting.

01:04:48: and they have that also.

01:04:50: So that's the amazing part.

01:04:51: So anytime that you can hover over that piece of evidence, and it plays and it

01:04:56: gives you a thumbs up, thumbs down, I think you can provide even more feedback

01:04:59: to that piece of the content that you're interested or that piece of evidence that

01:05:03: you were evaluating.

01:05:04: And you can tell back to the machine that, hey, that evidence was inaccurate, and for

01:05:10: what reason?

01:05:11: Or you could just basically give it a thumbs up so it knows that, hey, you did

01:05:14: an awesome job actually surfacing the right evidence.

01:05:18: So yeah, and then you have ability to replay that whole entire conversation too.

01:05:23: So if you wanted to evaluate and you're like, oh, I remember the patient saying

01:05:27: this, I don't see it anywhere in my notes.

01:05:29: And that's the type of feedback you could provide to it.

01:05:32: But ultimately, you have all of these different pieces, transcript, video,

01:05:37: audio, and the actual note, basically that summary, that kind of paint that holistic

01:05:43: picture.

01:05:44: And so over time, it builds trust.

01:05:46: So you know that it's doing that job.

01:05:49: Maybe it's missing one or two points.

01:05:51: I can figure out what those one or two points are.

01:05:55: But it's done 98 % work for me.

01:05:58: So why can't I just use it?

01:05:59: There's just such a big efficiency gains.

01:06:02: But also, if you look beyond that, just the documentation, you're now able to

01:06:07: spend more time face to face between the provider and patient.

01:06:10: So instead of looking at the computer screen, typing up notes, and making sure

01:06:14: you don't miss all of that,

01:06:16: you're delegating that all together to AI.

01:06:20: And again, there's ways for you to build trust and make sure that it does it in an

01:06:24: appropriate way through that explainable AI that we were kind of just chatting

01:06:27: about.

01:06:30: Yeah, and like you said, trust building, also helping in building out the solution

01:06:36: itself beforehand.

01:06:39: Yeah, it's really interesting topic.

01:06:41: I think it's also topic right into all the alignment of AI discussion because yeah,

01:06:50: even governments are not that happy that we working with black boxes we don't

01:06:54: understand fully.

01:06:56: And we will, yeah, I'm not sure if we will have a collect,

01:07:00: solution and complete understanding of everything.

01:07:03: Well, we can, we might get into a position soon where we at least can make educated

01:07:09: guesses.

01:07:11: Yeah, but yeah, like people also always don't know what's up in my head when I

01:07:16: talk, so.

01:07:19: There are some mysteries are not being solved this lifetime, so.

01:07:24: Yeah, and I just wanted to encourage people that when you're implementing these

01:07:29: AI solutions, there's ways that you could actually build these components into your

01:07:34: architecture to explain.

01:07:36: And there's no code tools that you could potentially use.

01:07:40: There's been a company that maybe we'll talk about in the future episodes that

01:07:45: does this through no code.

01:07:47: There's other components that if you have the right skill sets on your team, to be

01:07:51: able to, again,

01:07:52: integrated as part of a custom module within your solution.

01:07:56: In our case, we build our custom ones.

01:08:00: But yeah, there's multiple ways.

01:08:02: And if you want you to brainstorm or for us to elaborate further, please leave a

01:08:07: comment.

01:08:07: And we'll happily go into the tactics and maybe even bring a guest from a technical

01:08:13: perspective.

01:08:14: I don't know, Edgar, if you have experience to speak to that.

01:08:17: But we could even find an expert that does that.

01:08:20: could help speak to the implementation of explainable AI, because I think it's a hot

01:08:25: topic given some of the regulations that are already in Europe, again, and some of

01:08:30: the states, and it's going to become more and more important.

01:08:34: So yeah, if anyone's interested in us kind of diving even a level deeper into that,

01:08:40: leave a comment or send us a note.

01:08:42: know.

01:08:43: Yeah.

01:08:44: Okay.

01:08:44: I think we already are at the end.

01:08:48: I think we have a really extended, extended new section, but Elon always gets

01:08:53: us, right?

01:08:57: No, I'm...

01:08:59: more weeks to come.

01:09:00: I'm sure there's lots more.

01:09:02: just love talking about the new stuff.

01:09:05: It's just always making for interesting talk, I think, talking about what's going

01:09:11: on.

01:09:12: And also have to, like all the news are coming in and I think a lot of people are

01:09:17: like, why should I bother?

01:09:18: So if that's you, that's the channel for you.

01:09:22: And I think it's really a part of being kind of, you know, why we do this is to

01:09:28: make sure that you're kind of up to date and you know what it means.

01:09:31: Yes, you hear our kind of take, but you stay updated and you know exactly how and

01:09:36: what impact that provides to the industry and what it might mean for you.

01:09:41: So that's kind of the whole reason why we curate these news is to make sure like,

01:09:45: oh, well, will our audience benefit from hearing this and like, will this impact?

01:09:51: some of the work that they do.

01:09:52: I think again, some of the, hopefully by bringing that those angles of, you know,

01:09:58: industries that this could be impacting in the short and longer term, you find that

01:10:01: useful.

01:10:02: So yeah, let us know if there's other things that, you know, you're curious

01:10:06: about us covering.

01:10:08: Always, always open to feedback.

01:10:10: Yeah.

01:10:13: Lana, thank you.

01:10:16: Thank you very much for, again, a nice talk and a nice podcast episode.

01:10:24: I think and hope we get better over time and hope that we could provide some value

01:10:31: to some of you.

01:10:32: Yeah, I'm looking forward to next week.

01:10:36: and looking forward to another week of blazing hot AI news.

01:10:44: And yeah, let's see.

01:10:45: I think we are close to getting some guests.

01:10:49: And that's getting me even more excited.

01:10:54: Not sure if we will be already having someone next week, but yeah, we are really

01:10:59: close, as said.

01:11:01: And yeah, I'll...

01:11:03: some other folks, if anyone's listening and wants to be part of our podcast, that

01:11:10: we could spend some time brainstorming ideas and maybe...

01:11:13: hear this and you think, mate, they need me, then just try.

01:11:20: Yeah.

01:11:22: And yeah, with that said, I will say thank you for listening.

01:11:29: Leave a comment, leave a like, leave a subscription as always.

01:11:34: And have a really, really good week and as we hear you next week.

01:11:39: Yeah, thank you.

01:11:41: See ya.

01:11:42: Bye bye.