EP 003 - AI Isn't Perfect and That's Okay, AI for Business

Edgar Dyck (00:00)

Hello and welcome to episode three already of the podcast, of our podcast, the AI Boardroom. Of course, again, with my awesome co -host Svetlana.

Svetlana Makarova (00:13)

Hello, hello. Oh, it's been a busy week. But yeah, no, definitely learning a lot, continuing to learn from businesses and where they are. And yeah, I think.

Edgar Dyck (00:15)

How are you doing?

Yeah, you have a lot going on. To our viewers here, to our audience, audience is the right word. Svetlana is always, I'm always really shocked how much you are able to do in a week. It's really cool. It is, definitely. And the things AI makes you even more productive. That's awesome, right?

Svetlana Makarova (00:46)

my superpower. Being very productive with my time, try to fit as much as I can.

Oh yeah, yeah, I wouldn't be able to do a lot of the things without animations and yeah, for sure. We don't just walk, talk the talk, we walk the walk, I think both of us.

Edgar Dyck (01:08)

We have to, we have to, yeah. Still need some more AI to process the podcast, that's for sure. Okay, like we didn't do news stories that much the last two episodes, but like one big news this week we have to talk about, it's the Gemini release. What's your opinion on that? You tried it out already, right?

Svetlana Makarova (01:15)

Awesome.

Oh yeah, I jumped right on it when I saw a lot of people kind of talking about it and making it seem like, oh, it's huge. There are a few things that I think from quality standpoint and the output, I think it's still similar to what Bard was producing. So I wasn't really impressed by the quality. And I'm just talking about the chatbot is what I've kind of tested. But also I use a lot of the upload function

Edgar Dyck (02:01)

Mm -hmm.

Svetlana Makarova (02:02)

to...

kind of review things. Sometimes I write notes in a separate document that I want to upload and say, like, refer to it. And so I feel like other than imaging, you can't really upload documents. Sometimes I do some work on just kind of PDFs or research articles, right? Just like summarize this. And it's not able to do it for me. And I think the context window is still shorter than, if I'm not mistaken, ChadGPT. So there are a few things that I think they still need to work on.

Edgar Dyck (02:09)

Yeah.

Mm -hmm.

Svetlana Makarova (02:32)

But what I may have mentioned before, what I use BARD for is mostly research anyway. I just wanted to see if I could also use it for other things. But yeah, I think for me, it's still going to be one of those tools that I use for research and then summarizing latest events and other things that are much more current. Because that's the advantage of Google BARD, is that it has access to more newer data that they've

Edgar Dyck (02:58)

Yeah, yeah, yeah.

Yeah, they must have some continuous updating on their side. I'll still try to get some inside information from my brother, but he won't tell me anything.

Svetlana Makarova (03:15)

That's right. You said that your brother kind of is on the inside of Google.

Edgar Dyck (03:16)

Yeah, yeah, yeah, he's actively working on that stuff, so that's interesting. But yeah, like I told you, he's not allowed to tell anyone anything.

Even like, even if I ask, with my most dog -like look on my face.

Svetlana Makarova (03:43)

It's funny, the interesting thing that I wanted to note is that it's interesting to me, at least it was insightful, that Google is taking that Apple approach to secrecy, keeping things secret. And do you think more and more companies are going to approach AI, at least the larger corporations, not just Google? But I'm curious if we just don't know. But I feel like.

Edgar Dyck (04:02)

It depends, I think. Open AI is also pretty secretive. You basically get nothing leaked. Google has a history of stuff being leaked. Every Google phone is half a year in advance, you have the press photos. So that's something. Google plays with it on that part, but I think in the whole AI stuff, it's more crucial. There is also some security stuff and image stuff involved, I guess, because like...

Svetlana Makarova (04:13)

I see.

Edgar Dyck (04:30)

Startup putting stuff out which can help you build a bomb for example to be something to have something which should not work If all may I does it and it fires back it's not as big of an a backlash if Google does it They have a lot more to lose on that one. So I kind of get where they where they are coming from but in the end I

Svetlana Makarova (04:39)

Yeah.

Edgar Dyck (04:59)

I think it depends. Like you ask if it's going to be more secrecy, look at Meta. They basically have no secrecy at all. Like they put everything out open source or like, yeah, yeah. And like Vekaburg is also this week is told that they pretty much are going to make sure that AGI is gonna be open source. And they, like the one thing open source doesn't have like from the get -go is compute.

Svetlana Makarova (05:10)

They're like, welcome.

Edgar Dyck (05:28)

And with Meta on the open source side, compute is not that big of an issue anymore. Which I was really curious to, I am really curious to see what Meta is doing because I don't know if you see the social network like the film about the Facebook founding. And I don't know if Jesse Eisenberg portrayed Zuckerberg properly in that one, but I...

I feel I see that Zuckerberg again, like this guy that has an idea and he knows it is good and he's driving it forward. And I think that Zuckerberg is pretty much a force to reckon with, so we'll see.

Svetlana Makarova (06:12)

One other thing that I'll mention, just kind of, we'll go back to speaking on our topic, but the related news is, you know, AGI. Did you read or did you see kind of the talks about Sam Altman asking to raise $5 to $7 trillion to build AGI?

Edgar Dyck (06:34)

No, I know I didn't I have heard something but it was like in some tech news like just condensed in some some sentences. I know what wasn't it about about chip manufacturing also yeah yeah.

Svetlana Makarova (06:35)

Have you seen that?

Yes, to be able to support that level of... I think we'll get to it when we talk about our topics, but the raise for funding in startup land is like the game is really... Instead of asking for millions, we're now in the trillions.

Edgar Dyck (07:02)

5 to 7 trillion.

How much funding is going on in the valley each year?

Svetlana Makarova (07:19)

I don't know, I can do, maybe I, let me see.

Edgar Dyck (07:21)

Because what I was thinking is like, also the news, I told like, they said like it's three times what ship manufacturing does a year at the moment, like in revenue. Not even a profit or something. It's just a ridiculous number. For example, Germany has a GDP of 4 .8 or something, 4 .6 trillion.

Svetlana Makarova (07:44)

What says US venture funds raised $67 billion?

Edgar Dyck (07:48)

Yeah, basically if you go in the trillions, you have to be backed by either everyone or a country or several countries for that. Like they're only like, even the US would have had a hard time to back something with that much money.

Svetlana Makarova (07:53)

23.

Yeah, I was going to say, find the rich country.

Well, I think it's a little bit humorous to me that some altman is like, I'll build AGI if you give me the money. It's a lot of money you're asking for. Maybe you can make some small tweaks.

Edgar Dyck (08:17)

Yeah.

But was it serious or was it just a joke and people didn't take it the right way? I was like...

Svetlana Makarova (08:35)

I think it was maybe based on an interview, it sounds like, that he did. And I think he's mentioned that then, of course, media grabs onto what sticks, and then they exploded. But just the fact that he's mentioned something around building AGI. And I think it's not new, right? There's conspiracy theories for why Sam Albin was kicked out of OpenEI because of his pursuits towards AGI. But maybe there were

Edgar Dyck (08:52)

Mm

Svetlana Makarova (09:04)

It wasn't the conspiracy theory because I think he does want to start an initiative, but now again, it's coming more and more to the surface through some of these interviews. But yeah, I think there may be some initiative, but now he's like maybe labeling or putting a number to it too.

Edgar Dyck (09:21)

Interesting, interesting. Yeah, like I'm personally a big fan of Sam Altman. I think he's really, he really has a plan and he's really like intrinsically motivated and not by money. Like, and like he says it a lot, but I also believe him that it's not money driven, but it's like the, like, think about it. Like for him, it's like being the guy that made AGI possible. It's like nothing you can buy for any money in the world. So.

Well, maybe for five to seven trillion. For that matter. Talking about AGI, we are not there yet. AGI, well, like, if I imagine AGI, it's more or less like a perfect system, knowing a lot more than we do and doing everything better than we humans do. But we are not there yet, as many of you will know. And that brings us to today's topic.

Svetlana Makarova (09:51)

That's... Yeah.

Edgar Dyck (10:18)

Would you elaborate on that?

Svetlana Makarova (10:19)

Do you want to define AGI? Maybe for some of the audience members that don't understand what that might mean.

Edgar Dyck (10:26)

I think it's yeah, like AGI is artificial general intelligence. And that's like the G is a big one, a really big one. So AI, everyone knows artificial intelligence, but AI systems as we know them, as we sell them, as we can use them are specific systems. They are not, they have not, they cannot do everything basically, at least like from a knowledge work perspective.

But AI systems have some specialties. They may be not as good at math if you look at large language models, but then there are other AI systems that are. And having AGI, like Artificial General Intelligence, means having an AI system. It can have different models, but having a system that's capable of basically doing anything you throw at it. So be it math, be it poetry, be it drawing,

making music, stuff like that. So that's at least my understanding. I have to say though, there is a lot of definitions. There are also basically stages. It's like with autonomous driving, autonomous driving has five stages. There's level one and two is basically what we had was basic assistance system, like braking assistance and stuff like that. And then you have level three, which is partly autonomous, but you have still to be able to take over.

Part four is already completely autonomous, but you still have a wheel in your car. And level five would be completely autonomous. You don't even have a steering wheel in the car anymore. And there are also similar definitions on AGI of that. But basically, now we have specialized systems that can do a lot of things, but not close to everything. AGI would be able to do anything.

Svetlana Makarova (12:18)

Yeah, anything you throw at it, then I think that the systems, another term you may have heard is artificial narrow intelligence. And that's what you're kind of talking about when it comes to specialized systems. So they're really good at one specific task or like a subset of related tasks. But even, you know, you might say like, oh, well, ChadGPT can do a lot. It's truly multimodal. I think it's behind the scenes. It is very much, you know, multiple.

Edgar Dyck (12:27)

Mm -hm. Yeah.

Svetlana Makarova (12:46)

specialized systems architected together to support the chat feature. But it's not a single model that is able to really interpret and operate on their...

Edgar Dyck (12:56)

But I think that's, at least for me, that's fine. The thing is, if I have one system that something goes in and there is one result that's coming out, what's in between? If it's a mixture of expert model like GPT -4 was having different models working together, that's fine. It's for me still would be an AGI system if it's like that.

But it has to have one interface and one output, basically. Cool.

Svetlana Makarova (13:23)

Interesting.

Correct, yeah, yeah, so I'd agree. So I think that that's, yeah, just some definitions for our audience as we kind of dive into it, but what we wanted to talk to you all about today is the fact that AI systems are not perfect and why that's totally okay, and it's actually an advantage to organizations. So you want to kick things off?

Edgar Dyck (13:52)

Yeah, of course.

Yeah, like we have systems in place now that are not general intelligent. That's what we just explained. What does it mean? Like we have different approaches to classify that, but basically it means if you have a task, you can automate with AI today and with large language models like lately, a lot more tasks can be automated than ever possible before.

But the automation won't be perfect. Like maybe you get to 70 % of times that it's working properly. Maybe you get 50, maybe you get 90. You won't get 100. And for a long time, I think. But neither will humans. And that's something...

that always drives me when I build these solutions. I don't need them to be perfect, but I need to be able to work with imperfection. And that's basically the task. And yeah, what's your take on it?

Svetlana Makarova (15:07)

Yeah, I think it helps to understand how AI systems are built. And so if they're really pattern finders in data, so there's basically three key elements to all AI systems. It's data, algorithms, and processing power. And so if you have data, all AI systems are doing, again, is your...

you're trying to find the right algorithm to try to fit as much of the training data as it possibly can to come up with some way of representing that data via, again, like a mathematical equation or some complex algorithm. And so by definition, you're not going to fit every possible data. There's always going to be like those edge cases that's just not ever going to line up on these representations or basically these lines or algorithms.

There's always going to be, again, things that are uncommon of what you've kind of unplanned. And that's OK, because you don't also want your algorithms to overfit your data to not be able to handle these random cases. So just even by definitions of how these AI systems work kind of behind the scenes, you can't build, again, like a single

algorithm that's going to fit your data perfectly, and you don't want it to be. And so I think the key is truly understand that how these systems really find data and that it's even when you build the initial set, you're just basically trying to take your data, try to plot some or let the algorithm do its work to find some representation of that data. It's not going to be perfect. But the thing is, is you're going to have...

Edgar Dyck (16:44)

Mm -hmm.

Yeah.

Svetlana Makarova (16:57)

multiple iterations, you're going to put it in the field for it to improve over time. And so you just need to get it out at a comfortable state for you.

Edgar Dyck (17:08)

So like from a product standpoint, like you have issue or you have a topic you want to solve with AI and you imagine a product, a solution you could come up with to solve that.

you basically have the same which you would have had with like classical systems. But now we have like a non -deterministic system kind of. I'm not sure if that's mathematically correct, but I'm like the output isn't consistent. So if I put the same thing in, it's not getting me the same thing out every time.

And that's down to different factors like you just told, like it's a mathematical equation and we work with probabilities and they can go either way. That said, if you start with any product, AI or not, you have to go through different stages of usage to have it become better. The cool thing with AI though is,

Svetlana Makarova (18:12)

Mm -hmm.

Edgar Dyck (18:18)

If you are going to iron out this stuff now, you still will get improvements for free later down the road if you just update the underlying model. So a lot of stuff will get faster, will get more reliant, quality will be higher. And you can then take all the improvements.

to not then work on the stuff you would have worked on like if you started earlier, but then you can take all the improvements and do really valuable stuff and add value to the product itself. So starting early is always gives you a benefit of time, of course, but in the case of AI solutions, it also gives you the opportunity to make more of what's coming.

That's pretty interesting, at least for me.

Svetlana Makarova (19:19)

And I think with something that some of our listeners may have heard about is that we're running out of training data for these large language models. And what we're kind of talking about is quality data that's already available on the web. So these models do take in or ingest a lot of this data that we as humans just don't have the capacity to produce to continue to feed a data. So at some point, the developers have to.

take a call to say, OK, we've kind of gotten a big chunk of data, and then now we're going to have to put it into the market. So essentially what you're doing is you're capturing, you're creating more data. But instead of it being kind of the training data, it's user generated. So you are basically, when you push your production live, instead of, again, relying and trying to source all of this data and squ

buying data and all of that stuff, you're pushing it out and then you're letting your users engage with the system and it's creating more data that you could capitalize on. And I think that's what you were kind of talking about is that you're now building your own data sets that you own and you don't necessarily just need to use it to improve that particular system, but all of those data points become inputs to other parallel use cases too.

Edgar Dyck (20:30)

Mm -hmm.

Yeah. Yeah, and that's.

Also, maybe something like you maybe may run off pub out of public data at some point, but we still have a lot of private data lying around for your use case in your company. Sometimes like if you're talking about insurance companies, like billions of data points that just aren't used, which because there is no one able to really capture the whole picture because there are too many data points.

Svetlana Makarova (21:05)

Mm -hmm.

Edgar Dyck (21:13)

points, AI is basically able to capture a lot of data points by working how it works. Which then leads to a lot of data which you then start preparing for your new upcoming solution, which is also then ready. And you're like.

There are a lot of benefits to start as early as possible because there is a lot of work involved to get this thing off the ground. And if you've done it, you are even more able to extract value from it later down the road. Values you maybe don't even see right now and today, but will present themselves when the underlying technology gets better.

Svetlana Makarova (22:01)

And I think that kind of speaks to, I like that point of value, and I think it's really important for, I think, our audience to capture. And I think that's a huge differentiator between AI systems and kind of the way of building traditional products, because, and I think we talked a little bit about this prior to kind of recording this. So typical products go through four phases of maturity. And sometimes it can be expanded into even physical products too.

So you have the introduction phase, you have the growth phase when you're kind of getting that traction going and people discover more and then they try to adapt it. And then you go through this maturity phase that starts to taper off and then goes and enters the decline phase. And then typically, I mean, other than pushing more marketing, investing more marketing dollars in order to drive more users to the platform, typically the value of the product, the longer it is in market, it tends to...

become less relevant and it could decline and basically gets tabled. Of course, yeah, so unless you kind of rethink, relaunch with new features, new offerings, the tendency for a traditional product is to go into that decline phase. With AI products, and the way to describe it is you could look this up, it's called the flywheel.

Edgar Dyck (23:01)

If you do not reinvent it, of course.

Svetlana Makarova (23:25)

So instead of it having that kind of line with the decline at the end, it is truly like a cycle that kind of these products enter. So you kind of start with this data that you've generated, which provides some of the recommendations, which give you a better experience than the existing products, which drive usage, that those users generate more data for that platform, which leads to better recommendations to then lead to a better experience.

recruiting more users or having people talking about that more, which again, generates even more data. So it's a continuous cycle of improvement. And so again, this is kind of goes back to what we were just talking about is that you don't want these systems to be perfect when you launch them, because once your product enters that cycle, users help to improve the recommendations of your AI systems. And the sooner you enter or you launch that product into the market, I think the timing is really key, but...

The idea of shooting for perfection, again, is a little bit flawed, and I think that's where a lot of companies get caught up. AI systems are not perfect. We're really not trusting their output. Well, launch it as a small pilot. Put it out in the market to let your users really drive that accuracy.

Edgar Dyck (24:37)

Yeah.

It's like always like get a testing field, get it in some environment where it does not break a lot of things or cannot break things and try it out. It's like we have also with our AI customers, it's the same. Like of course, they all like, especially like we work with enterprise customers, they're even more inclined to really stick with what they have and not to try new stuff.

And for them we also tell them like let's just do it. Let's let us prove it We give it up. We give it to you for free the first month and see see how you How it's how it's working? Because for for us, of course, it's a bit of work, but right now we have to build up trust To to to people that we have a commercially usable product And that's also something

I might have already touched on it in the last episode, but if you build something, you have to really battle test it. So because with AI, and I told my wife like two days ago, so I feel with AI, the jump from the prototype phase to a commercially viable product is as huge as I've ever seen one. And...

that it has different consequences. What do I mean by that? Like building a first chatbot who does like basically 70 % of what you want it to do is a matter of half a day. Getting it to 80%, it's a matter of weeks, getting it to 90 % of you having the product at a place where it works like you want it to, it might be month.

And the sooner you start the process, the sooner you get it out, the sooner you iron out the problems with it, the sooner you get real high value and the sooner it will become a commercially viable product internally or externally, whatever you shooting for. And that's, I think, I will also like in the end have an example ready where we also try the new stuff before it's even ready for launch just to get the stuff out.

Svetlana Makarova (27:03)

I think timing is really key, it sounds like, with launching these systems. And again, there's data behind it. I think it's true. It does take time and effort to actually launch these products, I think, with chatbots. Again, depending on technology and if you're using established models, open source or something like that, that helps expedite that development cycle. So if you don't have to build a proprietary model, it's always great. But even then,

fine tuning it to your use case, to your users, and then the building that trust does take some time. So the sooner you actually launch a pilot or put it in the hands of consumers and users, the more data, again, you're starting to collect and it actually improves better. And I think this is true of all systems, what you've mentioned, and I think not a lot of people realize that because these systems are not perfect, the first improvements that you're going to see are going to be very exponential. So you're going to go...

Edgar Dyck (27:59)

subtle.

Svetlana Makarova (28:00)

Well, I think the subtleness comes back, it comes down later down, down the pike. But the first model, yeah. So I think what I'm trying to say is like, when you build the first use case, you're going to see, like when you fine tune or maybe specify what you want it to do between like just the traditional out of the box model to like fine tuned output, you're going to see 30 % improvement, like with first iteration. And then the second iteration, you're going to see 20. And then, so you're going to reach like,

Edgar Dyck (28:05)

I, okay, okay, yeah, I got you wrong on that one, yeah.

Svetlana Makarova (28:30)

And at some point, that improvement starts to taper off to say, oh, now it's improving at 5%. Are we doing something wrong? No. So I think it's, again, natural for you to see gradual improvements initially. And so you want to get those gradual improvements through usage, through fine -tuning. But how do you do that if you don't involve users? And I think the sooner you can get that ballpark improvements out, you're going to get a solid, good enough product.

to then consider more mature to then launch into the public. But you have to build that.

Edgar Dyck (29:05)

Or company -wise, so whatever you're aiming for.

Svetlana Makarova (29:11)

That could be, yeah, so like an internally facing product. But I think that's key is, you know, and I say this a lot, value delivered is when you put something in front of the customer. So you could sit there and fine tune and customize your model for years. But then when you launch it in front of the users, you could be completely off also. So you want to make sure that the sooner you can actually build and you feel good about where that model is based on your own best guesses and hypotheses.

Edgar Dyck (29:20)

Yeah.

and try.

Mm

Yeah.

Svetlana Makarova (29:41)

Launch a small pilot, get that feedback, fine tune it more. So those users are going to help to take that product to, again, that level of good enough to launch to broader audiences that, again, you're going to just enter that flywheel and improve that product over time. So don't shoot for perfection. It's actually good to get it to that state because users are going to help to take your product to what exactly they need.

Edgar Dyck (30:10)

And also, like for us,

Of course, we have basically started building a complete AI product. Of course, we try everything even remotely new and better directly and try to incorporate it because we have to because the field is so new. But also with every customer I talk to, I don't even need the customer to try it. I just need the feedback on the basic concept. And only that like broad,

product forward like 200 % in like two or three talks. So get stuff out, get stuff to your, like if it's internal, get it to a small test group of really, people really involved in the process that you wanna change and though you wanna improve on. And then yeah, take it from there. That's definitely one of the best things you can do.

Because yeah, we you will not bring it to perfection. You will not even bring it to To a level where you feel comfortable with but One thing I always think about it when I'm going to talk about this is like our humans perfect if someone's starting a new process in your company Will it be perfect? I think not so?

That's something to keep in mind. Also the benchmarks you see on AI systems. It's like always, oh they are only 87 percent. I feel safe as a human and then you look at the human score and see 83.

Svetlana Makarova (31:55)

That's a valid point. I think we forget that, like the quote by Voltaire, it's perfect as the enemy of good. And we don't recognize, again, we judge, we have these high standards for AI systems thinking that, well, their intelligence, they need to be 100%. But the thing is, is like, we've actually architected or we kind of frame them up in terms of like how humans learn.

Also, so like you have to kind of benchmark its performance also according to humans too. So how well is it doing in comparison to those tasks that humans do? And sometimes again, like when you look at statistics, some of these systems perform subpar to humans, but I think it's just a matter of time until they improve. Again, once they enter that kind of feedback cycle. But then there are other...

Edgar Dyck (32:41)

Yeah, of course.

Even getting close is already an achievement, which is like humongous, so yeah.

Svetlana Makarova (32:52)

Yeah, and some of them are already superseding them, right? So superseding us on certain tests.

Edgar Dyck (32:55)

Yeah, and it's becoming more every week to be completely honest, right? There are less and less fields, like the fields becoming more and more ruled by AI -driven systems. Take biology to find new proteins. Go into medicine to have pattern recognition on MRI scans and stuff like that. So.

Yeah, the field's just growing and it's not going to stop anytime soon. One thing, talking about humans, maybe also relevant. If you start early, you get your people skilled early.

and then your people will be able to extract even more when the time brings even better models, even better AI systems. For me, I do it...

from like one and a half years basically, that's what I really dove into into the topic. And now I have so much knowledge that basically every AI system I try to implement works as I expected because I just have learned how the system behaves.

given some circumstances or context. And that's something...

It's not because I'm vastly talented or something, but it's just I started early and I put a lot of time and effort in. And that's what you have to aim for with your staff too. Like if you want to build AI into your company, into your corporation, yeah, go ahead and train the people and get them skilled. And they only will be skilled if they work with it. Like being somewhere in some, like watching YouTube videos all day long will not bring you anything.

because yeah you have to you have to get into you have to start building otherwise you won't be of any use.

Svetlana Makarova (35:00)

It's learning by doing, right? So it's practice makes perfect. Again, we're talking a lot about perfection, but it's truly, I think the more, the more you do something, it's like you're building that muscle. And what I'm finding also by working with, um, leaders is that the sooner you get your either team started or you yourself get started with using AI, it's going to spark a lot of ideas. So if you start delegating it tasks and.

Edgar Dyck (35:06)

Yeah.

Yeah, yeah.

Svetlana Makarova (35:28)

different things to do. You're just like, oh, I wonder if it can help me with this. And you stumble upon a different use case in discussions. You're like, oh, I wonder if we could upload this into ChadGPT and have it do something else. And so yeah, I find that a lot that it sparks not only creativity from content generation, but the fact that you're using it and then you're delegating, let's say,

Edgar Dyck (35:32)

Mm -hmm. Right.

Yeah.

Svetlana Makarova (35:54)

one use case of it a day, just whatever task that you're doing a day, just pass it on to ChadGPT and see how it can handle it. But next time, I think you stumble again in discussions, you're like, oh, I wonder if I could do X. And you're starting to really push ChadGPT or these other systems to the boundaries. Oh, I wonder if, and that's exactly, the sooner you get to use it, the more ideas it's going to spark, not just for you, but your teams as well. So yeah.

Edgar Dyck (36:02)

Yeah.

Yeah, I think extracting value from this depends mostly on having understood the pattern of it. So you have to... There is a new paradigm, a new way of thinking about solutions, including AI, because you have a system that does not behave...

exactly the same every time and you have to learn to deal with it. That means you have to think about solutions vastly different than before. And that's the interesting part for me. I love that stuff, honestly. For me, it's really, oftentimes I have to disconnect myself from all my previous IT knowledge.

and think about how would you teach a human doing this? So what would you say, how would you formulate it? And then go back to the technical side because to be honest, I've never coded so little to get so much results and that's a good thing. But you really have to find the right language literally to get the results you want and the consistency you want. And you have to,

to also see what are you do, what are you going to do with the results. Do you?

give them to another AI maybe to check them or not, stuff like that. Also, if you type it into the strategy BT, do you type it in and take the answer? Or do you try to make a discussion from the get -go? Stuff like that. And learn the paradigms, get a feeling for that. I would love to give an example from something completely different.

I started to play football just to get some movement into my life. I like football, I always played it, but I never played it on the real big pitch with grown -up men. So I go in there, I play, and I basically have no idea what to do because I don't have the underlying... I don't understand the underlying...

concept by heart. I know the tactics from a theoretical standpoint, but I never stood on the pitch before. And now I have to always orient myself and always try to not be a waste of air on the pitch. And it only will get better. Now it gets better, like, of course, after time, because I understand some basic concepts and basic principles.

And that's where it starts to become fun because until then it's not fun, it's just like feeling lost a lot of

the time. And if you don't want to feel lost in AI, learn the rules, learn the concepts, learn the paradigms and get a feeling for it.

Svetlana Makarova (39:21)

You

Yeah, and I think you're spot on. I think initially it will be uncomfortable. Although I think that a lot of the user interfaces right now are built very user friendly. But how do you interact with it? What do you ask for it to do? I would say just talk to it like you're talking to a colleague or someone. Or maybe a virtual assistant or your executive assistant. Just, hey, I need to do x.

Pass it like some context and then ask for it to to do that task for you and just see what it comes back with Even as someone I Spoken with last time I was just like even me I was just like I don't know why I'm so silly I haven't thought about myself but you know one of the things that it takes me a long time for even my LinkedIn Posts is actually generating the images because I'm like I'll write the content within you know minutes But then I sit there for like half an hour. I'm like, okay, well

Edgar Dyck (40:18)

Mm -hmm.

Svetlana Makarova (40:23)

What kind of image should I be associating this with so people can really connect to the concept? Because why don't you just copy your posts and ask Dali to brainstorm what you should be posting? I was like, that's amazing. But there's just so many things. Even myself, I didn't really think of that. But I was like, that's brilliant. Why haven't I thought of that? So why do I have to do all of this thinking?

Edgar Dyck (40:39)

Yeah.

Yeah, you always, I don't know the English wording for it, but you have your vision covered left and right. Like a tunnel vision, yeah. And...

Svetlana Makarova (40:59)

like tunnel vision.

Edgar Dyck (41:03)

tunnel vision, oftentimes, makes you not see the obvious stuff that's left and right of you. Imagine how it would be with the Vision Pro where you have an actual tunnel. No, but one thing I wanted to add there is, yeah, we have to, I forget what I wanted to say. Damn tunnel.

Svetlana Makarova (41:17)

Oh my goodness. Yeah, I think we need to.

Edgar Dyck (41:33)

Hahaha!

Svetlana Makarova (41:33)

Yeah, took you off of the topic. No, but I think that you're right. I think sometimes even myself, like someone who uses AI tools, like I sometimes even find myself not thinking outside of the box and using it, but like through conversations that you have of others using it, right? So like the more people use it, even within the organization, they're going to share ideas. I think that's, you're going to have a network effect that way as well. So me talking to you like, Hey, I've used X, Y, and Z and then...

Edgar Dyck (41:59)

Yeah, definitely. Yeah, yeah, of course.

Svetlana Makarova (42:02)

Again, there's a lot of people who start benefiting. It's not just yourself. So I tend to just operate on my own. So until I had those conversations with someone else who's, again, a very heavy user of some of these systems, they're like, oh, don't you X, Y, and Z. Like, that's brilliant. I didn't think of that. That's awesome. So just think about it in your organization.

Edgar Dyck (42:06)

Yeah, it's crucial.

Yep.

I have it honestly all the time. Even talking to potential customers, not even talking to other AI experts, but talking about AI with people who might use it, sparks some new things because they just ask questions about stuff like.

I should have thought about that. You're right. I'll get you to I'll get back to you on that. So no, I remember what I wanted to say. It's also in the end for me, always a thing of also handling the frustration because a lot of times you have a discussion through different things, your context grows and you just get stuck. Sometimes you just tell GPT like this code doesn't work because that is wrong and he gives you back the

Svetlana Makarova (42:38)

you

Edgar Dyck (43:06)

exact same code and it's frustrating at times. But yeah, that's just, yeah, you have to have the ability to not let yourself down by that and it's not the AI per se that it's just not usable, but you might have to have a...

you might have to have another take on another approach on what you actually want to do. So, yeah, I think like always, like the same rules apply here, like they apply to life, you have to have some tolerance to frustration to get stuff to a point where it's really good and it's really helpful. So don't give up too early on AI because of course it is not perfect.

But a lot of it is a matter of how you use it and how you understand it.

Svetlana Makarova (44:05)

Yeah, and I think I know we talk about, you know, Gen .ai here as well. But I think a lot of this kind of feedback also applies to other proprietary systems. So I do have experience, and I think you do too, about building kind of your own models, proprietary models that are unique to your use case. So again, some of the same things that we talk about, like even if you're building it for your customers and other things.

the more they use it, the more feedback that they provide, the better your systems really become. And so I think that's really crucial to get out of that building mode the sooner, the better. And again, let your users kind of think about, because again, you're going to get stuck in ideating all of these different scenarios that may not be relevant to the user. And so the sooner you launch this with users,

and start to get that feedback, they'll tell you what features or what things matter to them so that you know exactly what to spend. Oh, yeah. And I think speaking of which, I think I want you to talk about some tactics, maybe just high level for how do these systems really learn? So we say a lot like, OK, well, just launch it. What does that mean? So how do these systems learn? Yeah. So what kind of data?

Edgar Dyck (45:03)

Yeah. Or you see it in the data. Yeah. Yeah.

Mm -hmm.

Yeah, how do you improve on it?

Svetlana Makarova (45:29)

Is it collecting and how is it learning?

Edgar Dyck (45:32)

Yeah, it's also something in your solution design, you have to keep in mind, like if you want the system to improve over time, you have to have mechanisms to first record history in a proper manner with proper context recorded as well. And you then have to expand your solution to make use of that data to improve the overall solution, the overalls.

the overall system. This is, yeah.

Svetlana Makarova (46:03)

And then so you would just bring in experts to help you. But ultimately, I think it's when you train the models or these models are trained, right? So you get either out of the box or you build your proprietary system. So you have to build in pipelines of data that's going to be coming in new as feedback that will again feed out those learnings. So everything as this part of that flywheel needs to be connected. So.

getting those users and engaging data when they generate data through usage by giving you feedback through the system. So it could be explicit or implicit, explicit meaning that they actually have to press something like explicitly tell you this is wrong, or they implicitly tell you this through usage. So they tend to see a result and then like forget it and they close the window. That's probably like an indicator that they were upset with your output. That's probably it was wrong. And they've kind of gone to to use a different system. But.

Edgar Dyck (46:38)

Yeah.

Yeah.

Svetlana Makarova (46:58)

Again, there's different ways for these systems to learn and really improve these things. And it's important, I think, as you mentioned, to build those pipelines to make sure that that data that's newly generated by the users is fed back to that engine to really improve.

Edgar Dyck (47:15)

I note this down for another episode, I think. It might be something to dive into a bit deeper. Yeah, just how to design systems for them to improve over time.

Svetlana Makarova (47:21)

the feedback.

Yeah, I think it's important to have, again, to tap into those network and learning effects of these AI systems and what, again, we've kind of spoken about it, but the network effect of AI is having more users basically using the system and then the learning effects is more people using it. The more data you generate, the better, or it has more access to data points to learn from. And so again,

Edgar Dyck (47:45)

Yeah.

Yep.

Svetlana Makarova (47:57)

There's a name and again, logic to this madness. But the sooner you tap into more people and tap into those learnings, the better you're off. So again, this applies to either out of the box systems or proprietary models that you end up building on your own.

Edgar Dyck (48:17)

Yeah, definitely. I would love just to give one practical example with kind of a shameless plug from my side. The AI system we work on should improve the support system or like the support processes for big companies.

And one thing I, like it's crucial for us is having the proper documents as a basis. And like from earlier startup experience, I know if you have people, if you want to sell a product and people have to put work in it for it to even work.

not a good thing for your product. Because people do not want extra work when they buy stuff, they want to be helped. And so I went ahead and tried to think about things we can do with current AI systems. And

Svetlana Makarova (49:14)

Mm -hmm.

Edgar Dyck (49:15)

one idea which I came up with was to having someone putting up a screen share just showing you what to do and how to handle a situation and then generate documents from that.

That stuff of course needs some kind of image recognition, but not like OCR stuff as we did earlier, but really like understanding what's happening. And right now you can use GPT -4 Vision for example.

Svetlana Makarova (49:33)

Mm -hmm.

Edgar Dyck (49:45)

or the new Gemini is also capable and give it a series of images and say, hey, this is a video, what's happening here? So it can recognize that series of images and it can work with it. But that whole stuff is still really early. And he sometimes screws up a lot. And even the model itself is still in preview, so it's not production ready for production loads.

Svetlana Makarova (50:04)

Mm -hmm.

Edgar Dyck (50:13)

which makes it like per definition best a better product. But I still already thought about this, I already thought about all

the...

ins and outs, how to get the screen share, how to save the data, how to improve on the experience. How can I maybe show some boxes in the screen share itself for people not to only show me, but guide them with video. So even think like one step ahead. And that's basically where I have myself experienced this to jump early on an op.

on a preview on an option to get all the stuff figured out and then when they update the model and it's production ready I just switch the model and I'm ready to go and then I can see what's now possible on up on that and I do not lose another three months of how long it will ever take to then start and then miss already the next updates because it's like the whole AI stuff so

There is so much going on and so much happening and the updates are flying in daily. If you lose track, you lose a lot of track. So because it's just that fast.

Svetlana Makarova (51:26)

Oh yeah.

And I think that there's some things that you could do to protect that. I think maybe a topic for another episode too. But there's a way to protect your architecture and make sure that it's not throwaway work altogether, but having that flexibility to accommodate new models. As you mentioned, you should not architect or hard code anything in the age of AI. Like, do not...

you know, commit yourself 100 % to a single product that I think, again, maybe something for another episode, but you want to make sure that it's flexible. So in some of the products that we've built, we've swapped models already. And I actually encourage the teams to have a research for basically a parallel path to our kind of production. Vue is like the POCs are about testing these models. And so like,

Edgar Dyck (52:01)

stay open.

Mm -hmm.

Svetlana Makarova (52:30)

plug them into our architecture and see if they're delivering better quality output for our specific use case. And I work in a highly regulated industry, so where quality and accuracy is really highly valued. So we want to make sure that we're kind of tapping into, so we don't design, we don't go through like three or six months even of building something around one model. And then someone judges us to say like, oh, did you know that there's something else? So there's a constant, again, like a backlog of things that our team is...

testing and validating to make sure that, hey, if we have access to some of these systems, that we're trying them, but also gives us flexibility. We build our architecture in a way that it's flexible to say, like, okay, well, that works. Why don't we just, like, bring that in? Like, let's swap them. So all of the other configurations and everything that you've built around the other models still are applicable. You might have to, like, do some tweaking, but it's not starting from scratch. So I think in the age of AI, I think what you've mentioned is you have to be flexible.

Because the minute that you think you've got the solution, next week something better comes off. And you're like, oh, now I have to accommodate. So.

Edgar Dyck (53:33)

Yeah, definitely. And that's why you also have to stay up to date. I even thought about building myself an AI system which just tries to collect me the information that I need from the whole AI research stuff because it's huge. What's happening?

Svetlana Makarova (53:51)

Can I have a copy of that once it's done? Oh yeah, I think we're very much needed.

Edgar Dyck (53:52)

If I have time to set it up, I'll let you know. I also think I might have to do some chorus development stuff or so where I get people up and running with that stuff and that thinking. So that we get... And then I get my own network of soldiers fighting the AI battle with me. Yeah. Okay.

Is there something to add? Do you have something in mind?

Because...

Svetlana Makarova (54:25)

I think one thing that we need to kind of just touch on, and I think the whole kind of idea of the discussion is not to instill any FOMO, like fear of missing out, but just kind of to inspire action. So once you've built or you've decided to pursue an AI path for one of your products, I think, again, the biggest detriment you can create is to shoot for perfection. But...

Again, the value delivered is when you put something in front of the customer and that's done for multiple reasons because you want to deliver value and start measuring it. But two, there's a huge advantage to putting an AI system in the market soon because it is going to get better and better through usage. And that also ensures that you're kind of not sitting designing and developing training and fine tuning and overfitting your model behind closed doors. So...

the sooner you can get that product out. So it's not uncommon to have these pilots be launched within three, six months. If you go beyond that, please find ways to get your product out in the market, like even with the small pilots of users, but make sure that you kind of launch it to start capturing some of that value and then to ensure that you've built the thing that truly your users find valuable. I think I hear a lot also is like, oh yeah, there's...

Edgar Dyck (55:32)

Yeah.

Svetlana Makarova (55:44)

We need to launch this thing, the sooner the better, there's an advantage to getting it to being first in launching this thing. But I think that's not what I'm saying. You want to make sure that you built the right solution. Use that time to make sure that you've crafted, because you could also build more of the wrong thing. So you could have invested all of this time and built something and invested all of this effort building the wrong type of model that produces the wrong type of output.

Edgar Dyck (55:56)

Yeah. And use the time.

Svetlana Makarova (56:14)

So again, there's multiple reasons for what I think the objective of this podcast is, is to inspire action so that you're capturing value from the users, you're getting that feedback, you're building more of the right thing, but also launch it to get kind of the benefit of that, the learning and network effect that these A models have.

Edgar Dyck (56:34)

Yeah. Yeah, like.

I'm the leading authority in doing stuff for nine months and then trying to sell it and it's not gonna work. I can tell you that. And this time I flipped the script completely and I built some prototypes to know that at least I can do it and do not promise stuff I cannot do. And still then I take what I can do, but put 10 % on top and then present

and see how people react to it. And then I came up with a solution, with a solution I promised. And this solution, that's the beauty of it. If you talk to it, if you get it out in the wild with a prototype, whatever, like all the stuff we told about in this episode.

The beauty of it is the system you then try to develop will be a system that people are asking for, actually. You always have to have a fine sense of what is an individual need and what is a general process need. You have to be careful with that. But also, even that, it's only coming with us all having tried stuff out, talked about stuff, and most importantly, talk with the people that are actually

going to use it and not with their executives only because there is a lot of disconnect from operations to management.

Svetlana Makarova (58:04)

Yeah, and I think one other thing that we say a lot. So if you build a model that is the right type of thing, and let's say for unique use cases where something like an automation of specific tasks doesn't really exist, and it works 60 % of the time, guess what? 60 is better than nothing. So either you invest your time and you identify its gaps. So I'm not saying like it's going to automate 60 % of your tasks, not 100.

Edgar Dyck (58:23)

Yeah.

Svetlana Makarova (58:32)

But again, I'll take that 60 % because that's going to provide 60 % of efficiency to my teams than over nothing. So again, don't shoot for perfection. Get it out there. And again, let the systems improve. And again, there's use cases of using automation computer vision in manufacturing. You hear that a lot, predictive maintenance and other things. But ultimately, the way those kind of...

Edgar Dyck (58:32)

Yep.

So.

Svetlana Makarova (58:58)

systems learn is through feedback. It just knows that, oh, hey, well, you missed, you know, we were producing these cups and like, and then it connects that feedback to say like, okay, well, what happened there? So they take an image that they were processing, let's say of like a manufacturing, like a cup, and then it takes that feedback. It marries it. Okay, next time I know that that type of defect is not acceptable by those humans who've reported me. So I'll do better next time.

Edgar Dyck (59:03)

That one.

those humans. I'll get rid of them when I can.

Svetlana Makarova (59:27)

So, and then again.

That's it.

Don't instill fear in our listeners. But that's kind of the idea, is to get that data and feedback. And I think we're also kind of witness some of it. Like you can't anticipate every piece of feedback too when you build these systems behind closed doors, which is why I think when Bard, was it Tay? System Tay that they, or Google launched that started...

Edgar Dyck (59:34)

Ha ha ha.

I think something like that, yeah.

Svetlana Makarova (59:58)

Yeah, so they didn't really test their system and didn't anticipate a lot of the feedback or usage types or what people would be asking the system. So yeah, it just kind of went into this... It was just a really bad media effect that they've experienced because it started to produce some discriminatory... sexist and discriminatory feedback back to the users. Same thing with, I think, you

Edgar Dyck (01:00:24)

Mm -hmm.

Svetlana Makarova (01:00:27)

Some of the other product launches really failed because they didn't really get a lot of that usage in. And I anticipate probably they've kind of sat with a group of folks, brainstormed some use cases. I think they've gotten it to a good enough state, but they didn't really invite a volume of users, actual users, anticipating some of the use cases that they stumbled upon when they launched it. So I think that pilot phase is really important to get.

Edgar Dyck (01:00:52)

Yeah.

Svetlana Makarova (01:00:56)

as much of that type of unanticipated user behavior also to iron out before you launch. Because, yeah, once you call it a production level systems, then you do have all kinds of other expectations from users. But that's why I feel like that product didn't launch, didn't go as well, because they didn't probably do their due diligence of going through a solid pilot case to really refine their system before it was considered...

production level ready.

Edgar Dyck (01:01:28)

Should have asked Asunu when he asked me. Svetlana then shakes again and then you're good to go basically. Okay, was again a lovely hour with you here. Really, really enjoyed talking about that stuff.

Svetlana Makarova (01:01:31)

When in doubt, send it to Edgar. He'll test it for you.

I'm the QA.

always.

Edgar Dyck (01:01:55)

Thanks.

everyone for listening to this episode. Yeah, I really think we get into some vibe here and some flow. So if you are, if you audience are, I think the same, so excuse me. If you think the same, then yeah, give us a like, subscribe to the channel if you watch on YouTube.

subscribe to the podcast. I don't even know how it works with podcasts. Do you subscribe to them too? Do you save them? Yeah. Yeah, subscribe. And yeah, with that said, next week, new round, we'll still have to decide the topic. But if there is anything plenty full, then topics about AI.

Svetlana Makarova (01:02:29)

I think so, yeah. Like favorite them or something.

Oh yeah, we have a backlog of ideas, but we want to make sure that we again speak. Don't just talk the talk, we walk the walk. So we want to be very much user centered. So if there's any topic that you'd be curious about for us to cover in the next episode, all ears. So.

Edgar Dyck (01:03:07)

Yeah, also if you're on YouTube, put it down in the comments, writers, theaiboardroom .ai, not .com like I said earlier, theaiboardroom .ai. There you can find some information, you can find also contact. Yeah, just hit us with all you have to ask and we'll see what we can do for you. And we'll see what we incorporate in the next episodes.

Svetlana Makarova (01:03:35)

Perfect. Well, it was a pleasure. And until next week.

Edgar Dyck (01:03:39)

Until next week, bye bye.

Svetlana Makarova (01:03:41)

Bye.