EP 001 - ChatGPT and Beyond: Strategies for Effective AI Implementation

Edgar (00:00)

Hello together and welcome to our new podcast, the AI Boardroom. Svetlana, I would love to hear first from you and tell the audience who you are.

Svetlana Makarova (00:07)

Hello.

Hi, I'm Svetlana Makarova. I am a, I guess an AI strategist and implementation advisor. I love AI and I help businesses adapt AI, understand what value it brings and how to actually bring it into their business. Yeah, and I'm an AI geek, so I love talking about these topics and I think we could share some of both of our knowledge in the upcoming episodes, but really bringing that practical angle.

to business leaders. That's what I enjoy having conversations about. How about you?

Edgar (00:48)

Yeah, and that's how we came up with this whole concept here. So my name is Edgar Dück. I'm from Germany. Like, I still find it impressive that we, like, you sit in watching this. It's like eight hours difference or so. And we can record a podcast. Like, I love technology. And our combined love of technology brought us here to talk a bit about AI, which also for me, I'm a developer for the last,

Svetlana Makarova (01:08)

Yeah.

Edgar (01:18)

14 years maybe, that's nearly, and also did a lot of AI development, have active AI projects like you too. And yeah, we'll love to talk about all that we collected and all that we hear. Am I right? Like a lot of people hear AI, it's everywhere, but the understanding is lacking in a lot of things.

Svetlana Makarova (01:46)

There's definitely a lot of buzz around AI people. And I say this a lot. When it's unfamiliar, it's uncomfortable. So people don't, they hear the buzz and they're like, OK, well, it works. I don't know what's different about it. But.

Let me just table it for now and come back to it when it's more familiar. So I feel like people are talking about Chad GPT. I think we just spoke about it prior to this. I did a quick keyword search on Google. So how many people even search AI or Chad GPT? And you'd be surprised with between 100 to a million searches across, I want to say, five to six different categories of different spellings.

of ChadGPT, people are really curious about what it is. They don't go immediately and go use the tool and just jump in. They want to learn what it is first. But there's not a lot of discussions on how it works, what it can do for your business. There's just a lot of just diving in deep, go and use it. But then, again, if people don't even understand what the heck to use it for, it becomes quite difficult.

Edgar (02:30)

Mm-hmm.

Yeah.

Yeah, so this brings us to the first episode. You might have seen it already in the title, but yeah, we wanted to start to talk about Chatchabit, because that's basically how it all went down, like I think in November 2022, when they launched it, and launching it even themselves, not knowing what to expect and what will happen. And I think the AI wave that has since then,

Yeah, become like a role over the world, if you have to say. There was already an open source community before JGPT, but I don't think open source was even close to being that big. Like just from like a research point of view, it's really interesting what happened in the last.

like 12, 14 months or so, let it be 15. It's like the rate of development, I've not seen anything like it.

Svetlana Makarova (03:47)

Mm-hmm.

And if you think about it also, GPT models, transformer models, have been around since 2017, until they truly kind of put a front end, I want to say like a more user-friendly, Chadbot style of communicating with these models. People didn't really make much of it. We in our organization evaluated GPT for one of our AI models.

Edgar (04:01)

Yeah, 1617, yeah.

Svetlana Makarova (04:21)

specifically focused on natural language processing, but it was just not suited well for our industry. So we ended up kind of passing it. But nonetheless, I think before 2022, unless you were technically savvy to really evaluate those models for your use cases, you couldn't really use it. And then, you know, I guess Chad GPT tried something or I'm sorry, OpenAI tried something different and they just kind of blew up.

Edgar (04:41)

Yeah.

Svetlana Makarova (04:48)

because they were like, okay, well, it's been around for five years. Let's just test it. No one's going to come. Like, we're just going to put it out and let's just see how it goes. I think it caught them by surprise.

Edgar (04:53)

Yeah.

Yeah, honestly, I already applied for GPT-3, I think it was, and it came like late 2020, and I got also access, and I was doing a startup during COVID at that time, and I really looked forward to use it, but I didn't get it, honestly. I wasn't that...

I didn't get what the... Because also it hadn't had the chat component. It also was text, it only was text completion. So you could type something in and it would complete the text. So it's not that different from chatting, but there's like a small, there's a small difference in how you use it and how do you interact with it. And that's basically what put me a bit off. And of course it was not even close to being as good as then the release, GPD,

But yeah, we like the term GPT is like around you, you already said it like transformer models, like.

I think it's kind of funny that they have chat GPT as a like trademark name because chat GPT is not the model itself. It's like it's an app. It's an interface. Like I said, a front end. And that's also one of the things we want to talk about today is like what is this chat GPT app even about?

Svetlana Makarova (06:12)

Mm-hmm.

Edgar (06:22)

and it got a little bit more buzz now with the launch of the GPT Store the last weeks I think like two weeks ago or so and we wanted to start off a bit with

with a small talk section where we try to advocate, like pro or against the usage of this set store just to introduce stuff. So I would love.

Svetlana Makarova (06:55)

And I think just to add to that, I would say specifically focused on business. I think personal use are uses that are different, managing your tasks. But I think we want to make sure that business leaders are equipped with knowledge as to what does it mean to use ChadGBT? What impact does it create to your business if you want to adapt the use of ChadGBT across your organization? How could you do it?

Edgar (07:04)

Yeah, of course.

Svetlana Makarova (07:23)

So I think a small preview towards the end, we will touch on data privacy and all of these other things that you should be considering if you were to kind of adapt or kind of roll it out in your organization.

Edgar (07:46)

So, yeah, gbt store.

We tried to, like one is four, like tries to find arguments on the positive side. The other will try to argument like more on the negative side. We at least in our mind thought would be a funny interaction. Let's see how it works out. Talking about the GPT store, like I think it's fair to say that OpenAI tries for chat GPT for the app to become the next iPhone, like.

Svetlana Makarova (08:06)

Yeah.

Edgar (08:19)

environment. And GPTs are basically the apps. So they can, they are tuned. Oh, I have to be careful with the wording. But there you have kind of neat little inserts, apps, customized, pre-customized prompts for talking to ChaiGPT so that ChaiGPT can

Svetlana Makarova (08:36)

customized.

Edgar (08:48)

On the one side, adapt persona, and on the other side, access external data. And now I ask you Svetlana, is it the next iPhone moment?

Svetlana Makarova (09:02)

Oh, absolutely not. I have so many different thoughts. And I think you could take a history lesson with iPhone, when they launched their app store. And the key to their success was having a platform. And it was all of their, not everything was perfect, but they did have their T's crossed and I's dotted when they launched it. There was a clear path, but.

for people to create apps and like monetize there were some hiccups I would say with the permissions and then you know kids buying thousands of apps from their parents phone without actually, you know getting their parents permission, but There weren't as many Kind of concerns with adoption and I think with We we saw another phase of this with Amazon was trying to launch their whole kind of you know store of their

own kind of devices, right? And they thought that the adoption of it is going to skyrocket. The problem was is that they had a small app store, but there weren't enough apps on their phone. And so people would just at that point, iPhone had such a lead time towards such a higher lead time, people were just so used to that experience that it didn't make sense to switch. And so when it comes to chat GPT, you have to make sure that there is enough value.

for on this platform for people to want to like switch from their old or kind of use that they're comfortable with. So when you're saying you know is this the new iPhone people are still trying to figure out what the heck chat gpt is and how to use it and then if you're really looking to monetize or like really build out this platform you're going to get a lot of usage. So the people who are more technically savvy more adapt to using chat gpt's are going to be probably your highest users of these systems.

as you even talked about, how do you create the GPT? What is it and how is it different from Chad GPT? And I think it makes sense maybe to even step back just a second and say, how do you interact with Chad GPT? How do you make sure that it produces quality content? So basically you prompt the system. So that's one way, I think that there's three ways that I could maybe summarize. One way is that you provide context via prompt. So you would just say like,

Edgar (11:14)

Mm-hmm.

Svetlana Makarova (11:26)

I assume that you or this person provided some context as to what you wanted to do. Consider maybe give it examples, and then that's how you ensure you get quality feedback. So you could do that every time. Every time that you use that system, give it a paragraph of a prompt with all of that context, and then you could basically get the output you want. You can go into your own Chad GPT in the system prompting. There's a configuration section where you could configure it.

Edgar (11:41)

Mm-hmm.

Svetlana Makarova (11:53)

and say, like, every time that you're running my prompts, consider this. Consider this context. This is the person that I am. And then there's a third way that I think it requires some integration. But let's say, chat-gpt, what GPTs actually do, they automate that. So instead of you prompting the system, they're like, I'll take the heavy lift for you. I'll do that system setup that you need. All you have to do is just assume that it knows what you're trying to do. And so when you look.

Edgar (11:57)

Mm-hmm.

Svetlana Makarova (12:21)

open up that chat GPT or that GPT, just ask it directly. Like, what do you want to do? And I'll give you all of that prompt back. But that's kind of the whole value of these systems is, if you're using, I don't know, some character specific GPT, it's going to act like that character. And you don't have to provide a context as to like act like this character and all of the...

different personalities that it must exhibit. So these GPTs all do that heavy lifting for you. But I don't know where I was going with them. Totally lost.

Edgar (13:01)

But let's focus on some upsides. So what do you think are some upsides? Maybe I come along and disagree.

So.

Svetlana Makarova (13:13)

Yeah, so I think with what it enables you to do, so a lot of the folks who have a lot of experience with GPTs, and so they kind of have started using it in the last year and really becoming prompt engineers who are really good at it, right? So those are gonna be the first people who develop these GPTs. So they have kind of that experience.

And there is something in it for them. They're not just doing it for the common good. Some of them are. But there is a revenue model behind it. So sometimes they're creating all of these different use cases based on their experience. So you could assume that they've kind of built that quality, like QA, in the system to ensure. So if you wanted to use it for a specific business use case, then you just jump in and use it, because the quality assurance has been taken care of for you.

So what do you think about that? Hathkar?

Edgar (14:12)

Yeah, generally you're right. I also think that the payment model they came up with is one of the fairest I've seen.

because if your GPT is getting people onto the platform, paying extra fee, and it's basically, it's like YouTube premium, where I have YouTube premium, I don't see ads, but depending on my watch time on someone's video, he gets a cut of the monthly payment I do. And that's basically the same with the GPT, and basically the cost and revenue stream,

pre-aligned. I'm not sure what the cut is but like the general idea of the model and the payment it's definitely something

that I like. I also think that you are definitely quicker just opening a GPT. I have saved this week some of them, just testing and trying it out for our podcast. I'm still not 100% convinced if without any actions, without any interfaces, just a prompt design, if it really improves the experience right now.

I'm a bit iffy on that. But I could also, as I have prompt engineered a lot last year, I can see myself preparing some things. But because yeah, you're right, you can of course go into the system settings and set one system prompt up, but you'd like to have different personas. You have maybe one Python coder, one JavaScript coder, stuff like that.

I'm still not convinced, honestly, that it's better than just using GPT-4. But... And that's also something I wasn't aware of. Are GPTs using GPT-4 by default? Because some of the responses seem to be worse than me using GPT-4. Well...

Svetlana Makarova (16:23)

Do you mean that the GPTs that are built that you have access to, whether or not they use... Okay, even if you have a subscription plan, you would assume...

Edgar (16:29)

Yeah. Yeah, so because we cannot choose anymore, right? And I was a bit, I had a coding task yesterday. I got into like a GPT coding assistant.

Svetlana Makarova (16:35)

I was here.

Edgar (16:45)

in the hope that it just reduces the explanation part, because if you ask for code, it explains everything. Sometimes it doesn't even give you code, it's just the explanation. And so I was hoping to get the GPT, which just has pre-applied the stuff that I have to write manually, otherwise to leave the comments away, give me the proper solution, only give me back the code, stuff like that.

I didn't do this. And it was the highest-rating coding assistant GPT. Yeah, I'm not convinced yet. Also, it's like the ability to copy it is pretty easy. Like, you even have a chat to create a GPT. So if you see a GPT that's getting a lot of usage, it's basically a no, like it's a five-minute task to recreate the GPT.

Svetlana Makarova (17:27)

Yeah.

Edgar (17:40)

as long as you don't have any Zapier actions, stuff like that, we will get on this later. But yeah, I'm still, I'm not sure how much usage this will really generate and how much this is really a business model to look forward to.

Svetlana Makarova (17:58)

But do you think it's because you're kind of, you kind of live in this world, right? You've been in the AI scene for quite some time. Is it because you have higher standards for what you expect of these AI tools? And do you think that there's value still for business leaders?

Edgar (18:15)

So if, I think if I have a high rated GPT, which is called coding assistant, it has to deliver better results, or at least better answers, not even results, but better structured answers than the normal GPT-4.

and it kind of was worse. Like it was worse to a point where I was thinking about which model is being used. And that's not good because the models are vastly different and not knowing which is used. It's not speaking for the GPT. So yeah, I think the idea itself is nice. And if you start interfacing to external data sources,

it becomes more and more useful. But then again, you have to be the owner of the API to really benefit from it, right? So for companies that already have a business model, that already have a tool which people use at work.

this might be a good chance. Or if you have smaller companies that don't want to do their own GPT, they might rely on partners to sell their GPTs in their name or stuff like that. But yeah, I'm not sure if a store... I wouldn't bet right now to build a business on GPT creation.

Svetlana Makarova (19:43)

for smaller creatives and maybe creators who are, could potentially, I mean, again, they build the skill sets. Sometimes I feel like they're building some of these GPTs just to try things out. So from a business angle, they may not be delivering a lot of value. So yeah, I think I would agree that there's still a lot of noise. And then you definitely with this initial phase of like the launch that just happened, I would wait out until...

kind of the strongest survive to see, which, and the strongest that survive are the ones who can deliver value, because they're attracting more and more users. People are providing it feedback, and they're improving it, right? The ones that don't get picked up over these couple of weeks, I think you'll be able to see. But I think that there are still, again, some things to explore. There are some efficiencies to be gained, especially if you're new coming into the chat GPT world, and you want to just jump in.

Edgar (20:21)

Yeah.

Yeah.

Svetlana Makarova (20:43)

start using it for specific tasks. That could be, again, worth exploring, but again, just wouldn't bet all of your work on it.

Edgar (20:52)

Yeah, like you said, so that's also where I think GPT creators should take a bit of responsibility because if I create a GPT, I should at least try to make it.

work better. But then it's all again like the prompt might work in Python and doesn't work on JavaScript which I was trying yesterday so might be there. But yeah, we will see how this develops at the moment. I think there is a lot of like quick fire people try to get to grab some attention. Like always, and it always has been like, like this, things

Svetlana Makarova (21:36)

Exactly.

Edgar (21:36)

And if it really delivers proper useful shortcuts for not me being the one to prompt everything again and again on every chat.

I'm fine with that. So like I said, I saved several of them because I have several topics which I have basically every day or every few days and just being able to click on one of the GPTs and then have preset chat environment. I see myself using it, but right now the results are a bit iffy.

Svetlana Makarova (22:13)

Yeah. And some things to also, again, note about these prompts, it's not just about providing context, but some of them are powered with frameworks, right? So those are really hard to teach almost, or sometimes they're not even embedded in the core chat GPT models. And so there are specific use cases, let's say personality or leadership assessments that...

Edgar (22:28)

Yeah.

Svetlana Makarova (22:42)

Yeah, you could potentially get access to them online or get prompt against it. But if you really wanted to get an assessment, let's say done, there are GPTs that are taught through these frameworks to achieve. So it's a guided conversations through the chatbot to get you to a specific result. Or again, it's

Edgar (22:51)

Mm-hmm.

Yeah, honestly, I didn't even think about it. But yeah, of course, you could like upload proprietary knowledge, which is then fed to the GPT. But honestly, I have, for example, a template built on botpress where you can do the same, then chat with the knowledge and charge after 10 messages, which like from a business perspective, if I have a fellowship anyway, a fellowship, then...

Yeah, it might be the better solution.

Svetlana Makarova (23:33)

There's nothing.

Maybe there's not a significant gap or value add. And I think you're saying it, I think that's what it comes down to. So I think folks who have experience, is it better from a user experience to give access through GPT or make your, kind of as you mentioned, web experience, what you do really well, kind of, if folks who you're working with are on your website, so maybe it's just better to build your own.

kind of experience on your website without kind of diverting them to the store to download it and use it via another app. It's been like a big, maybe a topic for another day, but I think the emergence of so many GPTs, apps that use GPT has instead of increased productivity has actually declined because there's just too many point solutions. So I think what you're...

the comment you're making, I think the connection I'm drawing here is, like sometimes it's just better user experience if you bring that chat GPT value to your own kind of enterprise or tool, like just to kind of streamline the workflow and for continuity of that experience. Sometimes it just doesn't make business sense to drive them to use a GPT.

Edgar (24:41)

Yeah.

I think it's also a good point to switch a bit the focus on the app itself. Because I would just try to explain some basics. So GPT stands for Generative Pre-trained Transformers. Transformers is a technology in AI research.

really, really a lot of text. And so it's basically, I always see it a bit like as a natural language processor more than anything else, because that's what it's really good at, it's that natural language and everything that comes with it. And that's, there are two models, they're like GP 3.5, which started the whole thing in 2022. They already updated this, but it's still like the base models to the same.

Svetlana Makarova (25:28)

Mm-hmm.

Edgar (25:50)

more capable but a bit slower model because it's bigger with GPT-4. If you don't have the paid version of ChetGPT you're only using 3.5. I think it's still like that, right? And if you pay like I think $20 a month you get the...

Svetlana Makarova (26:03)

Yeah, it is.

Edgar (26:12)

You get the GPT-4 version, which is still not unlimited. I think it's like 50 requests every three hours or something that you can set up because it's like, it's a big heavy AI model, which is triggered every time you send a message. And it's also something interesting. Like I can ask for, for like a fairy tale.

Svetlana Makarova (26:20)

Mm-hmm.

Edgar (26:40)

of like five lines.

or I can ask to explain the theory of relativity. And it's like triggering the same model and it's like the same cost involved in answering that mostly. And yeah, it's kind of interesting. But yeah, that's basically the difference. Like you have the models, and JJBT itself is an app. It's an interface for you to work with that models. Comes with a lot of additional stuff.

Svetlana Makarova (26:53)

Mm-hmm.

Edgar (27:12)

important differentiation because of course I can build stuff into my website and use GPT models to do that but then I have to get my own interface I have to get my own setup and all the tooling security stuff like that so there's a lot involved in that and that's what ChetGPT the brand and the app it does for you

Svetlana Makarova (27:32)

Mm.

takes care of all of that heavy lifting. And I think another idea is also what's great is that they have a huge user base as well. So just something to consider. So they're helping them iron out a lot of kinks out of the system. So yeah, there's definitely considerations. And I think we can talk about that more too. It's like, how could you bring...

Edgar (27:42)

Just, yeah.

Yeah.

Yep.

Svetlana Makarova (28:03)

a large language model into your business. And what ways could you use it? And I think there's, again, even OpenAI themselves have these multiple tiers of how you could actually enable the use of Chagy Pt, whether it's via Teams, Enterprise, or you build your own tools off of it. But yeah, I think that's worth even a much more deeper dive sometime in the future.

Edgar (28:27)

Yeah, definitely. I find it always funny that people are like also not really getting the potential. Because we have a high ranked politician and he was like trying it out on some really high level exam like for writing an essay.

And he was like, yeah, it's just language synthesis, but it wrote an essay and it was fine. And I was like, and he was like, as if it was nothing. And he was like, this is a machine writing an essay on a high level university grade level. So, and you're fine with it as like a really smart guy. And that's not an achievement, like, I don't know.

Svetlana Makarova (28:58)

Ha ha.

So I've given it some thoughts, and this is my take, because I was just speaking to someone behind the scenes. And I think maybe I'll explain. I share your same kind of thinking as to because people don't really realize what's behind the scenes, but that black box does to really find it impressive. And I liken it to an iceberg. So an iceberg, what users see.

Edgar (29:42)

Mm. Yeah.

Svetlana Makarova (29:43)

is what's above the water. So they fly, they swim by it, and it's a beautiful iceberg, and I can see, you know, and it looks amazing. There's penguins on top of it. But what they don't see is how big of a machinery, or how big that iceberg is below the water, and how impressive it's been built. And so I think you and I have kind of a deeper understanding as to how complex these systems are, and what they actually do, and I think we were just kind of touching base on that before we started, but you know, these are...

Edgar (29:59)

Yeah.

Svetlana Makarova (30:12)

I would liken running a GPT to starting a cruise ship. So every time that you're running a prompt, just think of a cruise ship that's standing by outside of your house. And every time you're starting it up, it's like all of these different engines or pieces of machinery that are starting to ramp up and really run. And then you basically turn it off. And that's why there's a lot of costs associated with them. But the infrastructure, like the infrastructure

Edgar (30:32)

Yeah.

Svetlana Makarova (30:41)

the different pieces that come together to actually power those models is really, really impressive. And I think the logic of how they're built, I think, Edgar, you probably can give it better justice as someone kind of that's more technical than I am. But it's the work that they do as far as like understanding the text or being able to train on so much data.

and then to be able to have such a broad reach of topics and then do those things really well, that's what's really impressive. And I think you and I could give it justice, but folks who are just seeing that top of the iceberg, they're just like, oh, just went from a prompt to give me an answer, like, what's so different?

Edgar (31:20)

Yeah, the other thing is, I have it a lot also when demoing my own AI solution, and it just looks like a chatbot looked the last 20 years. And people are not that impressed until you start. I have now a trick. I start prompting it, like talking to it in English, and then show them that all the documents are in German.

Svetlana Makarova (31:44)

Thank you.

Edgar (31:44)

And then they have like, ah, okay, that's interesting. So it's like on the fly translating stuff like in every language or every main language you need. And that's like where you have to emphasize also for all of you out there that are selling AI stuff, you have to emphasize the bits, like not the results, but the cases where it works even if it shouldn't. And then that's kind of interesting.

show that it, even if it's not in the document, can come up with a solution and then get back to the document and go on. Stuff like that. So I think there's still a lot to be learned also from our side. But I created like a five-minute demo video in English, which I'm kind of proud of because it was like the first time I was able to show where is the interesting part here.

Because of course, from a technical side, I know the interesting parts are like, it's ridiculous. What I've done beyond, like, the hood...

It's not comparable to anything I've ever programmed. Like it's not even close. It's a complete new paradigm. But what I want to emphasize also is don't underestimate the language processing part because I honestly, before I started using LLMs, I wasn't aware what part natural language has in our lives and how important it is for basically everything. And thinking about it,

a computer, like any PC or Mac or whatever, being able to understand natural language on a free basis, like you can throw everything in it and it basically can understand it. That's huge. And it's writing essays, writing novels, writing poems, stuff like that. It's like...

Holy shit! It's really groundbreaking because what you would have needed to do back in the days to achieve even close to that freeform style conversation, you just couldn't do it.

Svetlana Makarova (33:50)

Yeah.

Edgar (34:11)

and what you can then derive from the natural language processing and what you can infer and then go back to classical programming or classical solution architecture and take the natural language instructions and transfer them to actual actions, that's where the magic starts. That's something I think that opened my eyes with this Chai GPT app. They did a great job bringing people closer to that.

Svetlana Makarova (34:38)

I think they've rethought the approach. And so why is it different and why is it such a big, difficult problem? I think I experienced that firsthand because I think I've went into one particular use case where we build our own NLP kind of model. I don't want to say from scratch that we did use established components, but we had to tune them because if you kind of think about, they're really smaller models that are exposed to specific.

text sometimes, I mean, they're rule based. So they're trained on information that they've been exposed to. So if this than that, if they don't have exposure to specific dictionaries or specific. I don't know, bodies of work, they're just not going to be that creative. So you have to do a lot of fine tuning. You have to train. You have to do manual intervention. You have to come up with a lot of samples from humans to be able to fine tune the results. So.

Edgar (35:29)

Mm-hmm.

Svetlana Makarova (35:37)

is just a huge undertaking. That's why I think it's no surprise that training and opening eye, they don't do it frequently. And I think why don't they don't really have the frequent kind of real time or even more batched like monthly releases of their train models, because it takes a really, really long time to get the data. And so, yeah, sometimes I mean, you're three months behind on the training data, but it takes such a just again, if you wanted to repaint the entire.

Edgar (35:55)

Yeah, yeah, it's huge.

Svetlana Makarova (36:07)

cruise ship, how long will that take? You know, you can't do that in a day. It's a huge, again, machinery that's behind the scenes that it takes, you know, quite a while to train. But again, if you think even stepping back, well, before AI, how the heck, why is this so impressive that all of these AI systems can actually understand their interpret language? So before you did a keyword lookup, you looked at lexical search. There's no semantic meaning.

kind of an interpretation. So what semantic meaning means is that you search by a specific keyword instead of just looking by the meaning or the exact spelling of that word and looking basically at the index of your database. It's now looking to say like, okay, well, when you say this, you may also mean that. And it's looking at different ways of phrasing that same either term or phrase. And it's looking at...

more bodies of different texts to be able to surface again better results back to you. So again, before AI, it was very much rule based. If it doesn't exist in your index, and this is why I think old Google really suffered in qualities because it was really keyword based, right? So if he didn't, if that

Edgar (37:19)

Mm-hmm.

Svetlana Makarova (37:21)

keyword you're looking for wasn't included in an article or some website, just won't see a result. But now, again, AI is able to interpret that like whenever you

type in the word bank, you know, I want to put my money in the bank, you're not talking about the river bank, you're talking about bank, but you may also talk about finances and something else. And so it's looking for things that are semantically similar. So it's able to again, understand the intent behind what you're actually looking for, that just traditional keyword search is not able to do. And I think right now what Chad GPT is, I'm sorry.

Edgar (37:44)

Mm-hmm.

Yeah, you had a beautiful LinkedIn post about this some month ago. It's been some time where you talked about the beauty of understanding the intent behind the stuff. Because in a way it's a compression of language, but you don't save the language itself, you save what it means. And that's kind of cool.

Svetlana Makarova (38:23)

Yeah, and again, and I think the way to even break out all of AI, I'm going to only ruffle some people's feathers, but ultimately, what AI truly comes down to, they're pattern finders. So if you think about what they do is you give a data, they're going to find patterns and they're going to come up with some, you know, algorithm, some mathematical way of explaining it. Right. So that's basically how computers are able to reason. So when it comes down to machine learning, you know, you have different

Edgar (38:35)

Yeah, definitely.

Svetlana Makarova (38:52)

equations or algorithms that it's going to come up with. It's basically an equation when it comes to language. What are they going to come up with? Like they just need to get the gist. They're going to need to find patterns in language to be able to then generate it. So it's still they're figuring out a pattern in language because they don't really store it. And so the way that we can explain it is the gestalt. So when I say something to you, it's like understanding the intent. It's understanding the core meaning. But when you look at the.

Edgar (39:03)

Yep.

Svetlana Makarova (39:21)

knowledge or basically the actual behind the scenes, basically that iceberg of large language models, they don't store any of that data. They just understand the gist, those patterns in language. And that's what they basically pull on for that following step to actually generate back. But yeah, AI, all it is our pattern finders. They just are.

Edgar (39:30)

Mm. Yeah.

And honestly, it's reflecting on me, reflecting on how my kids develop. We are not, there it is. So for everyone on the audio stream, I just got a little notice from my Macbook in the video without asking for it. But yeah, just sorry to get back.

Svetlana Makarova (40:04)

Thank you.

Edgar (40:11)

So I'm really fascinated how much of the pattern recognition, how much of the types of errors it makes, are like how I was used to see in humans, I used to see myself, and how much we are just like...

odds calculating pattern recognizing machines in our brains like on a biological point of course but we getting really to something not only understanding and getting better technology but also understanding what language means for the world and for us humans and for our brains I think it's really interesting

Svetlana Makarova (40:59)

One thing that I'll mention is, well, because you brought up kids. We have kids of both sides. But I do think that it's important to touch on hallucinations and why they're different. So AI has been built in research. I think the way some of these systems are designed actually around understanding how children learn, because they're kind of, you could consider them blank slates. And so their initial learning experience could be potentially mimicked.

Edgar (41:22)

Mm-hmm.

Svetlana Makarova (41:29)

By computers. It's harder to mimic how we think because we just have all of these established systems in our head Through learning curriculums and everything but with kids is different One differentiator so if you were to basically say well is chat GPT or the these LLMs same as a child From their maturity level no, but the way that they were kind of designed in Help basically were thought initially about was through

how children learn and how they interpret language and reason and make decisions. The key differentiator that I'll mention, I think hopefully you will agree, is hallucinations. When you ask a child, hey, what's this object? And you show just some random object to them. Your child is going to be like, I have no idea, dad. Leave me alone. I don't know. Or tell me. Tell me so I can learn.

Edgar (41:59)

Mm-hmm.

Yeah.

Svetlana Makarova (42:23)

the difference with these large language models or a chat GPT when you ask it, it's just going to be confident in responding to you and make up an answer. And it's not going to tell you whether it knew it as a factual thing, because again, it's just looking at its database. It has some pattern that I found. And it's like, okay, well, based on what you're asking about, I think it is similar to this thing over here, this piece of knowledge that I have. So I'm just going to make up an answer that is

Edgar (42:34)

Mm-hmm.

Mm-hmm.

Svetlana Makarova (42:52)

close to, I think, what you're looking to hear. And it's not going to give you any indication that it's based on false premises. The difference with a child, it'll tell you right off the bat.

Edgar (43:05)

Yeah, that's also more of a technical challenge and more of like a conceptual challenge of this thing. I'm really curious to see how they solve hallucinations because in the current state of the technology and how the math behind this works, I don't see it being solved. I see it being reduced but solved.

don't know, because they would have to change the system or have the system to reflect on itself, which will be the way to go. We'll touch on that in another episode. Yeah, but you're right, hallucinations are stuff. That's also something, if you use it in any corporate case, you really have to keep that in mind. It is a nice language generator, and it's...

insanely capable already.

but it's not perfect in any sense of the meaning. But for me personally, I say, hey, if it's 85% there, it's already useful. And that's basically what I also suggest a lot of companies when I talk to them, is like, hey, it is only 85%, but humans aren't 100% either. If you look at the benchmarks for LLMs, you always have the human score,

benchmark and say like it's at 87% and you say okay it's not at 100. Then you see the human benchmark is like 83.

Svetlana Makarova (44:38)

Hehe

Edgar (44:40)

And that's basically how I approach everything I do with AI today, is like, hey, I have to find a way how I talk to it to get the best results, and it's the same I have with my employees. So, like, I don't want to humanize it because it's nothing to do with humans. But as a language processor, it's insanely useful if you accept that it's only 85% right.

Svetlana Makarova (45:09)

You touched on something that's just kind of sparked an idea in my head that I thought it was funny. So people humanizing chat GPT, people thank them. Oh yeah, thank you for your output. It's been like, I really enjoy talking to you. Yeah, it doesn't understand, again, it doesn't have a personality or anything like that. But one other thing that I find really funny is that there's actually a prompt that can actually, if you include it with.

Edgar (45:20)

Yeah, I think them too.

Svetlana Makarova (45:38)

your request, it'll actually improve the accuracy. No one really understands how the heck it works, but I think it reads, take a deep breath and work on this problem step by step. Which again, has like this humanizing component to it. Like you're asking Chad GPT to take a deep breath and then think about this problem step by step. But again, no one really understands why that works, but there is...

Edgar (45:50)

Yeah.

Yeah.

Svetlana Makarova (46:06)

some, again, evaluation has been done, if you include that phrase, it does tend to give you a better output.

Edgar (46:15)

So let's.

take it back a bit to the CHWT app. I just want to briefly talk about the subscriptions we have because like we said, there's three tier, there's a pay tier. We also have a team subscription, which is a bit more than the paid subscription and you can work together on stuff and share stuff. And then there is an enterprise offering, which is from my point of view and from a business point

maybe even the most interesting. Because we have one thing that's coming up, always talking about AI. In Germany it's even more like Germans don't even think to use anything before they don't discuss everything. So we have to talk about privacy. And of course we said like there is a lot of language data used to train the models. It has to come from somewhere.

And they scrape from the internet mostly. No, not even mostly anymore. Like right now, LLMs generating data for LLMs. I think it's getting more and more synthetic. But they also use what we type in an R3 tier. And if we say, hey, this was a good answer and click on the thumbs up button, then it shows on their end and they use it to train the model and to get the next model even better.

which of course can have privacy concerns.

Give us some more background on that.

Svetlana Makarova (47:59)

So I think, do you want me to dive into the different tiers? Because I think you've touched on a few things there about the different.

Edgar (48:06)

Yeah, like what does the enterprise tier do different to comply to privacy terms?

Svetlana Makarova (48:16)

So I think with the, so the different tiers basically, the most secure, I would say, is the enterprise plans. So they enable you to have your own almost like instance or secure access to chat JPC that does not share your prompts or data shared with the model to actually like prompt the systems.

And then you kind of keep that information private. So they basically keep a promise that everything that happens in your organization stays with the organization. So and that is different from what I think, Edgar, you were just talking about. When you have access to ChadGPT and you're uploading your documents, that's fair game for OpenAI to use. If they find that deem it useful to someone else, whether or not it has confidential data or anything like that, you've put

that document or uploaded it in OpenAI's platform, whether it's on paid subscription or not, GPT 3.5 or 4, it's fair game for ChedGPT or OpenAI to actually use it for training. And I think that's the key aspect. One, you have kind of a promise that whatever happens behind closed doors in your organization stays with you. You just have access to this soft kind of platform and lots of other features. And I think I just kind of did.

a quick search as to what other things that you can have. You're on enterprise dashboards, you have access, you have improved availability and performance. So again, it's not just about data privacy, but you're able to get better features on top of it. But from a perspective of privacy, enterprise plan is probably the most secure. From the, again,

that require no development. There are other options where you could use Chad GPT or the GPT models, and you can actually build your own software, which we can talk about more, but from what you have access to, that doesn't require a lot of those deep dive integration. Chad GPT or enterprise is probably gonna be your best bet.

Edgar (50:28)

Yep.

So, yeah, definitely. So I also want to add to that, if a model is trained with your data, like we told earlier, it's not saving the data, but it might be able to reproduce it from the gist of it. And to, like I think the New York Times just sued OpenAI and Microsoft for using OpenAI Times articles, because they were able to prompt GPT

to re-write a certain article word by word. So it is possible to recreate data at least to a certain point. So if you are uncareful with confidential data, it might pop up.

later down the road with someone random using the model. And it's not even, it's not only using it in ChetGPT, like that's the base model, GPT-4, which can be used by everyone for everything in every context and you wouldn't even know. So having your data secure is...

is important to think about. It's not a please don't use it, danger, no, but we have to see the subscription enterprise for example or with other tool links to be in an enterprise great compliance commitment.

Svetlana Makarova (52:01)

And I think what's important also that I'll mention why. I think the thing to note here is like, yeah, not to use it, but just abide by your company policies for what information you can't share with others. So whatever plan or whatever confidentiality agreements you have with your customers or the level of details you can share with your vendors and stuff like that, I think still applies here. So you shouldn't be including your customers' data.

Identify information or proprietary secrets to uploading those documents to Chad GPT. So again, some of those same rules you have in your employee agreements still apply or even your company documents to Chad GPT. So all we're saying is that when it comes down to open AI, you just want to be more careful, double checking your data that you're uploading so you're not including that because...

once you submit your prompt with the document to OpenAI, again, it's not one of those things that you can just call up OpenAI and say, hey, I've made a mistake. Two months ago, I've sent you a document. Could you just delete that record for me? It's unfortunately not going to happen. And it's much more complex than that. Because again, how do you remove this gestalt? And where does it?

Edgar (53:09)

Yeah, please remove it.

Yeah, it's not going to happen.

You don't. You don't. We don't know.

Svetlana Makarova (53:26)

Where does it live in the model? Where is that needle in the haystack? Yeah. So again, it's not to use it, but you want to make sure that you double check the documents or your prompts that you're

not including proprietary information before submitting, especially if you're using the kind of anything else other than the enterprise plans.

Edgar (53:41)

percent.

Yeah, definitely, yeah. And I think it's not only Chetchupiti, it's definitely something you have to be aware of using any AI tooling out there. Talking about any AI tooling, is Chetchupiti our only option?

Svetlana Makarova (54:06)

Oh, I use so many. I have a list of favorites, chat bots. So I actually use, and I'll say why I use multiple. I think that they have their own strengths and weaknesses, but in addition to chat GPT, I use Cloud, I use Bard, I use Proplexity, I use Pi. Again, they all kind of.

Edgar (54:28)

I read so much about perplexity, but I haven't used it yet. It's a new search approach, right?

Svetlana Makarova (54:32)

You should check it out. Yeah, I think that... So what I like about Perplexity is for fact checking. So, and that's why I think I would recommend friending other than Chad GPT friend another chatbot because you could use them to fact check each other. So Perplexity is awesome for that because it will actually surface the articles or things that it's able to find and it'll surface kind of the preview of what that article or where that information could come from. So you could just basically click and access.

So if you're again, double, if you want to, if you're checking the accuracy of Chagypti's output, I would just take copy and paste what the Chagypti did and then put that in quotes and then say, fact check this and put that into perplexity and it should give you kind of an evaluation of what that's used. Yeah.

Edgar (55:13)

Interesting.

Interesting. That's cool.

Definitely. Another thing that's in my space, I'm really deep into Microsoft products and Microsoft B2B space is like Microsoft Copilot. Everyone who uses Microsoft basically has heard of it. It's going into Windows and stuff like that. So everything that Microsoft does with AI is now named Copilot in some way, shape or form. And if you like, I noticed yesterday that if you use Copilot chats, like copi

Then you have an active business subscription. Your data is saved by default. So it shows a little protection badge in the top corner and says, hey, we are protected here, it's fine. Which I didn't have to do anything. I just logged in and I seem to have the right, I seem to have the right.

Licensees active and was good to go. It's not a general thing, but if you but there are office and Microsoft cloud subscriptions and licenses that automatically give you like a secure chat experience which is kind of neat and Did you use Google part I barely use it my brother's using it a lot, but he works for Google

Svetlana Makarova (56:47)

I do.

He's biased. I use BARD for more because it is much closer to real time. So they have access to, I'm assuming, in like some ragway, which again, we'll talk about more because I don't want to bombard everyone with all of these details, but they have access to more current information. So whenever I'm looking or researching a topic that is more current and I'm just kind of brainstorming some things or things that I need to know or like give me a summary of every.

Edgar (57:11)

Mm-hmm.

Svetlana Makarova (57:19)

or a different perspective on this event that just happened yesterday. I'll have better luck at, you know, prompting BARD than ChadGPT that's like three months delayed in its training data. So yeah, it depends on, again, on the use case. And this is what I'm saying. I think each model has their own pros and cons, but BARD I use for, it's not that great in generating knowledge. So if you want you to create like, you know, artistic pieces or, you know, some.

Edgar (57:32)

Mm-hmm.

Svetlana Makarova (57:49)

generate some content from it. Not that great, I would say. Still, ChaiJPT or Cloud are better for that. Cloud is definitely my favorite. But BART is really great for research. That's like my primary use case for it.

Edgar (58:04)

Okay, interesting. Yeah, I honestly I only use JTPT. I was thinking about trying Claude. I use a lot of open source stuff in my programming and try it out. Also locally on my machine because I'm really like even thinking about having stuff run local on my computer is really cool because in Germany, we have really bad reception, especially for mobile internet and if I'm sitting in a train down to Munich, it's a seven hour

It's a seven hour train ride and basically three hours of that seven you don't have on the internet. And having a local model with all the knowledge baked in, that's nice. It just might help in some situations. So yeah, it's really, really interesting to use. Yeah, besides that.

Like we said, there is a lot of options and possibilities, but I think we have to talk about it in the next episodes because we are running out of time. Yeah, so I think we could like

Svetlana Makarova (59:08)

That's an awesome conversation.

Edgar (59:16)

talk a little bit more about the GPTs and how you could use it in your company to access company interfaces, stuff like that. But yeah, I think we will move that one to another episode. So stay tuned. Follow this new AI boardroom podcast on the channel of your choice. We will be on YouTube with video. You can find us on Spotify.

Apple Podcasts, Deezer, I think. I don't even know how they all are called, but we will be there. You will find us. And yeah, Sotlana, do you want to add something?

Svetlana Makarova (59:51)

I'm sorry.

Now I look forward to it. I think, I mean, we're really fluid, I think, with some of the topics. So we'd love to hear from you. So if there is a topic that you've kind of heard us touch on during any of the episodes, feel free to reach out to us, because I think, you know, at the end of the day, we're doing it for you. And we want to make sure that you're getting the values worth. So of listening to us and taking the time.

Edgar (01:00:20)

I also love to hear myself talking honestly.

Svetlana Makarova (01:00:26)

Yeah, but reach out to us if you have any topics for business, if you're using it. We'd love to know more how you're using it for, if you're exploring use cases still for Trajit BTN that you're kind of thinking that that's going to be your first use case and you want us to dive deep into specifics. I'm happy to do it. So we can definitely be flexible and adjust to what you guys are curious about. So yeah, I would just invite people to reach out.

Edgar (01:00:49)

And we not only can, we want to be. So yeah, just hand us over your thoughts, your questions, and we try to consider it and to give you the information you need.

Svetlana Makarova (01:01:07)

Perfect. So what's on the next episode, Edgar? What do we have going? What do we get people excited about next week?

Edgar (01:01:12)

Honestly, I feel like just talking at so many things, I would say people just have to tune in. And maybe we...

Svetlana Makarova (01:01:19)

I'm sorry.

Edgar (01:01:25)

Maybe we also get some new news next week, so let's see. I'm not sure, I have so many things I wanna talk about after this episode and where I think people can derive a lot of value from. So yeah, like Sveta and I said, if you have anything, let us know, please. And we are really...

looking forward to talk about all the topics that are keeping you awake at night. Considering an AI, of course.

Svetlana Makarova (01:01:59)

Yeah.

Thanks for watching!

Edgar (01:02:03)

There is also one call I already making because I hope this episode will be heard like month down the road. And if you're hearing it, it's most likely that our website, theaiboardroom.com will be online where you can find summaries of the episodes, can find profiles, can contact us directly if you want and need to. Also you find us on LinkedIn, Svetlana is like a superstar on LinkedIn.

Svetlana Makarova (01:02:33)

Thank you.

Edgar (01:02:34)

at least in the AI space and rightfully so to just say the least and yeah you find us there talk to us chat to us we're looking forward to hear from you and yeah I look forward to yeah

Svetlana Makarova (01:02:52)

We're open door policy, I think. Again, reach out to both of us. Yeah, if you have any specifics, I think we'd both be excited to hear from you. So yeah, but check out our website. There's gonna be a way to communicate more with us, give us feedback and all kinds of things. But yeah, we would love to get your feedback.

Edgar (01:03:13)

Yeah. Then I have to say, Svetlana, thank you very much. It was an honor to co-host this with you. I really, really enjoyed our talk and all the preparation we did beforehand. So yeah, like to prepare some more topics for you all. And that's it from me. Last words.

Svetlana Makarova (01:03:22)

Likewise.

Thank you. Yeah, we look forward to, again, to continuing the series. So thank you, Edgar. Thank you for your patience also today with all of the technical difficulties that I've experienced. Let me just kind of summarize that. So he's really, really patient in guiding me through all of this stuff. So yeah, no, it's been an awesome episode and I look forward to the series. So I hope everyone has a great, awesome day.

Edgar (01:04:06)

Bye bye.

Svetlana Makarova (01:04:07)

Bye.