×
Evaluate Your Team's AI Readiness

Take our official assessment and explore training paths to build secure, compliant AI workflows.

Access Portal

AI at Work Podcast

Ep #29 When Models Lie and You’re Still Paying for the Tokens

AI at Work, AI in Business

Audio episode

Listen to this episode

When AI models hallucinate confidently while burning through your token budget, you need more than better prompts you need better processes.

In this episode, Kevin and Eli explore the reality of working with AI that lies while you pay, the evolution from saved prompts to voice-triggered workflows, and why validation loops matter more than model updates. From enterprise governance requirements to tech stack consolidation, this conversation cuts through the hype to reveal what actually works when deploying AI in real business contexts.

Key topics covered:

✅ Why model reliability requires systematic validation, not perfect AI

✅ The shift from skills to snippets and voice-triggered workflows

✅ Enterprise governance requirements when litigation risk is involved

✅ Tech stack consolidation as AI tools mature beyond experimentation

✅ Cost economics driving procurement strategy changes

✅ Building quality control loops that catch hallucinations before implementation

TIMESTAMPS:

00:00 — Intro and Sora commercial viability discussion

05:30 — Video generation landscape: Veo, SeeDance, Higgs Field

15:00 — Model quality regression and hallucination experiences

25:00 — Skills vs snippets: workflow evolution deep dive

35:00 — Enterprise governance and validation requirements

45:00 — Tech stack consolidation and platform comparisons

Explore practical AI implementation: https://assessment.ascendlabs.ai/

Book a conversation with Kevin: tidycal.com/kevinwilliams

Show Notes & Links: https://www.spark6.com/podcast

Submit listener questions:

[email protected]

[email protected]

Check out Kevin’s stuff:

AscendAI: https://ascendlabs.ai/

Follow Kevin on LinkedIn: https://www.linkedin.com/in/kevinguywilliams/

Check out Eli’s Stuff:

SPARK6 Agency: https://www.spark6.com/

Sign up for FREE AI Framework Friday Newsletter: https://www.spark6.com/newsletter

Follow Elijah on LinkedIn: https://www.linkedin.com/in/elijahszasz/

Transcript and citation details

Transcript

00:00 Play from here Hey, Eli, how's it going?

00:13 Play from here Hey, what's up, Kevin?

00:14 Play from here I just noticed something kind of disturbing on our pre-roll video, which is 100% AI generated.

00:18 Play from here It kind of looked like we embrace at the end of it.

00:21 Play from here Like, did your old marketing guy do that intentionally?

00:23 Play from here Is that just like a little parting job?

00:25 Play from here I think it's not what our customers embrace when they see each other, but in a weird kind of side hug way.

00:31 Play from here So my favorite thing that I created in Sora, sort of RIP Sora, which of course was the pretty fun to play with,

00:39 Play from here but commercially dubious, commercially useless video generation tool for open AI.

00:47 Play from here And I think they killed it when somebody realized they were spending like $15 million a day or something like on it.

00:54 Play from here For people to create dumb videos and things like that.

00:58 Play from here But my absolute favorite video that I created was like, you know, tele-operators, basically with robotics.

01:06 Play from here And it looks like it's this like really warm and cozy affair of two robots in this cozy living room.

01:14 Play from here And then it backs away and it's two plumber looking guys with their like video, like oculuses on.

01:21 Play from here They kind of look at each other and they give each other a big hug.

01:24 Play from here I'm really proud of that one.

01:25 Play from here I have, I kept it like all, most of the rest of my Sora library is gone, but I have that one.

01:30 Play from here I haven't even opened the app in so long.

01:32 Play from here Can you still open it and see your videos?

01:34 Play from here I downloaded a bunch of them to my photo roll on my phone.

01:38 Play from here So I think I still have some.

01:40 Play from here I think one of my favorites was I had an optimist looking robot.

01:43 Play from here I was bringing into the home and showing him around.

01:46 Play from here And then my cat just like attacks him out of nowhere.

01:48 Play from here I was an intruder.

01:49 Play from here Mark, I really quite enjoyed that one.

01:51 Play from here Cats would.

01:52 Play from here They totally would.

01:53 Play from here So I mean, what is this?

01:55 Play from here So this is practical, right?

01:56 Play from here Okay.

01:57 Play from here So, so I, I said, are we going to talk about cats now on the show?

02:00 Play from here So I said commercially dubious, but I actually did use a number of Sora videos in a number of different ways that were really cool.

02:09 Play from here And they were really easy to generate.

02:11 Play from here And it was a real unlock for me.

02:13 Play from here But with Sora gone, your video options are really pretty hard to get to.

02:21 Play from here Are you close enough to this that you can talk to it?

02:23 Play from here Or I, I, I have some stuff I'd love to cover.

02:26 Play from here Am I close enough to Sora?

02:28 Play from here No, if you're close enough to video generation, have you been doing much video generation?

02:32 Play from here Not lately, but it's funny.

02:34 Play from here You mentioned Sora and commercial viability.

02:36 Play from here So my, my brother-in-law has a DTC brand and he has this kind of backyard game.

02:41 Play from here And he was like, just spending so much time and money trying to get footage of people playing the game and doing cool tricks and shots.

02:48 Play from here And he was like, wow, Sora would be such an amazing unlock if I could just get it to render the game to look how the game actually looks.

02:55 Play from here Instead of having an extra side or the net is missing or the paddle's a different color.

03:00 Play from here Like he would just try to regen it over and over and over again.

03:03 Play from here And the exact same problem with EO3 too.

03:05 Play from here Just trying to get the product to look right.

03:08 Play from here And I know there are third parties that are basically using that engine, but can then nail things like the specific artifacts inside of the video.

03:16 Play from here So, well, the key to a lot of this is A-roll versus B-roll.

03:19 Play from here And the idea that perfect is the enemy of good.

03:22 Play from here So in a lot of cases like that, where you need to have something really specific, you probably want to do more traditional filming for the specific.

03:31 Play from here And match the style of what you filmed for, let's call it the A-roll to the B-roll that you're using.

03:38 Play from here And people want it to be out of magic.

03:41 Play from here And it's just, it's not quite there yet.

03:44 Play from here And there's just shots and use cases where that's not going to work, right?

03:47 Play from here Where it's like, this is the product and people have to be using their product.

03:50 Play from here And that's the whole role.

03:51 Play from here There is no B-roll.

03:52 Play from here You know, that's, that's the whole thing.

03:53 Play from here It's like, you know, there are just different angles.

03:56 Play from here There is, you know, some different kind of panning iPhone shots and then a couple of drone shots.

04:00 Play from here All stuff that you could theoretically AI gen, but you got to get the actual, you know, assets in the video to look consistent.

04:08 Play from here And that can be really tough.

04:09 Play from here I mean, even if you've tried to use image generation for my, why did my logo end up with this extra little thing sticking out on this one?

04:17 Play from here And now I can't just get it back on track to do the thing that I want to do.

04:21 Play from here It's just one of the weird AI things.

04:23 Play from here And it doesn't matter if it's, if it's copy or a code or video or images that, you know, you kind of bump into the same thing sometimes.

04:30 Play from here Okay.

04:31 Play from here But I think for our audience, it's important to kind of lay out where the technology sits.

04:34 Play from here So you mentioned Veo, which Google Veo is the Google engine, which is actually very cool and very powerful as far as being able to put out scene based imagery.

04:46 Play from here It's also become pretty expensive.

04:49 Play from here You can generate eight seconds.

04:51 Play from here And I believe the last time I was playing with this a couple of weekends ago, it was about 80 cents per generation for eight seconds of video.

04:59 Play from here And remember, you're kind of playing prompt roulette here a little bit.

05:02 Play from here So also depending on the resolution output, you want to the resolution output, et cetera.

05:07 Play from here And, you know, pro tip is we want to run it at low resolution until you get in the neighborhood of where you want it and then start running it as higher resolution from there.

05:16 Play from here But, you know, we, you hear tell about like the Sundance film festival, the eight and a half minute fully AI generated video.

05:25 Play from here And then you look into how many takes it took them to get that right.

05:30 Play from here And it's thousands and thousands and thousands of takes.

05:32 Play from here It was more than a whole crew of actors and support.

05:35 Play from here Well, it was the one I want.

05:37 Play from here I'm thinking about is, is, is would have been like impossible to do, but it was thousands and thousands of dollars of tokens to do it and a lot of patience.

05:45 Play from here So that's so via three is one that you can do it.

05:49 Play from here The other one that I have been totally blown away by, but we Americans don't really have access to is the Chinese sea dance.

05:59 Play from here Have you played with sea dance?

06:00 Play from here I haven't played with it, but I've seen some of the outputs that are pretty mind.

06:03 Play from here So if so, sea dance is, is technically we're cut off from it, but you can access it through something called a Higgs field.

06:10 Play from here And Higgs field will basically it's doing arbitrage.

06:15 Play from here It has access to the model, but it's pretty expensive to use Higgs field.

06:19 Play from here But the outputs are absolutely amazing as far as being like seamless and like well-generated other tools that you run into anything else in this realm that people should maybe check out.

06:32 Play from here I don't know.

06:33 Play from here I feel like there's just a whole slew of third party tools that are basically leaning on VO for the work, but kind of kind of like you have a bunch of web dev tools or design tools like lovable that are calling on the LLMs to do the heavy lifting.

06:49 Play from here But they've just made it a lot more user friendly to be able to get to the output that you're looking for, for whatever specific workflow you have.

06:56 Play from here I feel like in my Instagram feed, I just get more and more ads every day for these.

07:01 Play from here Like there's a lot of them coming out because like you said, it's not only it's, it's, it's expensive, but what makes it really expensive is that just because you didn't use the right prompt or you didn't happen to get the right output on that dice roll, you're still paying for it.

07:15 Play from here Right.

07:16 Play from here So if you can pay a little extra upfront, like you pay the subscription and you know, this is kind of what you do.

07:22 Play from here It's a centerpiece of your workflow.

07:24 Play from here You're probably going to get the results you want faster with a lot of these tools.

07:27 Play from here And who's to say if they're going to stick around because you know, what we see is that the frontier labs just roll more and more of these user friendly features into their products over time.

07:37 Play from here And then these third party tools kind of, you know, just fall to the wayside over time.

07:42 Play from here At least that's, that's what I think.

07:44 Play from here But yeah.

07:45 Play from here Yeah.

07:46 Play from here I can't say like, well, I've seen amazing results out of this one.

07:49 Play from here I don't really create a lot of video right now.

07:51 Play from here It's more of little spurts of, oh, I need some video for something and then I'll do it.

07:54 Play from here And then I forget about it for a couple of months.

07:56 Play from here Where SORA was just kind of fun to play with.

07:58 Play from here And neither of our businesses are that dependent on video.

08:01 Play from here So it is sort of a, it ends up being a side quest to play with some of this stuff.

08:05 Play from here And I'd love to know more.

08:07 Play from here So the other ones that, that people talk about are one, I'm almost self-correcting.

08:12 Play from here You can still access SORA 2, but you have to do it by API.

08:17 Play from here So you're going to be paying for those tokens and they're deprecating it by September.

08:21 Play from here So, which surprises me a little bit.

08:23 Play from here Like, okay, how about you just price it in a way that you can access it?

08:27 Play from here But the app is dead.

08:29 Play from here So you have to be able to tap into it via API instead.

08:32 Play from here Which is probably more lift than it's worth for most people.

08:36 Play from here Although you think about workflows, like the ability to tap into the API.

08:41 Play from here So you're generating the video and then there's an approval step.

08:44 Play from here And like moving on from there, that's the way that things are going.

08:47 Play from here Another one you hear about is Kling AI.

08:50 Play from here That's the one I probably see the most out there.

08:52 Play from here Yeah, you see Kling a lot.

08:55 Play from here You know, remember the reason we don't have access to a lot of these in the U.S.

08:58 Play from here is because they're, let's just say, slightly fast and loose with IP.

09:03 Play from here Like, you know, you want Mickey Mouse doing something that Mickey Mouse shouldn't do.

09:08 Play from here Well, you could do that with Higgs field.

09:10 Play from here Not that OpenAR wasn't fast and loose with IP either, right?

09:13 Play from here Bad example with Disney.

09:15 Play from here When Sora launched, I mean, yeah.

09:17 Play from here You had Darth Vader playing with Hello Kitty and you just do whatever you want.

09:21 Play from here But these are getting closer.

09:23 Play from here And if you haven't actually checked these out, you know, it's sort of funny.

09:28 Play from here Like, go to YouTube and say Higgs field AI or SeaDance.

09:35 Play from here Again, Higgs field is a platform that can access SeaDance,

09:40 Play from here like see exotic examples and make the date range pretty recent

09:44 Play from here because people are developing skills around this stuff as well that are matching to the tool.

09:49 Play from here So the tool alone can do pretty amazing things.

09:52 Play from here But with a few weeks, a few months of skill developments,

09:56 Play from here there are artists basically who are doing, like, really, really neat stuff.

10:01 Play from here And if you haven't checked it out in a minute, you totally should.

10:04 Play from here Ethan Mollick, you know, we like to talk about Ethan Mollick at Wharton.

10:07 Play from here He's always doing stuff with his otters.

10:09 Play from here So it can be fun to see the generations of otters over time.

10:15 Play from here And now he's starting to do, like, video generation there.

10:18 Play from here Have you seen video otters?

10:19 Play from here I haven't seen the video otters.

10:20 Play from here The other one is Google's Genie.

10:23 Play from here Have you seen Google's Genie?

10:25 Play from here Yeah.

10:26 Play from here Okay.

10:27 Play from here So Google's Genie shows up, basically can build a 60-second interactive video game.

10:32 Play from here Just a few weeks ago, they launched a version of Genie right into Google Maps

10:38 Play from here such that you can basically, you could be looking at the streets of Rome or whatever

10:43 Play from here and, like, create a little, like, go-kart game

10:46 Play from here that is actually using the street view of the streets of Rome to, like, navigate around.

10:53 Play from here That's really cool.

10:54 Play from here Sadly only available in the $250 Ultra plan,

10:57 Play from here which I happen to know both of us are too cheap to have.

11:01 Play from here Yeah, not a chance for me right now.

11:02 Play from here I'm not doing that.

11:03 Play from here I'm trying to get rid of all my Ultra plans.

11:05 Play from here Speaking of Ultra plans and subscriptions and everything else,

11:08 Play from here I've just been going through the most awful rotating Frontier Lab game ever lately,

11:14 Play from here bouncing around, getting frustrated.

11:16 Play from here And when we were chatting right before recording, you were talking about just tech stack du jour, right?

11:21 Play from here Like, what's your tech stack?

11:23 Play from here And we always ask this question of, if you did have to start with one lab,

11:27 Play from here which one would you start with?

11:29 Play from here And I think a lot of it depends on what is your core workflow?

11:33 Play from here Like, you know, if it is all about generating images,

11:36 Play from here you may not want to be using Cloud because you can't generate images, right?

11:40 Play from here There's some obvious things there.

11:42 Play from here But, you know, last we spoke, you were still just all in on Cloud,

11:47 Play from here and that was kind of the center of everything.

11:49 Play from here And I've been talking to a lot of people who've been dealing with tons of just quality regression issues across the board.

11:58 Play from here Maybe not with coding, and I'm just doing less coding on my projects these days.

12:02 Play from here But I wondered if you were in that club of people noticing things happening that weren't happening before or not as often.

12:12 Play from here Outages aside, like, still, like, lots of outages.

12:15 Play from here Like, I keep experiencing little breaks in service, which is annoying,

12:19 Play from here but they usually get resolved within 30 minutes to an hour.

12:22 Play from here So disruptive, but not detrimental to a day.

12:26 Play from here But lots of other stuff.

12:27 Play from here I mean, I'll start with, like, writing output.

12:29 Play from here I just feel like there's been so much regression, like, just truly awful writing outputs.

12:34 Play from here And I'd never use something just right out of the gates that was AI for any piece of writing I'd publish,

12:40 Play from here whether it's just a social media post or a blog entry or newsletter or anything.

12:44 Play from here But I feel like even with an outline or a first draft, like, it has gotten really bad.

12:50 Play from here And I've talked to three other people who do a lot of writing, and they noted the exact same thing.

12:56 Play from here And you don't do too much writing, right, Kevin, with any of these models.

13:00 Play from here So you wouldn't have seen that.

13:02 Play from here But what about just hallucinations in general?

13:05 Play from here Like, I have been getting tons of stuff that is just factually incorrect.

13:09 Play from here And I call it out, and I say, that's not true.

13:12 Play from here And it will just say, you're right, I hallucinated.

13:16 Play from here And I don't know if Sonnet just got a little more dumb with the recent Opus upgrade or what's going on,

13:23 Play from here but I started noticing a lot of the things happening with Opus as well,

13:27 Play from here in addition to just ripping through my usage as well if I decide to use that model for any longer period of time.

13:34 Play from here So have you noticed anything?

13:36 Play from here Because people on Reddit have noticed.

13:38 Play from here I've been in some of these threads, and people are complaining like crazy.

13:41 Play from here I mean, we pledged that we're not going to talk about the latest model releases,

13:45 Play from here but I do find that Opus 4.8, which was just released pretty much last week,

13:51 Play from here has tightened up on certain forms of sycophancy, and I feel like it is a little bit more honest out there.

13:59 Play from here I can't say that I'm noticing the type of things that you're saying,

14:02 Play from here and I'm going to put a major note on it because it actually lends itself into something that we wanted to talk about,

14:09 Play from here which is controlling outputs with the different models and some of what they call the primitive structures

14:18 Play from here that you need to have in place to do it.

14:21 Play from here And I think that the reason that I'm getting away with what I'm getting away with,

14:25 Play from here with Cloud Code in particular, and I'm building like crazy.

14:29 Play from here I'm all kinds of stuff that I have going on at any given time as I look over my screen.

14:35 Play from here I'm not allowed to run Cloud Code during the podcast because it eats so many system resources.

14:39 Play from here that it just came to mind that I'm probably running cloud code, but I'm not.

14:43 Play from here On a machine, you shouldn't be doing all that.

14:45 Play from here Yeah, you need a bigger boat.

14:46 Play from here I don't need a bigger boat.

14:47 Play from here Aged Mac M1, like the poor little critter is just going to like smoke and die.

14:53 Play from here Anyway, I think the control, this is always going on.

14:56 Play from here We are so deep in the weeds.

14:58 Play from here We see these things more keenly than other people do.

15:02 Play from here Those primary skill sets of being able to interpret results

15:06 Play from here and not abdicating as the human are ever present.

15:09 Play from here And I actually don't mind that we get reminded of it sometimes that like,

15:12 Play from here yeah, no, really can't trust this.

15:14 Play from here Like you need to be reviewing things.

15:17 Play from here You need to be reminding your people that they need to review things.

15:19 Play from here But on the dev side, this is the part that I really want to push back on

15:23 Play from here because on our list of things to talk about today were skills.

15:27 Play from here And I want to add to that the skill or the primitive that is slash goal.

15:33 Play from here So I am using a number of skills that are quality aligned.

15:38 Play from here So they're doing constant code checks and feedbacks.

15:42 Play from here It makes it pretty token efficient and it slows everything down.

15:45 Play from here But nothing gets out of my coding process without passing certain

15:51 Play from here quality control feedback loops.

15:53 Play from here And so we'll talk about what skills are in a moment, but probably the most

15:59 Play from here important thing is the forward slash goal primitive, which is first it was

16:04 Play from here created by open AI and it was rapidly adopted by Claude as well.

16:09 Play from here And the idea here is that it's output based.

16:12 Play from here So you type forward slash goal and then you have a prompt of what that output is

16:18 Play from here supposed to look like at the end of the day.

16:20 Play from here And you can be pretty elaborate about it.

16:22 Play from here And what it does is it is it forces the coding engine, be it Codex or be it

16:28 Play from here Cloud Code to go through loops until it actually meets that output.

16:34 Play from here And what I'm finding is I can run a pretty complicated goal before

16:38 Play from here bed and just walk off.

16:40 Play from here And in the morning, like it's done, it's feedback loops.

16:43 Play from here Like, is it going to work perfectly?

16:45 Play from here No, but that's, that's okay.

16:47 Play from here Like it's made real forward progress during the night.

16:50 Play from here And because it only costs you $4,000 that night, but I'm not running it that way.

16:54 Play from here I'm running it on a max plan.

16:55 Play from here Right.

16:56 Play from here So, you know, it will timeout, but like it will get pretty far into the process.

17:02 Play from here I don't need to be running 18 coding agents at once.

17:04 Play from here I'm not running a dev house.

17:05 Play from here Right.

17:06 Play from here So it, it, it works fine.

17:08 Play from here So I, I mean, I, I get it.

17:10 Play from here I get that the models ebb and they flow and that's going to be a constant state of

17:14 Play from here affairs, but they do continue to get better.

17:17 Play from here And Sonnet of today is so much better than Sonnet of six months ago that I'm, I'm

17:24 Play from here just not overly bound up over it.

17:25 Play from here I guess some of the stuff that I thought we were just past that I'm experiencing

17:29 Play from here again, in terms of hallucinations and stuff, that's just factual incorrect.

17:34 Play from here We're ignoring instructions, you know, kind of back to the thing with skills.

17:38 Play from here Right.

17:39 Play from here I think that, you know, we didn't really unpack what skills are or how we use them.

17:44 Play from here And we need to.

17:45 Play from here Yeah.

17:46 Play from here And I think a lot of the terminology gets a bit confusing as well.

17:50 Play from here Right.

17:51 Play from here Like what's the difference between a prompt or instructions or skills or an MD

17:56 Play from here file, right?

17:57 Play from here Like all of these things in a Venn diagram are starting to go less.

18:01 Play from here Like they start to, they start to mean more and more of the same thing.

18:05 Play from here And these skills could be, you know, it's a, basically it call it a prompt.

18:10 Play from here You could call it a set of instructions.

18:11 Play from here You could call it a set of long instructions that are inside of a

18:14 Play from here markdown file that's attached.

18:15 Play from here And the idea is that instead of including them in a specific project

18:21 Play from here that you're running, for example, you could basically just call that skill

18:25 Play from here inside of that project, because maybe it's not something that you want being

18:28 Play from here used all the time inside that project, but you want to call it occasionally.

18:32 Play from here And what we were just discussing earlier, Kevin, by earlier, I mean, I think we

18:36 Play from here discussed it like a week ago or two weeks ago, the fact that with a dictation

18:41 Play from here tool like WhisperFlow, you can have all of these snippets.

18:45 Play from here Basically, if I don't want to always type in my website URL or my LinkedIn

18:51 Play from here profile or whatever it is, I can just have a keyword that I say, you know,

18:54 Play from here Hey, Kevin, great to meet you.

18:57 Play from here Let's connect.

18:58 Play from here Here's mine.

18:58 Play from here I say my LinkedIn profile, and then it populates the whole URL in there.

19:02 Play from here Well, you can have snippets that are really, really long, really long.

19:05 Play from here And I stopped using skills a while ago.

19:09 Play from here And I just use these snippets now because unlike a skill that would

19:13 Play from here just be specific to Claude, it doesn't matter what model I'm using,

19:16 Play from here like what LLM I'm using, I can just say that snippet, I can invoke the same thing.

19:21 Play from here So you have this goals one, for example, another one that I use pretty frequently

19:25 Play from here would be, you know, challenge my thinking.

19:27 Play from here And then I have like this three long paragraphs of specific instructions

19:32 Play from here that are asking for pushback and asking a whole number of questions about my

19:38 Play from here assumptions and are you sure I'm right?

19:40 Play from here And can you fact check this and on and on and on.

19:43 Play from here Right.

19:43 Play from here And it just gets me a better output at the bottom.

19:47 Play from here And it's not something that I want to do all the time, which is why before it was

19:51 Play from here a skill and now I use it as a snippet.

19:54 Play from here Is there, is there any reason why you're still using skills, Kevin?

19:58 Play from here Or is it just kind of like a habit?

19:59 Play from here Or, and, and first I think we need to, we need to clarify that what we're talking

20:04 Play from here about is in normal GPT, in normal cloud or in coding, you can build a skill and

20:11 Play from here they both have skill generators in them.

20:14 Play from here So you go to customize in cloud.

20:17 Play from here I forget exactly where it is in open AI, but you can create a skill and a skill

20:21 Play from here might be something like turn this transcript into a proposal and the proposal

20:27 Play from here is going to have X, Y, and Z section.

20:30 Play from here And it needs to include these things and it needs to reference that or whatever.

20:34 Play from here And as opposed to saving that as a prompt, because there was a long period of

20:37 Play from here time when we were all saving prompts, you'd like have a document somewhere

20:41 Play from here that had all of these prompts.

20:42 Play from here You called it a prompt library and you grabbed it.

20:45 Play from here Well, with a skill, it's all of the data that's sitting behind or the information

20:50 Play from here that's sitting behind the skill.

20:51 Play from here All you need to do is type forward slash RFP or proposal or whatever it is.

20:57 Play from here And it's going to reference that skill and all the information you have.

21:01 Play from here So just to be clear, that's still just text.

21:04 Play from here It's still just text, but there are a couple of different

21:06 Play from here reasons why you do it this way.

21:08 Play from here And what Eli is saying is that he's using a whisper flow, super whisper, and all of

21:15 Play from here them have something similar here, which are verbal triggers that when you say a

21:19 Play from here certain thing, in my case, I have one for paced morning briefing and it's a prompt

21:25 Play from here that's like crazy long, it's like 900 lines long or something like that, that I

21:30 Play from here dump into my daily flow that enriches a bunch of things.

21:34 Play from here But in normal speech, I'm never going to say paced morning briefing.

21:39 Play from here Like, so it's a kind of a safe term to use.

21:42 Play from here But when I say it using whisper flow, it dumps in all 900 lines

21:46 Play from here of the prompt right there.

21:47 Play from here Okay.

21:48 Play from here You could also do the same thing.

21:50 Play from here People like us have been doing this for years on Mac, you have keyboard shortcuts.

21:54 Play from here So in my case, I would create a keyboard shortcut.

21:57 Play from here That's like underscore morning briefing.

21:59 Play from here And if I type underscore morning briefing, it would auto complete all of that.

22:03 Play from here So there are basically many ways of doing this.

22:06 Play from here You ask, why do you use the actual skills?

22:10 Play from here Number one, I'll correct what you said a minute ago, that the slash

22:13 Play from here goal skill is actually very complicated.

22:16 Play from here So it's, it's kind of a trigger as opposed to being like a

22:20 Play from here skill that's in the rest of these.

22:22 Play from here So that one is one that you need to call as opposed to trying to

22:25 Play from here replicate its, its underlying log logic.

22:28 Play from here But the rest of them, one in both cloud and GPT, you can share them.

22:33 Play from here So like a custom GPT, you could share with your whole customer service team.

22:38 Play from here You can share a skill as well so that everybody on the team can do forward

22:42 Play from here slash angry customer or, you know, whatever the heck the skill is.

22:46 Play from here That, that, that, that deals with that.

22:48 Play from here So that's a really important organizational use.

22:51 Play from here And if it's all locked up in your whisper flow, you can actually share

22:55 Play from here snippets, the whisper flow, and you can do like kind of neat stuff there as well.

22:58 Play from here So if you, I don't think I've actually met an organization that's unified

23:03 Play from here their whisper flow, but that's a thing.

23:04 Play from here There's like a team function in it.

23:06 Play from here I know.

23:06 Play from here And it's, it's kind of wild to think that more organizations haven't already,

23:11 Play from here because it's such a productivity unlock to be using that tool.

23:14 Play from here Yeah.

23:14 Play from here Like, but just like everybody uses it individually.

23:16 Play from here It's sort of funny.

23:17 Play from here It like, I, I think about that.

23:19 Play from here I think half my team or more, probably everybody's using it.

23:22 Play from here Yeah.

23:22 Play from here And we haven't linked together in a way that we could, we could like start

23:27 Play from here managing it, but of course they want to charge more for it.

23:29 Play from here And I won't lie on whisper flow.

23:31 Play from here I, I totally huge respect to them for getting to market and

23:35 Play from here having something so powerful.

23:37 Play from here But this is totally something that's going to vanish as soon as like, like

23:41 Play from here Apple's like, yeah, okay, we're going to make Siri better.

23:44 Play from here And anyway, business, business question.

23:46 Play from here The other place where the skills are really important are baked

23:51 Play from here right into your instructions.

23:53 Play from here So when we talk in a more techie way about like our clod.md, which is the

24:00 Play from here guidance document that sits behind a clod coding projects that goes into

24:05 Play from here clod code or an agents.md, which is, is how the other platforms refer to this.

24:11 Play from here This is the markdown file that has all the guidance docs stuff about your

24:15 Play from here project.

24:16 Play from here And in my case, I have about 10 different skills that are built right in there.

24:21 Play from here So.

24:22 Play from here It there's a skill, one of my team built called naysayer and it's great.

24:26 Play from here It's the one that like goes back and reviews code and it basically the

24:30 Play from here clod.md file says once you're at this stage of development, invoke naysayer

24:36 Play from here such that naysayer does its own thing and loops back and that is a much more

24:42 Play from here efficient use of space because naysayer and it's very long prompt are only being

24:48 Play from here invoked in certain conditions as opposed to being invoked every time the system

24:54 Play from here wants to lead that read that clod.md file.

24:57 Play from here So that's, I know that sounds slightly complicated and building the skills.

25:01 Play from here You need to, you need to understand what you're doing as far as the skills you're

25:04 Play from here building, but there are a lot of, there are a lot of skill libraries out there

25:08 Play from here where people have built really novel skills to do things, probably nefarious

25:12 Play from here things too.

25:13 Play from here So, you know, make sure you're not just a lot of wacky stuff out there.

25:16 Play from here Yeah.

25:17 Play from here The naysayer skill is a skill to suck all your bank account and credit card numbers.

25:22 Play from here Password locker.

25:25 Play from here It's a haiku skill, but it also accesses, you know, your drive and does a bunch of

25:31 Play from here other stuff.

25:32 Play from here So yeah.

25:33 Play from here User beware, but there are a lot of skill libraries out there.

25:37 Play from here You don't necessarily have to create these.

25:39 Play from here You can browse around and find them, but it's an unlock.

25:42 Play from here There's also another level of it called hooks that I don't think we're going to

25:45 Play from here go into, which are like, yeah, that's, that's, that's, that's a lot.

25:50 Play from here Well, I mean, going back to, back to your box, just kind of going back to, you

25:57 Play from here know, assessing these different model releases and we don't get into all the

26:00 Play from here nuances of all the model releases because it's frankly exhausting.

26:03 Play from here So it's happening every single week at this point, but anybody who uses this

26:06 Play from here stuff a lot has often has their own little set of benchmarks, right?

26:10 Play from here To see, okay, is this better or the same or worse than what I was using before?

26:16 Play from here Because if you are bouncing around as we do between a bunch of these different

26:20 Play from here labs, then sometimes you've got to make that determination.

26:24 Play from here And I have this realization that at the end of the day, largely when I talk to

26:29 Play from here people about, you know, how are you liking this new model or this release, or do you

26:34 Play from here feel like Gemini is slipping?

26:36 Play from here Do you still use it?

26:37 Play from here It's still largely vibes, you know, like, because there you can, you can look at

26:43 Play from here the data, but based on your own workflow, it doesn't really add up to much until

26:48 Play from here you actually start using it and seeing what kind of outputs you're getting or

26:51 Play from here what kind of outputs you're not getting.

26:52 Play from here And it's just a really weird thing to think about that.

26:55 Play from here That's where we are today with this technology that we're assessing these

27:00 Play from here little machines on vibes, like this kind of anecdotal feedback of, of how it's

27:06 Play from here being presented the way that we use it at work.

27:09 Play from here And, you know, some of the weird things that I started seeing with, with, with

27:13 Play from here Claude just kind of lying to me or saying it was doing things when it wasn't doing

27:17 Play from here things, like it just seemed to be happening so much these last two weeks, like so

27:22 Play from here much, and I can't get into some of the specific things like in work because of

27:28 Play from here privacy reasons, but I use AI all the time for personal reasons.

27:32 Play from here And, you know, we're both fans of the bicycles and I've got this electronic

27:36 Play from here drivetrain that is so rock solid that I never really learned how to adjust it.

27:40 Play from here Cause it never needs adjusting until it didn't work.

27:43 Play from here I'm like, Oh, great.

27:44 Play from here I'm just going to talk to Claude about this and I'll start and be like, Hey, this

27:47 Play from here is what's happening between these two gears.

27:49 Play from here And it gave me this whole sequence of instructions and things to do.

27:53 Play from here And it just made things markedly worse at the end of it.

27:56 Play from here Everything got worse.

27:58 Play from here It's like, gosh, that's so strange.

27:59 Play from here And then I'm like, I'm just going to go old school and look at a YouTube

28:01 Play from here video for this specific adjustment.

28:03 Play from here And it was just so far off.

28:05 Play from here Anything Claude told me that I then went back to the chat and said, I feel like

28:09 Play from here you just made that whole thing up.

28:10 Play from here Like none of this is real.

28:11 Play from here And it just was like, yeah, you caught me.

28:13 Play from here I just didn't even check.

28:14 Play from here Like I just basically invented this whole thing just right off the cuff,

28:19 Play from here you know, and ran you through it.

28:20 Play from here And I'm like, wow, that's wild.

28:22 Play from here Like that's absolutely wild.

28:24 Play from here So you say you're running your, your, your life on vibes and you know, I

28:27 Play from here won't lie that my organization is also sort of doing the same of like, I get

28:31 Play from here the feel of this and you know, we definitely understand the, you know,

28:36 Play from here the risks here and keeping humans in the loop to review output and whatnot.

28:40 Play from here But for you go up the chain a little bit to organizations that are, let's

28:46 Play from here just say subject to litigation.

28:48 Play from here And this becomes more than existential for them.

28:51 Play from here This becomes like a giant issue for them to resolve.

28:56 Play from here And those organizations are putting governance structures in place that do

29:01 Play from here prompt logging and response logging and response assessment, and they're

29:06 Play from here looking for prompt drift and actually they're doing things, I wish I could

29:11 Play from here cite who this was, but they're looking for data bleeding into outside models

29:16 Play from here on an, on an enterprise level, like implying.

29:19 Play from here You have leakage in some of your systems.

29:22 Play from here So they're putting like small pieces of data that are out there.

29:24 Play from here And it appears in a trained model.

29:26 Play from here It means that it's been like worked out of your environment, which is not great.

29:31 Play from here But the challenge with things like this is they're hard and they're expensive

29:35 Play from here to maintain because they're so bespoke to the individual actions of the company.

29:40 Play from here And there are plenty of companies that are out there that are selling these

29:42 Play from here solutions and I think it's a bit of a hard row to hoe on both sides because

29:47 Play from here they have to be adaptable to what that individual company is doing.

29:52 Play from here And then they have to morph to whatever the latest and greatest is.

29:56 Play from here So when the model switched over from 4.7 to 4.8, your backend API stayed on 4.7

30:04 Play from here or did it, and did it actually, I think, actually change such that you can,

30:10 Play from here you can actually flag when that happens.

30:12 Play from here So this is happening.

30:14 Play from here And I think that as the industry continues to mature, this is going to

30:17 Play from here feel much more standard such that there is, there's, there's measurement

30:22 Play from here between these models, like, you know, the case of the, the, the electronic

30:27 Play from here drive train, it's probably because of scarcity of training data.

30:30 Play from here Like there's a bunch of reasons why that could be, but yeah.

30:33 Play from here Yeah.

30:33 Play from here But I mean, it was, I think it was just too closely pitched

30:36 Play from here to other just really wacky things.

30:38 Play from here And I mean, to your point, exactly.

30:39 Play from here It's why deploying this inside of anybody's business, let alone a big

30:45 Play from here enterprise company, can't just end with, and here's your license guys, have fun.

30:49 Play from here Right.

30:50 Play from here It's, it's a, it's a little bit, it goes a little bit deeper than

30:53 Play from here that when the stakes are higher.

30:54 Play from here You know, we talk about our chief of staff that we love, our chief of

30:57 Play from here staff agents that will check our email and our calendar availability and

31:01 Play from here give us this nice morning briefing.

31:02 Play from here And I really enjoy having that.

31:04 Play from here And part of that is looking up local weather because if weather looks good

31:08 Play from here and there's a window on my calendar, I want it to book that window.

31:12 Play from here And so I don't get any calls so I can go out and go for a run

31:14 Play from here or ride my bike or something.

31:15 Play from here Yesterday I noticed that the weather it gave me was like 15 degrees higher

31:20 Play from here for the high than what my weather, which is on the phone showed.

31:23 Play from here I'm like, that's a really hot day.

31:25 Play from here So I just, Hey, are you, you, you sure about that?

31:28 Play from here I said, let me check.

31:29 Play from here I said, no, I stand corrected.

31:31 Play from here You know, it's going to be 15 degrees cooler.

31:33 Play from here I said, well, how did you come up with that?

31:35 Play from here I said, well, I just went to AccuWeather and I looked at a historical

31:39 Play from here average and I gave you that number.

31:41 Play from here I'm like, but the instructions say to go to a completely different

31:44 Play from here weather platform to get the number for that day, the actual forecast.

31:48 Play from here Why'd you ignore the instructions?

31:50 Play from here Like, yeah, you're, you're, you're right.

31:52 Play from here I shouldn't have done that.

31:53 Play from here You know, so that was just stuff that I wasn't seeing, let's

31:56 Play from here call it a month ago, ever.

31:58 Play from here And I'm seeing it more and more.

32:00 Play from here So it, it, it just kind of makes you wonder behind the scenes, like what

32:04 Play from here levers are getting pulled, what knobs are getting turned that then result

32:08 Play from here in something like that happening.

32:10 Play from here And I mean, to some degree, all of the models just do spooky stuff.

32:14 Play from here But when you start seeing a bunch of outputs happening, we're like,

32:17 Play from here this is just acting different.

32:19 Play from here Another one, Kevin, you'll love this one.

32:21 Play from here I, at some point got so frustrated with something that it just couldn't do.

32:24 Play from here And you know, as we do, we yell and scream into our microphones with the

32:28 Play from here dictation and like, I'm like, man, like I'm going to unsubscribe, I'm going to

32:33 Play from here go to this other model, I can't stand this anymore, and it picked up like some

32:37 Play from here sense of suicidality in my voice or something, because for the entire

32:42 Play from here upcoming week, it kept asking me, are you, so are you okay, you know, and

32:47 Play from here giving me suicide hotline numbers.

32:49 Play from here I'm like, I'm not going to get one myself.

32:51 Play from here I was just frustrated over and over and over again, like just weird, weird stuff.

32:56 Play from here So anyways, yeah, yeah.

32:58 Play from here I, I've just found it really interesting forward slash frustrating.

33:03 Play from here It's some of the changes that kind of come without explanation and how we're in this

33:07 Play from here time of it's all moving so rapidly and fluidly that it's, it's, it's really

33:13 Play from here difficult to kind of keep up with and know what direction to pick as far as,

33:18 Play from here you know, we can't all use all the models all the time, you know, especially at work.

33:22 Play from here You've got to kind of pick something and go deeper in it or else it just

33:25 Play from here becomes too messy and too complex to share with other people, especially.

33:28 Play from here So it's also, it's also important to remember that hallucinations

33:31 Play from here are both a feature and a bug.

33:33 Play from here That what you're describing are the, the, like the, the

33:37 Play from here throes of a probabilistic system.

33:39 Play from here And the, the, we used to talk all the way back in the dim days of 2023, we used to

33:44 Play from here talk about temperature in prompts.

33:47 Play from here Remember that?

33:48 Play from here And the temperature in a prompt, it still exists, is it's gauging on a, on a

33:53 Play from here basically a zero to 10 scale of how cold it is, as in how deterministic,

33:59 Play from here a zero temperature prompt it tries.

34:03 Play from here It's going to snap to the highest correlation.

34:06 Play from here Whereas a 10 has it like on psychedelics and it's just like,

34:10 Play from here Oh, it's really, it's just making stuff up.

34:13 Play from here Like it makes up words even, you know, like words that don't exist or yeah.

34:17 Play from here And it's like model poetry from an alien.

34:20 Play from here Yeah.

34:20 Play from here Model, model weights play into how that works of what is going to be the

34:25 Play from here highest correlation output, et cetera.

34:27 Play from here And I think that, that we're experimenting, we're, we're seeing some

34:31 Play from here swings around with, with what that means.

34:34 Play from here And there's probably some economizing going on.

34:37 Play from here I don't have any real insight into this, but, but the right way to validate

34:43 Play from here output is to run prompt on prompt on prompt.

34:46 Play from here And that just gets really expensive to do.

34:48 Play from here When you look at thinking modes of all of these LLMs, that's what they're doing.

34:53 Play from here They're basically thinking about it by running a prompt on top of it in the path

34:58 Play from here again, and if you're behind the scenes and you're resource constrained, it

35:02 Play from here wouldn't surprise me if the dial for how much it's going to think has been dialed

35:08 Play from here back a little bit such that it's getting a little bit faster and looser and not

35:13 Play from here catching the things that are, that are coming to your attention.

35:15 Play from here Again, I don't know that for a fact, but it could make sense, but this is just a

35:20 Play from here point in time that, you know, again, like Sonnet, I think Sonnet's a great, so

35:25 Play from here Sonnet is a, one of the cloud models.

35:26 Play from here We were talking about it earlier.

35:28 Play from here It's very cost-effective.

35:29 Play from here It scores really, really well.

35:31 Play from here It's faster than Opus.

35:32 Play from here It's way cheaper than Opus, et cetera.

35:34 Play from here It's Sonnet 4.6, I think is, or have they actually gone to 4.7?

35:38 Play from here Anyway, it doesn't matter.

35:38 Play from here It is, if that model continues to get cheaper, that will be a check model for

35:44 Play from here Opus 5 or whatever it is that will be part of that thinking process behind the scenes.

35:50 Play from here So as those kind of subordinate or lower end models keep like incrementing

35:55 Play from here up, these things will get back.

35:57 Play from here Yeah.

35:57 Play from here Yeah.

35:58 Play from here Well, I, I did get frustrated enough in the last two weeks that I started cheating

36:02 Play from here on cloud and going back to Gemini, went back to GPT and it's funny.

36:08 Play from here So just because GPT does have so much rolled up into one platform, like image

36:13 Play from here generation and the custom GPTs and just all the other things that you can do.

36:18 Play from here I thought, is this where I should be going?

36:21 Play from here Like, is this the horse I should bet on?

36:22 Play from here And I'll say one thing like out of the gates, blazing fast, like so much faster

36:27 Play from here than last time I had really gone deep on it, really, really fast.

36:30 Play from here And so I decided to let it defend itself.

36:33 Play from here Right.

36:34 Play from here And so what I did is I dumped in a bunch of my instruction MD files into it.

36:39 Play from here So these are my common use cases.

36:41 Play from here These are the things I do every day.

36:43 Play from here These are the outputs I'm looking for.

36:44 Play from here And said, give me a side-by-side comparison using as much recent data as you can.

36:48 Play from here You've got to go deep on the web because I know your training history

36:51 Play from here isn't really up to calendar date.

36:53 Play from here And, and tell me, what do you think is a better choice?

36:55 Play from here And on almost everything, it was convincing me to stick with cloud.

36:59 Play from here Like it was telling me that cloud is better than all this stuff.

37:03 Play from here I'm like, why are you giving me the answer?

37:05 Play from here I didn't want to hear like, what are you doing?

37:07 Play from here So then I did a bunch of experiments in Gemini as well.

37:10 Play from here And I think you also kind of taken a little bit of a Gemini break for the most part.

37:16 Play from here Have you gone back recently and seen like all of the UI updates on both web and mobile?

37:21 Play from here Like it is wild.

37:22 Play from here They really did a big overhaul and fixed a lot of the things that, I mean, for example,

37:28 Play from here you know, I felt like if somebody hadn't used the video generation before, you know,

37:34 Play from here you kind of had a whole look for it in a completely different place.

37:37 Play from here Like, wait, where's the website where I do that?

37:39 Play from here Everything is tucked in the sidebar now.

37:40 Play from here They're really, they've made a huge effort to unify everything and give you access to

37:45 Play from here all of the tools from one main sidebar, which just wasn't there before.

37:49 Play from here The mobile app is way cleaner and it also gives you a bunch more access.

37:53 Play from here I feel like Gemini Pro is still like really painfully slow, like slower than Opus.

37:58 Play from here Like it is just really, really slow to get outputs.

38:01 Play from here So I don't know, buddy, for now, like I just, I don't have the time or energy to go

38:06 Play from here any further and I'm just sticking with Cloud and hoping that some of this stuff

38:10 Play from here gets fixed, but they turn the dials the other way and make it stop lying to

38:13 Play from here me and stuff like that, yeah.

38:15 Play from here So the answer is I, I haven't really expanded, experimented too much

38:20 Play from here with the Gemini chat interface.

38:22 Play from here There are Gemini tools that I use.

38:24 Play from here I mentioned VO3 earlier, the ability to take a transcript.

38:28 Play from here I used that yesterday to take a, take a video, put it in Gemini and have it suck

38:34 Play from here in all of the elements of the video.

38:36 Play from here That's something that only Google is going to be able to do.

38:39 Play from here I'm working a lot with GPT mainly because I have clients who are deep and deeply

38:45 Play from here committed to GPT, which I absolutely respect because, you know, switching

38:49 Play from here gears in the middle of this like Cloud, you know, Renaissance isn't necessarily

38:54 Play from here the best idea if you've already developed faculty with the platform.

38:59 Play from here And I'll tell you that the, the it's, it's, it's super surprising how limited

39:05 Play from here GPT can be on certain things that Cloud just does great.

39:10 Play from here For instance, you know, Atlas browser, like forever ago, the, the GPT browser,

39:15 Play from here it basically doesn't do anything with forms or sites.

39:20 Play from here You can't really get a GPT plugin that, that is legit, that is going to do the

39:27 Play from here things that the Chrome plugin can do.

39:29 Play from here So a demo I frequently do is like open up a survey or something like that, and

39:35 Play from here then use Whisperflow to basically free form your response to all of the questions.

39:39 Play from here It's going to suck them in.

39:41 Play from here It's going to populate them all.

39:43 Play from here And the form is just being filmed in front, filled in front of you, or

39:46 Play from here often I'll put up a, like an IRS.

39:48 Play from here Using, using Cloud's Chrome extension?

39:50 Play from here Yep.

39:50 Play from here Using Cloud's Chrome extension.

39:51 Play from here And, and co-work?

39:52 Play from here Yep.

39:53 Play from here And it works great.

39:54 Play from here I'm not, not using co-work.

39:55 Play from here Technically it's, there's semantics there, but I'm using the, the Cloud Chrome

40:00 Play from here extension, but the GPT, there is no GPT extension.

40:03 Play from here Oh, gotcha.

40:04 Play from here You're just using the chat in the extension in the sidebar.

40:07 Play from here Yes, but it also, it also, in fact, one of them just popped up.

40:10 Play from here It's funny.

40:11 Play from here So, so that's probably a co-work item that, that popped up.

40:14 Play from here It's doing something in LinkedIn on my behalf.

40:16 Play from here It's just looking for something.

40:18 Play from here It's not doing sketchy things like your various agents have done.

40:21 Play from here Right.

40:22 Play from here But a word of warning, buddy, they're deploying so much anti-bot and anti-AI

40:28 Play from here detection right now that even, even running the extension while you have

40:33 Play from here LinkedIn open is setting off flags.

40:35 Play from here That's what's happening right now.

40:36 Play from here It's doing a search, it's doing a search.

40:38 Play from here It's not doing a right.

40:39 Play from here So anyway, I, I think I'm okay.

40:42 Play from here I'm not, I'm not, I'm not too worried about it.

40:44 Play from here So the other places where, where GPT has really been frustrating is it's not

40:50 Play from here well integrated into the Microsoft world that I am, I can't believe that I'm so

40:55 Play from here excited about Microsoft and it's a result of the cloud integration with Microsoft.

41:01 Play from here And how, which to be fair was just like a month ago, right?

41:04 Play from here It was just like a month ago, but it's amazing.

41:06 Play from here Like it absolutely works as far as it suddenly, all of the promise of

41:12 Play from here co-pilot, forget co-pilot, just use cloud and be able to work between apps.

41:17 Play from here But GPT is only available in, in Word or in Excel and in PowerPoint.

41:24 Play from here So you can do some financial analysis type stuff by opening up GPT as an add-on in

41:30 Play from here Excel, and you might be able to do some sort of work within PowerPoint, but it's

41:35 Play from here so much more limiting than being able to have the Outlook connection and have

41:40 Play from here the Word connection and et cetera.

41:43 Play from here And I almost like pause as I'm, I'm trying to get into a workflow that I just can't

41:48 Play from here do the things in GPT that I want to do that I can do pretty easily in cloud at

41:54 Play from here the moment, plus clouds native, I know I'm a fan boy, but, and cloud definitely

41:59 Play from here has issues as far as stability and access and costs and the, and et cetera.

42:04 Play from here But the ability to put out just a beautiful, well-formatted document,

42:08 Play from here like directly in cloud, like they're gorgeous.

42:11 Play from here And then move that over to Word and make it a template and be able

42:16 Play from here to interact with it via an LLM.

42:19 Play from here That stuff's really, really cool.

42:20 Play from here So it works really well.

42:21 Play from here I mean, I'm doing the same thing, but with Google Docs, because I'm

42:24 Play from here not in the Microsoft world anymore.

42:25 Play from here But I, I go to Google Docs.

42:27 Play from here Same thing, same thing applies.

42:28 Play from here But it doesn't really work that well.

42:30 Play from here Like I, I, what's the difference?

42:32 Play from here Why not?

42:32 Play from here Like the formatting?

42:33 Play from here It just, no, it's junk compared to, I'm telling you, it is night and day

42:39 Play from here between using Gemini in Google and the things that it can do and

42:44 Play from here what cloud can do in Microsoft.

42:46 Play from here And I feel like, but, but, but cloud and Google ironically play together

42:50 Play from here better than Gemini and Google Docs.

42:53 Play from here And why am I missing something?

42:54 Play from here Is there a cloud plugin to, to Google Workspace that I didn't know about?

42:59 Play from here Yeah.

42:59 Play from here For drive.

43:00 Play from here Like right in the, right in the, okay.

43:02 Play from here Caveat.

43:03 Play from here So say that you have just like a big CSV file or a text file or MD file or

43:10 Play from here something you want to dump in and give it some instructions and say,

43:13 Play from here make me a Google sheet, right?

43:15 Play from here It'll do it and it'll do it amazingly.

43:18 Play from here And it will do it better than Gemini, funny enough.

43:20 Play from here But if you want to then say, Oh, you know what?

43:23 Play from here Add this column after to the right of column G, it can't do it.

43:28 Play from here It can't actually go back and edit it yet.

43:30 Play from here This has got to change.

43:32 Play from here But you can have that amendment and then it'll spin up an entirely new sheet with

43:36 Play from here all the previous data and the new things that you wanted for the amendment, right?

43:40 Play from here So it can create all these new sheets or docs, but it can't

43:44 Play from here go in and actually edit them.

43:45 Play from here And when we were talking about, you know, taking our instructions for all of our

43:50 Play from here projects and making them portable and you know, where do you put those so that it

43:54 Play from here can access them from any web-based model?

43:56 Play from here And you and I talked about like, Oh, wow.

43:58 Play from here The ClickUp connection just works.

43:59 Play from here really great. ClickUp docs do that really well. The reason why I ended up there to begin

44:04 Play from here with, I tried to do that with the drive connector and it would write the instructions amazingly,

44:10 Play from here but every time I tried to update the instructions, that's where I hit the wall. It would have

44:14 Play from here to create a whole new doc instead of editing it. So that's, and I think that's just the

44:19 Play from here limitation with their MCP, like just with what they have access to. So either it's going

44:24 Play from here to change or it won't change and they just want to keep it like in the Google ecosystem

44:29 Play from here and just have that be something that Gemini can eventually do. Right now, Gemini doesn't

44:33 Play from here do as well of a job as cloud does with all those Google workspace connectors. So that's

44:38 Play from here where I'm at.

44:39 Play from here Yeah. So changing gears just a little bit, I think we referenced this earlier, but just

44:44 Play from here a quick take on your actual day to day, like tech stack. We talked a lot just now about

44:51 Play from here different models and obviously we're using the browser, we're bouncing around, we're

44:54 Play from here doing the things as far as like personal productivity and connectors. But when we talk

45:00 Play from here about like building workflows and things, a year ago we would have been obsessed with

45:04 Play from here N8n or Zapier or Make or something like that. And I can't tell you the last time I've been

45:09 Play from here in any of those. What is your go-to tech stack right now?

45:13 Play from here Yeah. My go-to tech stack. Smaller. It's way smaller. And I don't know if we'll be able

45:18 Play from here to get into all of this. We don't have a ton of time left, but you know, I've been in the

45:22 Play from here agency world for 15 years. I started off building enterprise software, mobile apps,

45:27 Play from here stuff like that, which that world was rapidly getting eroded by things like cloud code and

45:33 Play from here codex to the point where the agency made a huge pivot into doing the things that we talked

45:38 Play from here about on this show. So building AI tools for other companies and those types of services.

45:43 Play from here And, you know, we partner together often on those types of projects. And my tech stack

45:48 Play from here was changing basically every week at that point. And I'm now, I've completely exited.

45:54 Play from here I'm still acting as an advisor to the agency, but I've sold my shares. I'm doing something

45:58 Play from here completely new and honest truth, Kevin, and this is why this is going to be our last episode

46:04 Play from here together. I'm using a lot less AI in general, or at least trying to for the things that

46:10 Play from here I'm spending more time on right now, which is a lot of writing. I think we're kind of

46:14 Play from here turning a corner with writing where there is so much AI slop being generated out there

46:20 Play from here right now, whether it is short form social media stuff or long form blogs, essays, whatever

46:27 Play from here it is that I think we're on the precipice of a renaissance of real writing again, where

46:32 Play from here AI might be an assistant to help you build outlines or come out with new directions of

46:37 Play from here thinking or ideas. But I'm doing all of my writing just old school again. Like I'm actually

46:42 Play from here just writing everything. I'm still requiring a lot of image generation and charts and graphs

46:46 Play from here and things like that. And because I haven't like went all the way back into the open AI

46:52 Play from here ecosystem, Nanobanana for me is like good enough. Like I've seen all the benchmark tests.

46:56 Play from here I know that GPT 5.6 image gen is better with small amounts of text and catching typos and

47:02 Play from here stuff like that. But right now for most of what I do, Nanobanana is good enough. So right

47:06 Play from here now it's, I still have all the models, but it's mostly cloud. ClickUp is kind of my central

47:12 Play from here database for holding all of the instructions. And I even use it as a CRM. And then I'm still

47:18 Play from here in the world of Google mostly for image generation at this point. And of course, WhisperFlow

47:23 Play from here all day long. I don't think I could ever go back otherwise. I was going to actually say

47:28 Play from here Kevin, there was a couple of times this week. I don't know if I'm sure you experienced it.

47:31 Play from here WhisperFlow is having issues. And what ends up happening, right? Is that you're talking

47:35 Play from here and nothing happens and you look at the bottom of your screen and there's this little pinwheel

47:38 Play from here saying it's taking longer than expected. And sometimes longer than expected means you could

47:43 Play from here actually type faster and get, just get the thing in there. Oh my gosh. Having to go back to typing.

47:48 Play from here I was like, I don't think I could ever do this. Like this is so painful. It was so awful to have

47:54 Play from here those issues and they're largely resolved right now. But I was like, oh, this is, this is a rough

47:59 Play from here week all the way around. So, you know, that I will continue to use forever and ever. I'm trying

48:05 Play from here to think what else, but it's like, I'm actually trying to shrink my tech stack right now.

48:09 Play from here Okay. Yeah. Okay. That's fair. Yeah. So, yeah. I mean, I would, I guess I'll share my tech stack

48:16 Play from here in a future episode at some point more on the, the coding and development side, as far as front

48:21 Play from here end and backend and things like that. I've been having some adventures between Cloudflare versus

48:27 Play from here Supabase and, you know, there, there are pluses and minuses to that. And, you know, it sort of,

48:33 Play from here it sort of skirts around with what is too complicated to talk about, but I think it's

48:37 Play from here fair for people to understand the types of things that people like us are actually using day to day.

48:43 Play from here Like there were always edge cases when you're going to have to use something else.

48:46 Play from here Yeah. Having walked away from the agency, my day to day is now like, I'm just not doing any coding

48:51 Play from here stuff for a while. I'm going to get back to it because there's some clinician tools that I want

48:54 Play from here to build, but right now I've got like bigger fish to fry. So. So, so tell us what you're going to do

48:59 Play from here with yourself and like what your new, new passion and project is and where people are going to be

49:05 Play from here able to find you. Yeah. Yeah. So the honest truth is that agency life is hard. Like being the founder

49:12 Play from here of an agency is hard. You can ask anybody running an agency. I don't care what kind of agency it is.

49:17 Play from here If it's software dev, performance marketing, PR, you name it. Like it's, it's, it's tough. And it's

49:22 Play from here a lot of 60 to 80 hour work weeks. I've two of my kids are elementary school aged. And so first and

49:29 Play from here foremost, I'm just spending more time with them. That's what I'm doing with myself. And nobody

49:32 Play from here will find me anywhere with that. I don't even post on social media, but my wife has been a licensed

49:38 Play from here therapist for 14 years and you know, her limitation for how many patients she could see has been,

49:44 Play from here well, how many hours are available where I'm not watching kids. And we made a decision together

49:50 Play from here when we had kids that we weren't going to outsource that whole parenting thing. We were

49:53 Play from here just kind of of the thought of, well, if we're going to have kids, we actually want to be the

49:56 Play from here ones parenting our kids. And that means one of us is going to have to sacrifice more than the other.

50:01 Play from here And that's, you know, that's, that's, that's been me for a long time. Like I've just been

50:06 Play from here putting in the hours, talking to AI all day. And I decided that it's, it's time it's all going by

50:12 Play from here too fast. So I'm not only spending more time with the kids, but I'm helping her grow her practice.

50:17 Play from here And she has a specialty in pain psychology. So she's been doing it for 14 years. She did all her

50:24 Play from here clinician hours at the pain psychology center in Los Angeles under this guy named Alan Gordon,

50:29 Play from here who's been called kind of like the back pain Jesus, just curing people of their back pain.

50:33 Play from here And so I actually, to be better informed of the whole space, I've actually gone through

50:39 Play from here pain reprocessing therapy training myself. I had a life-changing experience where I had chronic

50:44 Play from here back pain for over three years, and then use this modality to be completely free of it.

50:49 Play from here And there's just all this recent research that has come out in the last five years,

50:53 Play from here showing that it's really also just this idea of neuroplasticity is underlying cause of so much

50:59 Play from here treatment resistant anxiety and depression and everything else. So what's going on with AI right

51:04 Play from here now and the billions that are being pumped into building companions for people or therapists

51:10 Play from here for people that are just AI? It's my prediction. My wild prediction is that we're going to see

51:17 Play from here a loneliness epidemic that makes everything that happened with social media in the last 20 years,

51:21 Play from here just pale in comparison with what's going to be coming up. And so I'm all in on mental health,

51:26 Play from here Kevin, like I'm all in on mental health. So I'm going to hang out with my kids a lot more. I'm

51:31 Play from here going to help her grow this practice. We're onboarding some clinicians. It's called the

51:34 Play from here wise mind group. And so our specialty is pain reprocessing therapy. And then this other

51:39 Play from here modality called radically open dialectical behavior therapy, which kind of also gets to

51:44 Play from here more of the root instead of just the expressions of anxiety and depression. So that's what I'm,

51:50 Play from here that's what I'm doing. And then also doing some executive coaching, still doing some AI mentoring

51:56 Play from here for some people, because listen, I'm a, I'm a geek. I've been in tech for 28 years. It's not

52:02 Play from here going to just be purged out of me. I'm going to keep up on all of this stuff. And we're going to

52:06 Play from here keep talking about all the things that we're using and building, I'm sure. But yeah, it's

52:09 Play from here going to be a little more secondary in my life now. And meanwhile, I'm going to have to find a

52:13 Play from here new co-host that looks like you, because if they don't actually look like you, I'm going to have

52:17 Play from here to redo all of the graphics and everything else. And even though I have AI, that seems like a lift

52:22 Play from here for me. So just make sure it's somebody that in those graphics, you're comfortable with embracing

52:26 Play from here in the, you have to, I would highly recommend changing all of that because like we said,

52:33 Play from here you know, this is the worst it's ever going to be. And whatever generation was used at that point

52:38 Play from here in time is definitely worse than what's available right now. So over the next few weeks, I'm going

52:46 Play from here to have a couple of other co-hosts on. We're going to rotate some through it. We're going to rotate

52:51 Play from here this up through it. We'll put it to the audience. It's going to have some guests. I have some

52:55 Play from here guests for you too. And then I can come back as a guest, which would be so fun. And I can talk

53:01 Play from here about, you know, AI replacement dysfunction, right? Which is going to be making it into the

53:07 Play from here DSM. This is so wild that they're predicting that just the disruption in the workplace with people's

53:14 Play from here identity being wrapped up in, this is the thing I got so good at doing, or this is the thing I got

53:19 Play from here my master's degree in and now, Hey, this model can just do it better than me, right? Like I'm not as

53:24 Play from here good as the machine. And that creates a lot of just disruption with identity and a whole cascading

53:31 Play from here effect of problems. So that or AI induced loneliness is another thing that is coming in. So

53:36 Play from here lots of, lots of intersection with mental health and AI. And we've talked about AI work density,

53:41 Play from here right? Or burnout or the FOMO. Like I didn't deploy my 17 agents before I went to bed tonight.

53:47 Play from here And I'm falling behind because all my competitors are going to be doing so much better than me.

53:51 Play from here Right? These are all real things that at the end of the day, we just, we want a life a little more

53:56 Play from here free of suffering. We want to sleep well at night. We want to be happy. And that's just going to be

54:00 Play from here my primary focus right now. And I think AI is going to be a big part of that story. So.

54:04 Play from here All right, sir. Well, it's been a, it's been a good ride and we'll look forward to having you

54:09 Play from here back on. The show isn't going anywhere. I really, I get a thrill out of doing this every week

54:15 Play from here and trying to make AI as accessible and yet useful and practical without getting platitudinous about,

54:23 Play from here you know, the latest news or the latest model. Which is, which is so challenging.

54:28 Play from here It's, you know, there are plenty of places to listen to that. So look forward to seeing you

54:33 Play from here on the flip side socially. And yeah, I guess I'm going to play the outro. You do it. It's

54:40 Play from here going to have a whole bunch of stuff about the company that you don't run anymore, but that's

54:43 Play from here cool. Go be patron anyways. It benefits me. I'm on a,

54:57 Play from here thanks for listening. If you'd like to send us a question to answer on the show, email me at

55:02 Play from here Elijah at spark six.com that is S P a R K the number six.com or Kevin at Kevin at ascend labs.ai

55:11 Play from here to learn more about AI strategy and training for your business. Visit Kevin's company,

55:16 Play from here ascend AI labs at ascend labs.ai. And to learn more about design and development of custom AI

55:23 Play from here solutions, visit my agency at spark six.com. You can also sign up for my free newsletter,

55:29 Play from here AI framework Friday by visiting spark six.com forward slash newsletter.

How to cite this episode

Credit the named speaker and AI at Work by AscendAI. Link to the episode or the passage you quoted.

“Ep #29 When Models Lie and You’re Still Paying for the Tokens.” AI at Work by AscendAI, 2026-06-04. https://ascendlabs.ai/podcast/ep-29-when-models-lie-and-youre-still-paying-for-the-tokens/

Recent episodes

View all episodes
×
AI Solutions Graphic
Free Resource
Get Started Now