Lean Product Management - Sept/Oct 2025

So, There we go.

So welcome to day 4 in recording here, which you will find on clarity flow again later, where Aaron knows how to find it and and we can help get it later in Matthew 2.

but yeah, post class email with links to mural, linked to clarity flow, um, I'm gonna do a PDF of mural, uh, the mural board, uh, and, and make sure that that's linked.

I will at least copy that to clarity flow.

And um an updated version of the deck, uh, since it's changed since the thinking of the course, uh, so all that will come your way.

Um, is there anything else you you you would like to have me send you or link to to make it easier from like the, the course and any of the materials or anything that we talked about? I think those are the major things, right, recording, yeah, and output of everything we've covered the material we've generated, and then.

Yeah, the side decks, I think.

Unless there's something else we've been missing out on.

No, I don't, I don't think so, but you actually triggered something else for me.

Um, I will get, uh, links or PFs for the um lean product canvas deck.

It it's an easy thing to find, but if you've got it quickly available to you, that's just another good one to have.

Um, but that's all I can think of right now.

I just, I'm trying to think if there's any other concepts that we talked about that I should like to.

You can always ask, right? You can always reach out to me and say like, hey, you got something for this? Is there, uh, you know, oh, I remember I was talking about this, I was watching the video, you got a link for that, I will help you out there, right? So don't worry about that.

Um, yeah, yeah, like, in re-engaging with my students, always a pleasurable experience for me.

I'm like, yeah, cool, you're doing stuff, this is good.

So that's cool.

Um, the other thing that, uh, we love, so you can, you can go through this, uh, at some point I'll try to make a little time in the class later on, but, um, if you can fill it out it'd be great.

I just put in the Zoom chat a link to a um a class survey.

Uh, and so this helps, helps me, this helps, um, sense and respond, uh, kind of learn what, what are we doing well, what do we need to work on, uh, where else should we go with this material? Um, is there anything else as well that you would have to say about the class, um, that others might want to know? Um, and that kind of thing.

So, um, that tally link that just went in there, it's short, there's like a couple of NPS scale items with some opportunity for feedback, uh, on those pieces, um, and any other recommendations, improvements that you'd have to suggest.

So if you could go through and give us that feedback, be super, super helpful.

Um, also down at the bottom, there is a list of, uh, a place where you put your name and email for a certificate of completion.

I think I was messaging Natalia about this earlier, um.

Yeah, so that do do do do, ah, she actually wants you to fill that out in order to get your Credley badge.

So there's a certificate of completion, I think a PDF that you get and then as well, um, we've worked with or we've uh connected with Credley so you can get a little badge on the post and let everyone know, like I've done this thing.

OK, thanks.

Um, all right, one last bitches, I, uh, I want to talk with you more in the future.

I know you said like you'd love to get more people engaged in this kind of thing.

Um, lean product management would maybe be an interesting class there.

Uh, the product discovery course could even be good, right, just as a note.

We can catch up on what more of what it's about, but this class is is um really about being a product manager, right, and leading into discovery.

The product discovery course is actually more for teams that would be doing the work all the time and so still cover a lot of these similar concepts, but I think like we don't, we don't get it so much in a strategy.

It's kind of more assumed you'd have one.

there's still going into proto persona, still going into experiments, still thinking about how you incorporate this into your workflow.

And like, and going through that whole lean product canvas of like popping in all those pieces.

So if that feels more appropriate for your team, um, You know, that's cool too, but let's, let's maybe find a time, um, when, when things are a little less crazy for you to chat about them further.

I, I, I was actually thinking about the course on OK hours if you do anything with.

You know, a deep dive into OKRs and how to do that, um, that might be useful for the rest of the team, even if they have no no idea about anything else.

Um, that might be useful as a tool.

Um, and, and people who are.

People who take the product management course might be able to guide the rest of the folks with just the OK at least that's, that's, that's the idea I had in mind.

I don't know if it's uh if it's valid.

I wanted to chat with you about that too.

So we can, we can talk about it offline.

It is, and I'm working with, um, so two things.

I mean, I can always talk with you about a private class offering if that's something you want to do, um, but I am scheduling some public OKR classes um for November, December time frame, um, and I'm looking into 2026.

I just haven't looked at my calendar for that yet.

Um, those, it's, uh, the course is about half the size.

So, um, with that, I'm gonna do 23 hour blocks, um, and I'm gonna schedule some that are like a week apart, but I'm also gonna schedule some that are like do your first class on Monday and the second class is Wednesday, so much more, yeah, yeah, more compressed time box, you know, time frame right there.

Um, so yeah, it, it is like a one day in person or like a, you know, that Jeff and Josh do.

22 hour classes and then offload they say about half the material.

I, I just put an extra hour for one if we get into questions, topics, things like that, we have the time and if again if we don't need it, we don't need it, we just end early.

It's never, no one's complained to me ever, ever ending a class early for the day.

Is this a, is this a uh are you going to publish this or is this more like a private thing for The ORs class.

I'm going to publish it.

Um, so, so yeah, so we have at least with Lian within Lian product management and OR's class, um, it is essentially who does what by how much.

It follows that book and patterns.

Um, that class, so some other interesting things about that class, we dive into OKRs like 3 different times, right? We go into A lot of the exercise is about outcome impact mapping.

We use that to drive um OKRs a little bit more.

We talk more about what makes a good OKR.

Um, we, uh, and we talk about it from a, a work perspective, a personal perspective, right? Um, and so you can try in different contexts and see how OKRs can apply, um, to both product and process management.

Um.

And then the class some some key differences, other things that get added.

Uh we talked about OKR cycles of review and trying to learn and adapt based off of um our OKRs and, and know when to change them.

Um, and that course at the end of the Think if modules as a um as a quiz that got built into it, um, Sensorrespond is partnering with OR mentors.

Uh, to provide an OR mentors certification, so a third party certification of your knowledge based off of the course.

So they're still working to adapt and learn about that, but it's, uh, it's a, it's a little something extra, which I think is attached to that course, which is cool.

Um, so I don't have those published yet, but I will email you when the public ones are live and we can definitely talk about the private ones too.

OK.

Cool.

The private ones might be next year, but the public ones, uh, I'm gonna do it.

I'm gonna ask Aaron if he wants to do it, uh, of the OK one especially.

Yeah, so, uh, it'll be um a little review for you in some ways, but also a refocusing, maybe a deeper dive too.

So, OK, um, cool.

Anything else you want to chat about before we Get into today's materials.

Right.

Well, let's roll.

Uh, so, working this way, that's what day 4 is going to be all about here.

Um, we're gonna get into, like I said, we have gone from talking about strategy, where we're gonna play, how we're gonna win, the main business problems that we can work on refined multiple times throughout the course, right, the objectives that we want to uh go after, the key results that will help us know, the, um, the assumptions and hypotheses that we've been Making about our work too, right? And then trying to actually run experiments and today we're gonna talk about it as well.

How do we actually start like doing this consistently, right, and incorporating this into our work.

Um, my question for you, as we get into is your experiment.

From last time.

So this is mainly gonna be and now she had to drop early before we uh before we got to most of that discussion.

I unfortunately will report and do not complete our homework in that regard.

OK.

Um, all right.

That's right, yeah.

The purpose was in my mind as we left, right, trying to validate, you know.

interviewing what we believe are representatives of a proto persona and validating the assumptions we've built in in the mural board, how many of them actually are relevant.

Yeah.

Yeah, I unfortunately I lost track of time, and so yeah, we didn't get to the interviews and I originally was thinking I could have the time to at least quickly come up with a survey of questions.

And I was kind of at that point I was like thinking more on the execution part of it and it's like, how do I mix in kind of more like the basic yes no questions and I know, right, you had emphasized making sure that, you know, having those open-ended questions that allow you to have that story, you know, allow them to tell the story.

Um, Yeah, but yeah, unfortunately I think it's anything.

OK, OK.

Um, so it happens and you actually, you, uh, you, you covered the key point, right? The whole point of the exercise is to learn quickly, right? To learn if your assumptions are correct, to learn if these people really exist, um, and to do that, um, quickly and, and cheaply in a way, right? Like the, uh, if you were to run this.

Take an hour, 2 hours, 5 hours, I don't know, something like that, and you'll learn something, and it's a really, uh, when I've done that in the past, it's a really cool experience, especially if you don't interact with your customers ever, um, which I haven't heard much about there like, but that's, you can get all this cool information in in a short period of time and tightening that feedback loop before we decide to make all these other investments and build all these other things, um, is so valuable and so powerful.

Um, now, I want to talk a little bit, uh, not, not that I'm like angry about not doing the homework, but I just want to talk explore some of the other things about, um, what caused it not to happen, right? So you just said, you know, I kind of forgot about it.

Sure, as you were talking before, you got this big thing coming up today, uh, and there was a lot of other work that was associated with it and some changes there.

Um.

So, you know, some of those are like, there's chaos that got imposed upon you.

Um, or like just the timing wasn't quite right, but what, what is there anything else that prevented you from doing that kind of experiment? I mean, I know, at least double checking um based on what was in the recording and at least referencing the original slide deck that's on I think if it.

Um, The thing the main thing was just trying to remember what the actual homework assignment was and then they.

You know, since I wasn't immediately executing it on after a previous session, you know, you know, after that weekend, it's kind of like.

I know we had to execute the experiment, but what was the exact I know we had discussed interviewing, but then it was kind of like figuring out the exact details of like, you know, what I just described.

Was it to, you know, validate the proto persona and the assumptions we made there? Was it to follow up on the questions and the different topics we had put into the mural board regarding what do we think are the key items that we need to validate? Mhm, yeah.

Yeah, that that's a good point and like how do we know when to do that? So I, I know I highlighted week one when it was all new and fresh and we're like, whoa, what's going on that the um putting some time aside, right in the, in the middle of that week to do that is important, um.

For our next group at 0.

I will, like, I think I'll try to get that information out as well earlier, right? Um, and including in a post, uh, pre-class email like, hey, get ready to schedule some time um to do this, you know, this kind of experiment because of the, then you won't forget as much, right? Um, and that's, that's just a different aspect because you might even want to schedule as you're doing your regular work.

Right? And so we'll talk a little bit more about that today too, of like, when do you schedule this discovery? When could you do it? How do you make it part of your general flow? I, I have two other things.

The one is the project management thing.

This is for my personal, if you task management.

One of those is that you, you mentioned this in the first class, you said, you know, you need to schedule this right now.

But because it was so far away, I pushed it off and with the mode we're running in right now, anything that's not uh needed in the next 20 minutes is not fire.

So I, I, I pushed it off.

That is completely on me.

Uh, the second thing was that we even yesterday we were at the, at a dealer um where we might have.

We could have, we even discussed uh interviewing one of the technicians there because we, we know that they're likely gonna be dirt bike riders if we could have interviewed them there.

Um, I don't know about Aaron, but in me, I was, I sensed.

Just as you asked me this, I, I, I, I went back to that moment.

I distinctly remember having this hesitation to just go talk to them about it.

Uh, and, and I had this thing that, oh man, I have to prepare for this.

We have to get a, we have to get a questionnaire out, we have to get the survey out, and we have to prepare a whole list of things that we need to ask them, and then we can go talk to them.

Uh, I wish that weren't the case.

I wish we could have, you know, even if it was, uh, we had, I think we had enough of their time.

We could have, we could have spoken to them about this for like 30 minutes, uh, easy, and we could have also done that with, uh, some of our colleagues.

Uh, not yesterday, but, you know, uh, earlier in the week.

Uh, there's there's some.

At least prioritization things that need to be fixed there, at least.

For me.

Yeah.

Um, you don't.

So, I'll give you an analogy.

Firefighters should spend most of their time cleaning the engine.

Um, you think of like, when there's a true fire.

You need to go fight it.

Uh, but the, but hopefully, we're actually not fighting fires all the time.

We're not like constantly in a state of chaos, um, cause that it's just that, that will wear on you over time.

So I hope, you know, for your, you said you're in this mode right now.

Um, think about.

Not related to this class at all.

My personal recommendation is, is, is think about when can you get out of that and how do you, how do you get yourself out of that? Like, what are the things that will start to trigger, we need a shift, right? Or what are the things that trigger, like, if we stay in this mode.

Bad things might happen.

Um, cause I've worked with companies like that, and I've talked with a lot of people and it's like, you just, you've seen people leave, they get burned out, they, you know, they can't, the, the quality of workshops.

Um, we don't know, we actually don't have focus on what we're working on because we're just used to respond to emergencies, um, so.

That's my suggestion is um To, you know, if you can take a little bit of time in the near future, think about how you'll get yourself into the next mode of operation.

That's fine.

So you mentioned Hesitation.

Um, and he said you, you, and then it sounded like part of that was you wanted to feel more prepared.

Yeah, definitely on my case it was.

You know, the uncertainty of like, well, I don't want to feel like I'm wasting, you know, their time, um, especially since, right, this was originally uh a dealer visit we're planning to do, um, for following up, you know, with the getting a beta version out of our application to the technician, our main user, and getting their feedback and kind of doing an AB test of the existing application versus the new one.

Um, that was our original plan, and of course, We had several roadblocks with that, so I think we're already kind of weary from travel and driving for an hour and a half up to to the location, up to the dealer, and then the fact that we forgot certain things, um, like the programming cable we needed, it's like, well, we're already so frazzled, and it's like.

OK, what's one more thing.

Mhm, yeah.

Yeah, it's um I I think it's fair, right, to force it into a situation that what didn't make sense, uh, might have been awkward, and then it adds to your stress, right, in that kind of way, um.

But the, the, the, the crazy thing is that um from my personal experience, uh, you're, you are aren't often wasting your time, uh, their time, and you, the planning, you do need to plan, but you maybe need less planning than you think, right? Sometimes just popping over and and just asking like, hey, you know, seeing somebody like run around, do you, do you commute often, you know, or like do you ride on these shows, you know, like much.

Are you just here today with a friend? OK, cool.

Well, not my target, right? And that's a key piece too is when you're starting to talk with these kinds of people and, and do these types of sessions and while I You know, the uh years ago before this class existed and before I was doing this kind of thing, I worked with groups um in my previous consultancy, uh, that did stuff like this.

They would just kind of like roam around and say like, hey, they're working on mobile apps, do you use these, like, do you use the um power tools at home? And they're like, um.

No, it's like, OK, not my person, move on, right, so finding that, OK, I, I could ask you my questions in this interview, but it doesn't make sense.

And so when you start to think of your personas and, oh, do you use power tools on? Yeah, yeah.

What, what kind of tools? like what kind of work do you do, right? Oh, you know, we got a drill or whatever, and I do a lot of woodworking.

I, oh, interesting.

Tell me more about the woodworking, right? And uh what, you know, what kind of things to me? Tell me a time that you just struggle with settings and so this was all about um.

We were building, we're helping Milwaukee Tool uh build an application that configured their tools.

It's called OneKey.

Uh, so if you've ever seen, if you ever go out and see Walky Tool and Oneey, the company I used to work for created the initial launch of that.

Um, and so it was neat just like, and why, why does anyone think about using their phone and a power tool at the same time, right? So, um, yeah, you can just, you know, sometimes chat with people and and see what's going on.

Um, I did a little bit of that for you over, over the week, um, over, over the weekend.

Um, so we talked about the company and, you know, because just because you attended the class with Jeff and Josh on Lean uh Lean AI, um, they were curious of what's going on and everything, and I asked a little bit about getting around and like, oh, do you all walk? You guys taking the subway very much? How you do, you know, do that and somebody mentioned, oh, well, you know, we have our, our motorcycle.

Oh, what, what kind of motorcycle is that? Right, um, oh, how far do you go? What do you do? And like, oh yeah, we don't actually walk very often here anymore cause it's so easy to just hop on the bike and, you know, even put the two kids on the back and zip around.

I'm like, oh, OK, interesting, right? And, you know, started asking, and some people are like, I don't speak good English, and I can't help you, but OK, so but it was just neat like I, I had a couple questions and started figuring out like, yeah, they're they're um.

Uh, they just hop on a bike and take it for short jaunts.

Um, no one talked to me about Uh, like little tours, like out to certain areas on their bikes.

They didn't do that kind of thing.

It really was in the city or within different districts and such, um, and, and they talked about like getting around and it's easy and like, and we just, you know, you can park them anywhere.

I was like, really? Yeah, like any, I mean, we just ran on the sidewalk, right, right, right on the street, like all the time.

That's, you know, just go and you don't have to worry about finding parking or what you're, you know, or how far it is a walk from the the station, right, the subway station to wherever you want to go.

You just park right in front of the thing and and you're good to go.

I'm like, really? And so I then went to the next part of my experiment, which was observational review.

Yes, there are bikes.

Everywhere.

Right, like, literally everywhere I see people on bikes and putting bikes out and about all over the place.

Yeah So I thought that that was really cool.

They're just all along the sidewalks, um, and in various other little spots and uh I think, yeah, that was my last picture of the mics there.

Um, so it was neat, like they're parking all over the place here, and I started learning something about this, which I thought was neat.

You're talking about EU commuters.

I'm in the EU.

Here are people commuting on their bikes, and there were some patterns that I noticed, right? Um, they do just park on the sidewalk.

In the evening, it looks very similar.

OK, uh, I didn't get a picture of that.

There were less bikes, but there are still a lot of bikes just out there with these like they got a little lock that kind of comes down right here, that connects to the handle.

Um, so there's a lot of them they're just sitting out there like that too, uh, which made me think a couple things like where, where do they fill them up? Where would they charge them? If this was the case in EU, right? Um, how far would they need to go? They, they can't just bring them into their apartment.

The, if so, the elevators were really tiny.

Um, there was, there was a lot of underground parking, so I didn't see what that looked like.

So I don't know how many people were moving, uh, underground at night.

Um, I also noticed that this bike here and this bike here, and one more way back over here.

Or some of the few like what I would say look like standard kind of bikes, they look like most of them had more of the scooter, more of the scooter style, yeah, yeah, and so people were riding around on a variety of scooters, all of these little boxes on the back, right, so many of them like that, uh, just sit around, go to work like she's just for work, right, right, and then got nice shoes on and section this guy in a suit.

I was like that's, that's cool.

Um, so that maybe tells you a little more like, what, what is an urban commuter in Europe expecting? From their back.

So I just thought I'd share that with you.

You know, you're so focused in your class and you can and you mentioned that I was like, I can add some context, you know, I'm not running the same experiment, but I thought it would at least be kind of neat to bring that up and see what you want.

And, and one of the key points is right, you highlighted, you know, I think at least I was approaching it from like it's a very formal structured process, but right, you're able, you're showing that, you know, just by going out into the street and having a general idea of like, oh, you know, I wanna find out about these things, just go out and do it and it doesn't have to be like, yes, I need a step by step like this is the exact list of things I need to cover, and this is, you know, the exact feedback forms I need to follow.

I think yeah, allowing for a more natural, you know, just.

Not, um, spontaneous is the word I'm looking for.

You know that, yeah.

Yeah, the, um, the, the downside of that is if you just if it's totally spon uh spontaneous, uh, the structure of your results is not as good, right? Yet, um, I I certainly was at a point where I needed structured results.

I was looking for themes.

Right, like, what kinds of bikes are people on? Where are they? How many people are actually riding on bikes? A lot, right, a lot in that area.

So like, oh, OK, cool, right? So you, you can start finding out some very quick themes just by making some basic observations and a couple quick conversations, and then use that to design more structured.

Interviews or surveys for round two, right? Because even just that very light touch proves whether you need that or not.

Yeah, right, you know, if the, if you're able to identify a theme, you know, that's opportunity to be like, OK, do I need to actually then formalize something and dive into that theme, that's something I need to now track following forward or what part of the theme is that theme tied to.

Yes, but, yeah, exactly true, right, like you've identified, yeah, there's a theme of You know, there's a heavier bias towards the type of scooters versus the traditional motorcycle we think of in like North America.

Right.

And, and what, what came out of the question for me is there's a bias towards them.

Is that what they want? And would they accept a different kind of bike, right? Cause it clearly there are some that are out there, and I walked past a couple of bike shops, and it seemed in each bike shop, I wish I got a picture of that.

There was a blind.

Right? But if there's, there's a blend, and they're not like, are they selling so many of the scooters and just a few of these, or did I just not see them or like, there was obviously gaps in the information and more things to go and discover.

Um, but, you know, observation, I'm like, OK, well, if we were to build a bike like this, whether we wanted to or not, there's clearly like a bias towards people wanting to ride something like that.

It feels familiar, and we can just off we go.

Right, um, and you might find out, you know, you could run experiments like that too, or, you know, is it just, why is, why is the market used that? Like why do they care? or you know, how many people are riding on bikes with multiple people.

If it was an adults, I didn't see it once.

If it was with kids, I saw it often.

Lots of kids riding on the backs of bikes.

Uh, one or two deep, uh, not as many as I saw in India, like, whoa, like, like there are a lot of people on bikes everywhere and, and like to, you know, at least 2, maybe 3 adults and sometimes like, you know, an adult and 3 kids all riding somewhere.

I'm like, how, how, how, I don't, I don't know.

Yeah, I remember when I was in India, my coworker and I, it's like at some point we'd see, yeah, you know, you see an entire family, like 55 people on a on a motorcycle or a scooter we're just like.

I don't, you know, safety aside, I'm just impressed they're able to pull that off.

Uh-huh.

Everyone's leaning in the right way, it's at the right time like I, it, it was impressive.

The, the thing is in India, it's, it's different because the speed of movement is also much lower, so it's not like uh no one's, no one's riding at 90 miles an hour.

It's At most 20 kilometers an hour or 25 kilometers an hour.

Yeah, when we, when we were doing 40 kilometers an hour that felt bad in India, and I was like, and it just it blew my mind when I realized later how what that speed was.

I was like, oh, OK.

Yeah, most times you said it's at fastest 25 kilometers an hour, and more like 5 to 10, right? OK, um, so a little bit of, of insights I say, just, um, get out there, talk to people.

it's amazing what you learn and um here's the other one, right? uh.

Yes, it, it feels uncomfortable, especially if you've never, if you don't do it, right? If you haven't done it before, if you're not used to it, the more you do it, the more comfort that you'll get in just having these types of conversations and just going out and talking to people.

And what what helped me with those is, um, you'll be amazed how few people get asked questions.

About like their own habits and like what they do, and for like just general, you know, anxious showing up as a as a human, curious about what they do, and seeing if you can help them in some kind of way or or change things for them in a positive light, um, they're often more than willing to provide that kind of feedback.

If you know, you were to come in like, hey, you know, um, we're from zero motorcycles and, you know, we make bikes and we're trying to figure out how we could sell more of them to you.

Yeah, please walk away, right? It's like, hey, what's the problem that you're having with bikes? And then when you, when you focus on that, people feel seen and um and they often appreciate it, and it feels good to like talk to these people that way.

OK, um.

I'm gonna move us on to the next phase, if that's OK with you guys.

Mhm.

Yeah, OK.

So, uh, let's, let's get into some tactics here.

We've done some a bunch of strategy and we're gonna get into a little bit more on, like I said, how do we do this work? How do we start making these connections, how do we incorporate this type of product discovery work, um, and, and focusing on outcomes for people into our general workflow.

Uh, so, first off, we're gonna talk a little bit about OKR based road maps.

Um, I got a question.

Do you do anything like this now? No.

OK, which is, which is why we want to do the street.

Awesome.

So, uh, there's, man, there's, uh, this reminds me of a time.

Did I tell you about uh the Gantt chart product owner insecurity? Did I tell you that story? OK.

So, um, I, I worked with a product owner once.

He was in charge of security at um uh an investment company.

And he uh and when he took over, it was like he was part of a small part of security, and he ended up taking over everything, um, because he had a good view of like, where do we need to be and how do we prevent intrusions and all this stuff.

And so he was asked by leadership in his organization, uh, or by the executives to Give up, we need a roadmap to look at where, you know, where the initiatives and what's going on and, you know, lay it all out quarter by quarter, what will happen, OK? Um, and so he had this thing, um, 8 quarters long, 2 years' worth of work.

I think it was, yeah, I think it was, yeah, 8 quarters long, 2 years' worth of work, um.

At least 10 different initiatives, uh, at the time, only 3 that he had direct control of.

But the others that he knew uh impacted and like correlated, and then, you know, bubbles of like when they would get worked on, which quarters, what was happening and kind of gang chartish looking thing.

And he added one extra row at the bottom, and it said confidence.

How confident am I in this plan? And the first quarter, the confidence was medium, right? 0 to 3 months medium.

Quarter 2 was low, and then every quarter after that was none.

So he delivers all this and then uh Ahmed, one of the the leaders in there, love this guy.

He goes, he says, um, so I gotta ask you about the confidence here.

You have prac you have practically no confidence in this entire plan that you just showed us.

He goes, yeah, right, Bob the proctor, yeah, so why did you do it? Like, why did you, why are you showing this to us? And he says, because you made me.

You said you need to deliver this kind of flow and show us what's going on, and I just want to show you how useless it is.

Because so much is, we're gonna learn so much along the way, and it's gonna change what we do.

So I, that's why I have no confidence that this is going to happen at these times, uh, and that this is even important.

And he's like, huh.

OK.

Gotcha.

So, It would have been better if he did something like this, OK? So, um, Another way of of running road maps.

Something I actually, I think is pretty cool here.

Uh, time, right? Time going from earliest on the left to furthest on the right, uh, we should start laying out the types of things that we've actually been pulling together here in this class, right? Like our longer term strategic theme, the objective, the where do we play, how do we win type of stuff, right? Like actually listing that out, uh, this is what's driving all of these ideas that we see below, right? Um, and then per quarter, let's start listing, what are the key results that we're aiming for, right? What are the quarterly, OKRs, the quarterly goals, uh, that we're trying to go and achieve? Because some things need focus down, some things we maybe will start focusing on later, or we need time before we can get to it, or there's other things maybe we'd want to build, so.

You know, let's start seeing what makes sense to do now versus later.

We want to know what are the features that we're looking to discover, right? Um, and the discovery questions that we can go out.

So, right, longer term strategic themes, uh, as I mentioned earlier, you know, your objective should make and your business problems should make sense for your context.

So if you have different teams with different focus areas, it's OK to have multiple road maps here, uh, OK our roadmaps for different groups.

Right, um, then again for quarterly goals, uh, we, we commit to these on a quarterly basis.

What I think is important here is, and I said, I'm a big scrum guy, uh, and in scrum we talk about committing to goals and forecasting work, right? Because specific scope, I don't know.

It's like it's uh it'll be done when it's done, it's the one thing that I know.

But committing to a goal gives me focus.

I will do everything I can to try to make that objective happen.

I can commit to that, right? Um, so that's, that's what I see as the difference there.

OK.

Then um we think about quarterly, you know, the features, right? Things we may build, not things we're going to absolutely build, things we may build, um, are, you know, all the hypotheses, all the best guesses about what we can build in order to hit those goals.

So going with what we talked about last session, right, here's our OR now, what are the things that we need to learn and what experiments might we run to go and do it, um, write those hypotheses about if we Made, you know, um, hot swappable charging, you know, or batteries.

If we, you know, um, I'm trying to think what the all the, what all the other, um, experiment, you know, features were, but that kind of stuff, that was what we can list here.

And then the discovery questions, the things that we need to learn, right? And how we might learn those, we would list at the bottom.

OK.

So, um, the further we go in time, the less we're gonna have on the board.

Cause it's just we don't know, right? We can We can see out there, there's maybe some things that we're gonna build and when we get really far out, when you're you're out, I, I don't know, we might learn so much, we might not even go after that that quarterly KR anymore because we've learned earlier that the, you know, maybe in this section that this doesn't even make sense and we need to shift.

So, you know, we don't want to.

Yeah, we can't predict it very accurately, so we also don't want to make a huge investment.

And that stuff.

OK.

Um, and every quarter, minimally, we should be checking in, right? see how are we doing? Did we see any change in user, uh, outcomes, right? Did we see their behavior shift that, uh, indicate any progress towards these key results or not, right? Um, and then what might we do, how might we shift? The next, you know, quarterly, uh, goal based off of what we learned.

Um, Jeff gave me a great example when I was in class with him about this, and I just, I want to recount his example, um, because it was so very relevant.

He said like, as they were working through this, they, um they were looking at a really low.

Um, employee and job seeker connection rate while he was working at the ladders.

So the ladders was all about trying to go and find job seekers, um, you know, and, and who are paying for access to this platform to connect with employers who are looking for good people, um, you know, like you might do on LinkedIn or something, and, and say how do we find and pair these people up to get jobs filled, to give people these opportunities to help employers staff their teams.

Um, and this connection rate was really low.

Um, it was coming at about 14%, which he thought was like, that's really bad.

I mean, they're both paying for this, and we have that low of a connection rate, like this, something's wrong, right? We got to change this.

So, uh, they set a goal of getting it up to 75%.

We want 75% of the of the employers and job seekers that we're trying to pair up to actually have a meaningful connection.

Um, and, and so it's ambitious, right? It's way more than 14%, but it's also not like everyone will do it, not 100%.

All right.

And with that, they, they ran a couple different experiments and things that they built to try to make that shift.

They did some discovery along the way, and in one quarter, they got up to 68%.

OK.

So what do you think about that? Set a goal of 75, they got to 68.

What comes up for you? That was the dishes.

It was fantastic.

That is, that is the kind of stuff that we should go after.

But I, I'm wondering how did he get to 75%? What did he based it off of? They just Did that like big, big audacious number.

Let's go, I mean, if we can get 3 out of every 4, you know, people pairing up, it'd probably be worth their money.

Well.

Yeah, and then they got, they didn't get to 75, they got the 68.

Yeah.

Erin, what do you think? It just shows an environment where You know, it's, it's OK to say that, you know, the The objective itself, right, that's technically a failure.

But as a whole, right, being able to say that realistically, yes.

Um, operationally, this is a fail, this is marked as a failure, but As a whole, right, we've gone from 14 to 68.

If you're ignoring that and saying that's a failure, just because we didn't hit what we initially said, then, you know, we're missing the plot here.

Yeah, very good, right? And you're right, like, it grew, it grew quite a lot, right? And no one, no one guaranteed 75%.

They just said we're gonna try to see if we can, we're gonna do everything we can to get as close to that as possible.

And they came pretty close and they made a huge improvement along the way.

And so at this quarterly check-in, their leadership took a look at this and actually that's pretty awesome.

Like 68%'s pretty good here.

It might grow a little bit more.

We could probably do a whole another quarter, just trying to hit that 75% if we really wanted to, but why? Um, so instead, they just said it's better to move the team onto something else, right? Rather than try to squeeze out that last 7%, and they shifted, right, towards a different key result and to a different goal for the next quarter.

OK.

Erin, are you there? Your screen froze.

Your face and your eyes moved, so that's good.

I'm like, not just on my side.

I'm looking at on the.

And the little thumbnail.

Yeah, I, I wonder if it's, yep, yeah, there you go.

Something happened.

We will see him again here soon.

I said laptop just made a funny sound and my screen is frozen.

On your side? He, no, he said, he sent me a message.

He's he's texting me.

Oh wow, this laptop looks bad.

OK, it might take a minute.

That's fine.

OK, we will, uh, yeah, we're, let's see, what's the next thing I just want to take a look at where I was going with this next so that I don't accidentally do things that will take them away.

I think it's just like the have everything fade to gray over there, you know, like, don't worry about that stuff, focus on the stuff on the left.

Um, Yeah.

Yeah, OK.

So we're, we're gonna, we're gonna jump into an experiment here in a minute, so I, I can really use him being here for it.

Uh, or the exercise.

So, um, I, thoughts, questions, anything you want to chat about before he comes back? My trouble with this is, how do we negotiate this? With people who might not be.

I'll, I'll give you a real example.

Um, we have new bikes coming out.

That bikes Um, sales folks want.

Apps for that.

The SVP of sales is asking me, how the heck don't we have an app for uh a dirt bike in 2026.

It's not 1990.

So from that point of view, sure, maybe that's uh there's some validity there but the thing is we don't know what we don't even know what value we're adding to our customers' lives, the benefits we're delivering with the app that we have for our.

Of, you know, full on expensive premium bikes.

And to just release an app for the heck of it for uh.

For a dirt bike is inherently risky because if at best what we'll have is a check box.

Because we haven't, we don't know what value we're going to deliver.

But the thing is, if we don't do a good job of even the checkbox, the, the trust factor is gonna take such a large, uh, hit.

It'll just torpedo our reputation straight away.

So we, we did discuss all of these things and given the constraints we had and the, you know, the time we had in to figure out what value we could actually deliver with the app, we said, first version, let's get a rock solid bike out, and then we can figure out what what people are using it for.

And then we'll see if we need to add, add an app.

Now, I'm, I'm.

Whether or not an app is needed, I don't know.

I want to find out, but it sure as hell isn't.

Isn't gonna be found out by someone just telling me how, how, why is it that we don't have an app in 2026.

Yeah, um, you negotiate that.

Let's say I, I have an OR, uh, I, I become an OR expert tomorrow.

I want to find out how do I help me negotiate this thing and drive, is it an education problem to be solved? Is it, uh, is it, uh, is it me connecting it to a dollar value that, that they might be able to see? How do I do that? I can't be.

No, uh, so you, you brought up, you brought up actually some really interesting points so like you don't know, um, and There are, there are two conversations that come to mind, um, with the salesperson, right? And I think walking through with them on this discovery is actually really key, right? Not just coming back to them with like, here are the things we're gonna do cause discovery is an activity and building an app is an output, and neither one of those might actually change the outcome, which It's the first question to to work through with the um the the the user there, right, or with the salesperson it's like you're asking for us to create something, some output, right? But what is the thing that we're actually, what do we expect that it might do for us, right? Um, what, what's the goal? Well, if it had a, what would, you know, why would you have a bike that doesn't have An app.

Again, I don't see user behavior changes.

What does this actually create for our users, and what does that in turn create for us? So we had an app, right? Um, we, we have a a hypothesis.

About some OKR and so you know, you can maybe help them focus on what is the what is the OKR that's important here, right? What that users um are more likely to buy and and pay for extended service with their bike if they have an app that lets them manage things more along the way.

um, and we can see that by a certain percentage or we expect that um if, you know, as a hypothesis like.

Uh, if you're talking about sales of bikes and things of that kind of nature, by having an app, we actually create a more attractive version of our bike.

By making it by having an app that allows users to modify the ride experience of the bike, we will sell more bikes, right? I think we were talking about that in terms of the, the dirt bike one the other day, right, our last week.

How does it help them get to the um to the, the trail where they're riding? How do they tune it for different kinds of trails, right? You know, how often are they using it, those types of things.

So it's easy for a salesperson to say, if I had it, they would.

I'd make more money, right? Yes, so we, in fact he had a, he had a great anecdotal reference point too of a dealer dealership somewhere in Utah.

Where people walk in to see the 0, they're interested, they want to see what it is.

It's uh it's slightly more expensive than the competition, but they can see that it's a premium product.

And then the technician goes or the salesperson goes over a list of all the things, and then an app is clearly missing there and.

They don't even wanna know what's in the app, right? He, he's, he, he, he's really good at this, and he's telling me it's, it's an unemotional purchase.

It's not like, uh, it's not like they're, they're thinking deeply about this and then, you know, considering pros and cons and so on.

They hear, uh, no app, and then it's, it's just one more thing to, uh, one more thing that sticks out against us and then we have to.

That is what they're dealing with, so I don't know if it's a marketing thing.

Yeah, well, there, there's an interesting question you can dive into that with them, right, of, um, like I said, it, it's an investment like getting an app is not free, right? So we also need to make sure that building whatever set app, if we want to do it actually creates the type of results that we're looking for.

But before we build it, um, one, we want to know if we need to build it, and two, we need to know what should go into it, right? And that's, that's, that's many of these items.

Right, um, and that might tie to a key result of, um, you know, like increasing customers, um, uh, connectivity with the bike, you know, um, you know, something like that.

Uh, and so some discovery questions are like, yes, what, what would they do with the app and how would we go about it, and like, what are the kinds of features that they would find it's missing.

But another interesting question, right, one for your sales team and two for your customers, um, is, do they care about an app? Right, and so if you were, if in the sales process, you're listing all the features.

The discovery question I would want to have right off the bat is, how, how often are people asking for the bikes app? Right? Cause if we just tell them there's no app, then they feel something's missing.

If we, if we just tell them all the features, how often do they respond back with and how do I like, and, you know, how do I reconfigure the bike? Where's the app? it's like, is it just a general question like, oh, there's a mobile app, you know, tell me more.

Like I wasn't aware.

It's like I thought it was just a motorcycle, right? I thought it was just a motorcycle.

And if you tell them about the motorcycle and you don't talk about the app necessarily, right, cause it's, it's not a feature of that bike, um, but it's a feature of your other bike, so maybe they see that gap, and then if how many times do they ask, how much do they care? That would be a discovery question that I'd want to get into.

And if they're asking a lot, I'd be like, oh, you know, where's the app? Actually, we're we're currently looking into that.

Now we get another discovery questions.

What would you do with the app for a dirt bike versus a road bike? What would you, you know, what, what are the important parts of this here? What kind, how do you ride your bike, right? And I can go into all these other questions that then, like, I'm not saying we don't, we're not going to build an app.

I'm just saying when we go to make this investment.

I want to make sure that this investment is actually worthwhile because customers want it, and it's targeted based off of what customers need.

Right? So really, that will help me understand their personas a little bit better, that will help me understand what are the different things I can do again, to get to that same OKR just in maybe a different way, versus like I have to build that.

Um, so I would do some discovery in that way and and do it with the salesper, maybe even get them to help you figure out what are the experiments to learn about customer behaviors cause Sales people are most focused on the customer behavior of get them to buy.

But now we can say like, what are the different things that they that we need to know about them that trigger their impulse to buy.

Mhm So that's where I would take it if I were you.

That's that's actually.

There's some pointers that I can try with uh.

They had a sense.

OK, some of it also when we get into the the next little bit here, um, we might find other ways to to show that that work a little bit more, um, but it's not totally listed here yet, OK? Um.

Now you're back, Erin.

I want to get towards our, our next last little piece here before I turn you all loose, uh, with the experiment.

Um, so just something else to consider, we've gone through all of these 4 things so far.

Business problem statements, right, um, and for that affect our users and, and talk about why, what's the value prop there? The OKRs, uh, what are the objectives, uh, what are the outcomes that we're looking for the key results, the hypotheses we have about how to go and solve them and understanding the risks towards actually working on that hypothesis like hypothesis, we need an app.

That's a hypothesis.

Right, so what are the risks of doing that? What are we, uh, what's in the app, how much you needs to get built there, all that kind of stuff.

So, um, we bring this up to show, right, uh, business problem statement is kind of a bit above your longer term strategic theme.

It, it might, I think there could be an error that goes here too, but, you know, your top level OKRs might also fit into that longer term strategic theme, right? They also definitely fit in the quarterly goals.

It all depends on which ones you have, how they tear with each other.

You know, that kind of thing.

Um, your hypotheses are all of the things that we're considering building, all the features that we're considering them, and the risks are in your discovery questions.

Right, so all these pieces that we've been talking about, you can map to a board like this and start to see where might they fall out.

And what does that mean for the direction that we're taking is? How do we use that to communicate with salespeople, with executives and say we're not just doing this stuff because we want to do it, we're doing this stuff because it helps, we think it helps us achieve this goal.

And here's what we're going to work on to figure out if that's worth making an investment in, so we don't waste a bunch of time and money building the wrong thing.

Right, so, I, I've found in my personal experience that that that kind of tie, um, It's what your higher level leadership are certainly thinking about.

They might ask, like, what are you doing? What are all the activities, what are you building? I want to see the detailed plans and all that kind of stuff.

They are somewhere trying to relate to how does this keep us in business, how does this find us new clients, what do they care about, what are they buying? What should we, you know, what do we need to do, and how do we not make the wrong investment.

So if we start doing some of that work for them, show that like, yes, actually, You're doing that up here and we're starting to tear that down into our teams work in certain kinds of certain areas of work like diagnostics tooling, um, the OS, whatever that might be to help users get a better outcome, uh, and to help us get a better outcome from that, or a better impact.

Um, it makes it a much better story, like we're speaking their language, right? So I think that's, that's a really cool thing to do.

OK.

So, Why don't we go? Yeah, hypothesis, turn this into many things.

Um, let's build a road map.

And see what you can come up with.

For your cares and hypotheses, and where might the next two quarters go.

So maybe this you might even be able to use this to help you get away from the chaos and the firefighting that you're doing right now.

OK, uh, so here's what I do.

Let me pull up the mural again, um, and just get you, you're both already in there.

I was like, OK, I'll get you the link, but I don't have to worry.

Um, over here on road mapping, um, we've got an opportunity to start listing that out.

This talks about now, next, future.

Um, you can consider those to be quarters or something like that.

Um, that's just another way that if you don't want it to feel so.

So rigid.

Now next later is another way to, to look at that, that bit there, right, to go and say, here's what we're focused on now, this current quarter.

Here's what's focused on next quarter, this is out there, right towards the future, OK? Um, so how about this? I am going to start, uh, a 10 minute clock here for you guys to work.

We can go longer, but I'm gonna check in on 10 minutes just to see where you're at, um, between some of this mapping and what else you discover as you start to mapping it out.

Like maybe you find there are other features that we need to work on.

Maybe there are questions that you learn to discover, and maybe there's even a no care or two that you want to write, um, uh, in addition to what you came up with before.

OK, um, do you have questions for me on that before I turn you loose on trying to map out your OKR roadmap.

What's AML? or is that AIML? Oh, this is just uh over them for AML analysts.

I don't know.

What is that? RPP for the seamless effect for AL analyst identify machine learning on car rails by the end of the year.

I don't get that example.

I wish, I wish I could say, I don't even know what card rails means.

I'm gonna now I'm gonna follow up with Josh and Jeff like, yo, what's this cause that was not in my board when my class ran through this, um, but I do know one of the things that we talk about in this case is, um.

I'm talking about like a learning management system and perplexity and like the integration between those kinds too.

So my guess is there's something related to that in there.

But what I'm gonna do in the meantime is I'm gonna delete it because that's not your objective.

Um, so I can let you map your objective over here so you can start looking at your OKRs features and questions.

All right.

Yeah, more, more for me to figure out about that actually.

Let me just undo this really quick and copy this data because it's clearly weird like that paste.

All right, now, I'm gonna dig more into that.

Maybe I'll have an answer for you, uh, in under 10 minutes' time.

All right, uh, I'll stop my screen share.

You can all jump on the board.

Uh, I'll give you 10 minutes before I check in.

We'll see how it's going, go from there.

OK.

All right.

So I guess for now we can continue using the OKRs we made around the XCXP.

Right.

Plugged on let's just take the at least directly copy paste the XCXP uh yeah injection.

Right, and then also cause we can tie into, you know, I guess for you to catch up on.

When, uh, at the end of last session in the last session when Rituals walking us through the experiment planning and right, you know, so part of that is that.

You know, the discovery questions of, you know, we have all of these things we need to answer.

And alongside, right, the behavior that we wanna, uh, just.

Alongside, right, we have the, you know, 2A and 2 B's.

On this where we were targeting specific user behaviors.

In that frame, I imagine.

We would want to Great, OK, what's our, what would be a road map for if we're targeting.

Let's say, you know, you use your confidence.

What's something we would try to do now versus later.

Um, So actually The example Rich had on the slide deck.

So our business problem statement, we start off with, we want.

We want to solve why people aren't buying the XCXP at the rate.

We expected, right? I cannot copy the entire.

Doesn't let me copy it.

So I wonder if we say.

Start off something like.

Just Objectives and key results is.

So that we make.

Just kind of feel it out.

So we say, OK, if that was her.

OK.

You know, we need to validate that.

OK, we're testing.

Confidently complete purchases.

Right.

Target market confidence and purchase.

And, and when you say target market confidence and improve target market confidence and purchase, do you mean are models change or uh models and hence our goals change or are you talking about actually making a change to.

Um Well, so that's where right it ties into like when we look over here back in our OKR Swifty.

Right, 2A and 2B, you know, right, so these are some of the objectives we had tried, uh, you know, the the graft and it's like, OK, what can we do regarding converting preorders to purchases? What can we, you know, what are some leading indicators on.

Oh You know, a likelihood of that conversion actually taking place and how can we support that? Mhm.

It's kind of what the the latter two are.

Right, so what are, what are things we can target that would improve this.

And then hypothesis would be, you know, does do these actually matter, you know.

You know, it's like validate, I don't know.

Well customer referrals actually matter.

So in that original slide is indicating.

Risks.

I'm not quite sure what we would put there as a risk.

The, the risk is for 12, and 3.

12 or 3.

Which one? Are all of them? Well, because uh cause right normally at least the chain and the slide that goes.

We have the Business outcome.

OK.

Yeah.

Um, hypotheses and risks.

I guess what else can we, I mean, I guess how would we flush this out it's like.

What what details would we provide here? was it more of, more indicating Additional parallel work would be the best way to do it, or is it do we need to? Explicitly.

You know, provide more details on what each of these items are, as we're kind of creating this road map.

So What I don't What I didn't get is um.

So the initial thing, initial idea that you had of, you know, dedicating one month for each of those, that made sense to me.

So if we have.

That's a risk here.

I pop in a little earlier.

Pause the hour.

No problem.

Um, so the things you listed up at the top.

Um, identify reasons where ICXP is not being its target, improve, uh, confidence.

So where did those come from? So that was based on, I guess kind of at least rigid trying to rigidly follow the pattern that at least the layout you try to describe is Um, right, we initially.

Uh, talked about, right, I guess kind of right, the business problem statement was right inverting, and we talked about the OR is kind of the inverse of business problem statement.

Um, so going back to business problem statement we crafted with F is I guess really the last sentence we were targeting this understanding and solve what is preventing buyers from completing the purchase.

Yeah.

Um, so right, this, this initial thing, this identifies kind of more of the tactical, like, what do we actually need to do to accomplish this, you're right, identify why.

Why we're not meeting a sales target, let's say, I mean.

In the business problem statement we do directly call out, you know, we see that at least initial indicators, the conversion rate.

Um, so maybe is it better to, you know, be more specific to directly tie into the original problem statement? I know, so, um, here's what I was thinking.

This, this item almost feels like a question.

It's not a like a direct question itself.

Right, but it's uh why, why is it not being the target mark? What's, there's something to discover.

Yeah, right, um, but it's not an OK car in and of itself because it, it says nothing so much about the behaviors and and what's going on else um, there's, uh, so if I were to take what we had, right, just looking at what you've already built, um, you've got tris, you know, uh, hitting their confidence.

Let me just go to some of these here.

Um, 85% of pre-order buyers scheduled demo ride, right, up from 65%.

I could put that one up here, right? So maybe that's a high level outcome that we want to work towards, or um they complete it quicker, right? So, um, quicker would be great, but maybe it's not as important as actually doing it, right? Um, so maybe I'll move that one next, or, you know, so it's, it's in that one month pace or maybe it's, it's in this next column here.

OK, um, so I then, if I were to go towards some of the other things we did, we thought, hey, how would we, how do we do this? What are some of the things we think could actually get us there? Um, so looking at our hypotheses, we had some stuff about hot soppable batteries, isn't a is an objective right on something that would help them do that.

Um, it's not, you know, do they want to see how that works from a demo ride.

Um, what about, um, you know, switchable traction control.

Maybe that's another one that comes in here.

In fact, I can also take.

Just the item that we brought up here about, hey, we believe that, you know, this discovery, this feature right of hot solvable batteries is very tied to this one, right? That's a hypothesis that we have directly related there.

And now, how would we know that? What are the questions that we need to do for that? Um, well, we need to, what do we say? Um, we want to watch, we ask people how long do they, do they ride in a day? Uh, we need to see how far do they go before they charge.

Um, we want to actually go take a ride ourselves, maybe, right, or we want to track rides and use capacity.

We thought that was a cool one.

It was just that was more than a day kind of thing, right, of, uh, planning it would take a little longer than doing one day.

Um, That's the quick way to fill your room.

So, so what we actually needed to use there were product outcomes which are leading indicators which we actually have control over and then figure out uh features that supported that.

OK, I think I'll see, see where we missed it.

OK.

Yeah, so yeah, all these maps directly too, and, you know, I gotta say there's one good thing that you didn't do, uh, which I've seen come up in filling out things like this before, um.

A lot of times, uh, you know, people will go and say, well, I already have a product backlog, all these things, we gotta build a map, it's gonna have these things in it, whatever, and so you just start listing those in features and then be like, where are we gonna put them? And then you kind of maybe try to work backwards for the outcome.

Remember, this is about discovering what actually is needed and what works, not just a new way to manage the work that we have, but to actually try to say, this seems to be important because of this outcome and this OKR that we're going for.

Um, these are the things that we need to learn about it, to know if any of the stuff in the middle, any of these features are even worth doing.

Right? So this is here to help us minimize waste and really find the right product that drives the actions in our customers that we desire.

OK.

So kudos for not doing what I've seen so many people do, like I I'll just map it backwards.

No, no, no, no, no.

We don't know if any of that's even correct, right? Those were all other assumptions we made without even understanding why were we doing it in the first place.

Um, so, Here's a question uh for you all now, um, kind of show you where some of this stuff maps out.

Do you want to go further on this? Do you wanna try to add a little bit more on your own and get some of that experience, or are you feeling more comfortable with How this is supposed to lay out.

Yeah, just really just need to let it percolate a little bit, a little bit longer.

OK.

Uh, Mm I'm OK with percolating, letting it percolate, but I'm, I'm thinking, uh, how much time do we actually have? Yes.

I was trying to be mindful of that too.

I wanna like so I was like, I don't know if 20 minutes is really the right exercise.

Um, so here's what, let's do this again.

Yeah, yeah.

Um, do you all need a break or do you wanna just keep rolling? Can we get like a 5 minute break to uh just get some water.

I think that's a great idea.

So, uh, here's what we're gonna do.

We'll, I'll put a new 5 minute timer back up on the board.

Um, if it's less than 5 minutes and we're all here and ready to go, we roll, OK? 5 minutes so I'll see you guys soon.

See you, bye.

Yeah Miss something, disappeared.

OK.

Here we are.

30 seconds section.

Um, all right, ready? There I see you, can you hear? Yep.

I'm looking for the Uh, yeah, right.

I'm sorry, I'm I'm really slow today.

It's OK.

Um, I think, can you say something for a second? Yeah.

OK, good.

Hey, I saw your mouth was moving before but no audio came through on my side and I didn't know why, and I wanna make sure it was everything was good and I think it was just a synchronization thing.

That or might not have, I was double checking if I had muted myself on Zoom or not.

OK.

All right, um, let's go to our last little piece here.

So this is more, um, let's exercise, more concept, uh, but I think she also maybe gets into some of your questions like when do we do this stuff and how do we start getting this to fill in, um.

So the thing about integration, right? At some point we've probably got a backlog we're working from somewhere, right? List of features, stuff that we build, um, like I said, I, I am heavy in the scrum camp, um, and, and medium in the combine camp backlogs our way of life, right, knowing what's coming up next.

Um, so we'll talk a little bit about bringing this stuff together.

If we think about You know, dual track, uh, actually like dual track road maps, um, which is a concept that's like Jeff and, and Josh bring in a lot here.

I think it's important to understand, but also like where they blend, um, right, you've got the same kind of thing that we're just looking at what's the roadmap look like and and the delivery stuff is really on executing on those features, right? What are all these ideas, things that we could build, not must build, but could Build that help us go towards our quarterly goals.

We have to, we deliver on them.

We see if that affects behavior.

Are we getting closer to our goal, and then sometimes there's things that we need to learn, learn about should we build those, what else might indicate a better key result that we could be going after a missing one, something like that, and that's discovery work that we have to be doing on our product and they need to go on simultaneously.

Right, with each other, um, where, cause you, you can't just do a bunch of discovery and feel like you've got all your questions, and then let's go build it.

And if all we're doing is building things and not coming back to learn from them and do some discovery along the way, we might end up building the wrong thing.

OK.

So we're focused here a little bit more on on bringing those things in parallel and trying to run them at the exact same time, OK? Um, the original concept for dual track agile, um, it looked like this, right? Uh, and so this was actually created by a woman called Desiree C.

Um, Jeff Patton and Marty Kagan took the concept and ran with it for a while.

They often get credited for what it is, uh, but, but, uh, Jeff, uh, actually pointed out to a paper that that Desiree wrote where she had this concept of like, when designers are getting in, right, and doing all this different work, um, they, they would often make designs, hand them off to developers later on, go and do that.

Turns out we didn't actually understand it or we couldn't build it.

And so she envisioned this, this kind of dual track piece here, where um we start getting some customer data, right, and then we, we start going through and You know, at the same time, let's implement some of those things that have very low UI costs, right, um, to start getting that there while we're designing for the future, and then they sort of cross, right? We, we go and hand our test group and our design group what was built and let them figure that out and start designing for the next cycle, and we take all these other designs, we say, let's go implement those designs in the next phase, right? So that's the original dual track actual diagram.

What's, what do you think about that? What do you think about that pattern? I mean, it's basically the sense learn respond circle, it's just right, we're tying in different explicit activities together, you know that.

Talking about, you know, parallel activities need to happen, how they'll be different.

You kind of like creating a giant Venn diagrams essentially what overlaps, what doesn't.

Yeah.

Like, and, and what are the specifics about what's happening in those areas, right? Mhm.

OK.

Sure.

Um, give me a minute.

I'm, it's something's bubbling up.

I'm trying to figure out, I'm trying to Put words to what's coming up.

OK.

Um, First, let me see if I understand this correctly.

The goal is that the, the The idea behind this is that uh both the developers and the design folks start with an initial guess of what needs to be done and the developers.

Um, so cycle one for the developers could be something like architecture or the infrastructure that's needed to pull off future experiments.

Is that fair? Yeah.

OK.

So, and when, when they're done with that, then now they can take inputs from designers and Implement things that can be ordered for the next round of um.

Experiments that we want to run.

And So, in this, what I wanna then understand is when do we start polishing things? From a discovery point of view, this is great.

Yeah.

At what point do we go, OK, we've, we have something which answers our answers the questions that we had uh framed for our experiments, but we need to get this thing ready for prime time.

Which is now back to a lot of um.

Development and maybe some input from uh uh designers, but uh so what happens then, or am I missing something here? No, it's not on here, uh, explicitly.

I, I mean, I will give you my, my personal opinion on that, like, when do we start polishing things and, and my, my answer is always and never.

Um, and the reason I say that is, uh, again, if I go back to my scrum roots, every, every time I build something.

It's got to be usable.

I actually have to have something I could put in front of a customer, um, and it needs to meet whatever quality standard we have for that.

Doesn't mean it needs to look pretty, but it has to work, right? It has to work at a, at an expectation that matches what the customer is.

So like, if it looks totally different from any of our other applications and it makes them confused and it doesn't associate with our brand.

I wouldn't release it.

That's actually not, it doesn't meet the right quality standard yet.

But then when it comes to like, well, let's learn about this thing, we build it, we ship it, we were, you know, we, we get, we sense their their response, you know, their reaction to it, and it's like, actually this is cool.

It's just like, I don't know, it's pretty clunky.

All right, so that works, and they have it, and I've now learned, I, you know, I sensed that polish would be better.

That's more work that we could do.

That is a value-based investment, not a quality-based one, but a value-based investment at that point, right? So let's make, let's make it a little less clunky, less button clicks, whatever, um, more performance.

I don't know what that needs to be, um, in order to provide more value to an end user so that they could use it as it is, but it would be better for them if we polished a little more.

So I always, I treat polish like.

Something that I I do with because I've validated its need.

OK, so I think you clarified something for me here.

So the, the cycle 1 cycle 2, cycle 3 here, those are um Small chunks of work that we're actually shipping, right? It's um it's like a end of a sprint or whatever those time boxes are, so the polish happens within implement designs.

It's not implement designs is not just just to gather more data but to actually finish that uh piece of work.

Yeah, yeah.

OK, that's fair.

Let me tell you, this diagram drives me nuts.

Um, And I, and part of why is uh is on the next slide, um, but also I see a lot of delays that happen here, right? So we do some stuff that that these people, they don't even know that what they're building on top of.

They're just designing for the customer.

And gathering some data for it says for cust cycle 3, like we're going way further out, like what stuff helps us implement here because we're gonna take this data and feed it in to design for that cycle and move ahead, right? So there's this lag and so we gather data in cycle 1, which feeds into designing cycle 2, which gets implemented in cycle 3, which we then deliver towards people to test in cycle 4.

That feels like a immense lag and like a mini waterfall and, and a lot of context switching to that is the thing that as a manager now if you think about this now the design team and the for any issues that come up.

For bugs that come up, now we have to resolve that out of band, we have to break the cycle for the designers and designers on the other hand, are doing a context switching constantly between two time scales.

So I, I can, I can see why.

Why it drives you nuts.

I don't know if we could pull this off without him.

Well, good.

I don't want you to pull this off.

OK, actually I'm gonna give you a better way.

So the, the, um, this, I worked at a team or a company that did this.

Um, we had a a design track that was going, there was a design team, they were working for things, right? They were working on a track at the same time development was going and our team, um, we had to fix.

The user experience of um a charting area in an application.

It was just really clunky and there were a lot of low hanging fruit items to go and start knocking off and um and when it came to it, our team was like, that we volunteer, we'll we'll work on that.

I mean this has clear customer impact.

Like, you know, in Valium, yeah, sure, we'll we'll do that.

Nobody wanted to touch it because the team that I had been working on had actually was told to stop working on this product, we need to get you into training, OK? So they, we pulled them offline and we're like, well we, we'll we'll, we'll deal with that, we'll get into this mess.

And when we went to go and do this, we're going to work on these things.

There is no, there was no design for it.

It was just feedback that was saying like, hey, we have to go and improve the way that users interact and experience these charts and to get the data that they need, um, and, and answer the questions that they're after.

And so we're kind of curious about what those questions are.

And so, uh, a member on our scrum team had the wonderful idea of saying, well, let's go talk to the designers.

And maybe we can get their help, right? Making something that's just got better user experience, cause this is really not our forte.

We have the technical chops to do it.

But that doesn't mean we're gonna create a good design.

Um, so we go off to talk to the designers about this.

And we come in and we say, hey, we're working on this charting area and we're going to try to make some, we're gonna be making some improvements in here.

We would love to get your help.

And the guy we talked to said, Oh yeah, actually, um, we have that planned out in in this board, let's say, uh, for cycle 3, like it's going to be about 6 weeks until we start diving into those designs and then we can get back to you on, you know, what we've discovered so you can go and build it.

And our, no, you don't understand.

The design, like the design is changing this sprint, the product that we are actually doing this work like today we're starting on it, and it will look different tomorrow and the day after, and by and in 3 weeks, we are going to show a different charting application to our stakeholders.

And like, well, we're not ready for that yet.

And we're like, we did that that's not why we're here.

It's going to happen.

We want to know, would you like to work together? Could you help us, right? Because we don't understand user experience and design nearly as well as you all do.

And they're like, oh, we, uh, here we got all this stuff going on, and one guy volunteered, right? Uh, he said, yeah, I can swing over.

um, can you, I'm a little busy today.

Can I pop over tomorrow? Yeah, sure, right? So he came over, we'd already started working on things, and he's just like, so what's going on here? We explained the problem.

Here's what we're hearing from customers, here's what's going on.

We want to improve the way in which that they that they that they use is by helping, making it easier to understand where they're interacting on the chart because they didn't know which chart they were using, make it easier for them to find the the the metrics that they were going to bring up, uh, those kinds of things.

Oh, OK.

What if we like highlighted this chart? Can we do that? Sure, we made it ugly, like bright red, you know, somebody's like.

Yeah, that's garish, like maybe we can do something else here.

Oh, I see we have a border, it's not very visible.

Can we like make the chart slightly bigger than the other one? Yeah, we can do that, right? So we just in the moment, working with the designer to craft a different solution, right, um, was pretty cool and then like launched it, it was a much smoother result, right? So we didn't think of a a design team and a developer team.

What we thought about was creating one team.

One team that oscillates between all these different things, between, we did a bit of discovery, and now we have to like actually discover and start delivering something, and then let's build it, and then we kind of pop back up.

Well, is this working? No, let's go back more.

Let's get it out there in front of customers, right? Um, it's two different types of work, but it's one cross-functional team moving together, right? Like, like testing out different things at different times and supporting each other in order to achieve the outcome, not about getting the output done, but to say, hey, we can all, you know, we all have different perspectives, different skills that are helpful here.

How do we as one team, work to go and make this objective happen? In earlier, the 1st 5 or 10 minutes when I was driving, which we were talking about what changes we've made in the team.

This is the direction that the big change in direction that we had for, for the diagnostic tool that we're actually gonna review today.

Uh, one of the, just to go back to what you had in the previous slide, one of the things that I've noticed is that it's tempting to assume.

That maximizing um employee utilization.

Is Strongly correlated with delivering value, they might not be the same.

Fast is not fast is not quick, so.

My martial arts teacher used to tell me that all the time, fast is not quick.

That is definitely not quick.

Um, God, I saw that in in a movie recently too.

I thought that was really good.

Uh, I can't remember the movie, but it probably was somebody else's martial arts teacher that brought it into the movie, um.

When, uh, Yeah, are you struggling with that at all with anyone? Appreciating that concept.

Aaron says yes.

I can feel on your face.

Uh I think I have that trouble.

It's not so much within my team, but outside the team, it's even more difficult to convey that because for most people.

The, the way I, I used to do it in the past was to play this game, right? You you one person holds their hand and then the other person taps, right? Most people when they wanna tap, they start moving faster, but in the process, they're telegraphing so much motion, they're looking at it, the the it, it doesn't always lead to better outcome, uh, but, but just slowing down and seeing minimizing action and then doing it slowly is, is more.

Is much easier than uh.

Then trying to do it faster, but I, I don't think I have.

That's, that's one thing that I want to convey to both inside and outside the team.

So I, I, I agree, like it's, it, some people are really stuck in that mode of thinking.

Um, I try to think of a, a couple of pieces there.

I Um, Try to always understand, do you think that people would like are just gonna work less? Is always a concern and it's like, you know what, they, they might, and that's actually OK.

But you know, cause, but I'm gonna keep paying them as if they're working 40 hours a week, like, no, you, what you're doing is you're, you've hired them to make your company achieve success, right? I didn't bring these people with like, here's the plan what they need to do all the time.

Maybe if they're working a support line or if you know, you're, you're doing literal like factory kind of work, that makes a little bit more sense.

But when we're trying to build things that haven't been built before, and, and I'm trying to say like you were hired because you've got a great mind, you fit our culture and your skill set can help us achieve more, right? So what that more is, I want to focus on that more, right? And so those are outcomes, those are objectives right here, right? So tying it back to that, so if I can get my people to to achieve these objectives.

More efficiently, right, or and and in whatever time it it means for them, right? If I'm learning that it's actually taking longer to do that, right, they're putting in all this time and effort, um, maybe that one actually I don't need to invest in this same anymore, but if, if I just let them run.

And they, and they're not working 100% of their time.

We still get the objectives done and maybe quicker than we thought.

That's a good place to be in, right? I don't care how you did that, right? Like you made the, the good thing happen? Cool.

Hey, um, well, if that's not actually, if that's only keeping you busy 20 hours a week, um, and I mean, you know, you're a full-time employee, so I kind of expect like 40 hours a week out of you.

What else could you do? Let me ask, I try to ask.

Employees that question.

What, what do you think that you could be doing that could help us achieve more outcomes? Not what can I give you to keep you busy, right? Um, and where is the right slack time to learn? To discover, to deal with emergencies, right? Uh, that kind of thing.

And so I say Slack time very, very specifically, um, Tom DeMarco's book Slack is a great, great read, uh, for help, like, and it includes a lot of data and math around why 100% efficiency, right, and using employees' time is actually one of the worst things you can do for most organizations.

Uh, yeah, I think.

I think I can see that.

I also, I've read somewhere that uh servers, if they hit a hit 80% utilization, that's like a All hands on deck kind of situation because it's it's red water, uh, red water mark levels, so.

It's the same.

Why should be different for human beings? And those are repeatable kinds of tasks, right? Yeah, so you, if you were, if you get to that case, right, um, once you get over that 80%, and I'm, I'm a computer engineer by trade, so I know exactly what's happening under the hood, um, the queuing time to organize where processes go starts to actually affect the workflow.

Right? So that's where some of the other 20% is, and it, and so things can't just flow because they have to like be reorganized to fit the right cores and find the memory and all that kind of stuff.

And then when you hit 100%, things are processing and other stuff's waiting and not getting done.

And then that comes in, which means that this thing has to wait and so you're just in this constant, the oscillation back and forth.

Makes everything take longer, right? If you want one thing done quicker, the best thing to do is make it the only thing that gets worked on.

And that might mean that you're only using 10% of your capacity, but the focus says, look, this got done and now I can go do something else, and I can go deliver on this thing and put it out there and wait for the market to go and respond, and I can go do some other discovery or go do some other work, right? So if you don't have slack, In some kind of way, you do not have the opportunity to to learn and you slow everything else down.

It's like asking, asking kids, you know, when they're, when there's, uh, when they're playing.

If, if we went to them and said, hey, only 80% of the time you play, what are you doing the other 20% of the time, they won't even understand what that question means.

If you're in a state of play, we're playing all the time, you, you'll find something to do.

Right.

And if you're, if you're, yeah, if you're in a state of play all the time, right, if there if um I was talking to somebody recently about this in the lounge yesterday, they said they sent their kids to a Wharton school um and there are a lot of just outside time and there's a lot of time where they're like they're just like sitting out.

Sign in nature board.

Right, and there's this kind of like, you like your kids should be bored from time to time.

And the main reason, right, is because in this moment of like if you just, here's a, you know, here, here's an iPad, go play games or something like that, that all they've got is like an activities, activity activity activity all the time.

What they don't do is they don't discover what do I actually want to be doing with my time.

They also don't go and say like, well, I'm sitting here, right? Like, oh, there's nothing to do.

I'm so bored.

Cool.

Yep.

Good luck with that.

And then sure enough, 5, 10 minutes later, there's like a pillow fort that starts to emerge and we created invented a whole new game like that creativity happens in the gaps and we let our minds roam.

So we need to do the same thing in our real work.

That's when we find our best solutions.

OK, I'm gonna move on here cause I know you're short on time and there's just a few last little things left, OK? Um, patterns, uh, uh, within dual track agility even with one team, right? Um, you can have separate backlogs, one for discovery, one for delivery.

Um, you can do one backlog where each thing is its own item, like this is just a discovery piece, this is just a delivery piece, you know, that kind of thing.

Um, and then there's blends right where the discovery work is actually in the stories, the product backlog items wherever they are that like in doing this, as we get it, move it up higher on our backlog, we have to do some discovery and when we go to do the work, like I was saying before, we're going to start building this thing and we're going to like interact and learn from our customers as we're doing it.

So you might have some things that are, you might have things that are like a lot more discovery or even full discovery or like, you know, mostly delivery, but there's a supposed to be a bit of a blend on all of these items because we don't know exactly.

What what gets the job done until we're done.

Do any of these look familiar to you and your work? Yeah, we're trying to get as much as possible on the, the one on the right.

Uh, one of the things that's actually helping us is that we have two teams, one here and one in India.

So based on, especially for this tool that we, we were working on, uh, based on the work that we had to be done for, let's say the next 2 or 3 weeks, the right kind of pod would be formed.

So if we had to do a UI revamp, then uh designers and the UI developers would get together, right? They would form a pod.

And then And then we do because we are, we have a time uh uh time zone difference.

We, we, we can, you know, the designers design something and then, you know, they, they sync with the developers and then by the time they come back, they have something that they can play with for the rest of the day and see uh what the users like and don't like and so on and go back.

So my personal preference is to do something on the right where people are working together and talking together all the time.

Um.

You also, though, it sounds like you're doing this a day at a time.

Mm, perhaps, yeah.

If your UI people and your designers and whatnot are working on something and then it goes off and they can build it essentially over the night and then the next day you can see something.

You know, it's a very short cycle.

But that's what it sounds like to me.

Right, um, But at least in this case, the work, like, we have to make the work visible, we have to create some, you know, some visibility to it.

And if you're planning on doing it that integrated, right, or even like that closely integrated in time, not necessarily in team, um, then yeah, on the right, you're you're getting closer to this right side where it's like, we take it on, we're doing some of that work, we're delivering at the same time, we're feeding the cycle constantly.

Um, OK.

So you're trying to get more to the right, you're maybe more leftish right now, but you're trying to get closer to the right or middle maybe right now? Is that correct or no? We We, we definitely don't do what's on the left.

Uh, we're trying to, uh, in the past we used to.

I'd say maybe between The center and right is where we, where we are.

There are some of these things which are just discovery only, uh, like design only, sorry, that, that we still do.

Um, Yeah, I'd say from 2 to 3 is what we're trying to do, trying to move.

Oh.

Um, that's good, cause yeah, we definitely don't want the one on the left, right? Cause now, this, this is the left is the easiest way to create multiple teams focused on different work.

It's the easiest way to say like, how do I know this discovery work actually influences this delivery in the right way and vice versa.

Um, who is managing these backlogs, right? Um, what do the teams pull from? Do they choose? or do we choose, right? At least having one backlog there helps make sure that we know why we're doing this work.

We know that this, this is coming up next because it helps us answer a key question.

It helps us deliver on, um, uh, a solution that can help towards to create.

Uh, an outcome that we're after, right, you know, all that kind of stuff.

Um, the one on the right, there's something I think is really important to also call out here too, that in my opinion, Um, again, like I scrums my kind of thing, like, you know, teams might pull in the these top three items in their current sprint, but we know as well in that case, they don't really work on just these 3 items, they're refining these other ones and discovery as refinement.

Can be a really good thing to do, right? So if I'm looking at something that is A couple sprints out worth of work a little bit further down the product backlog, and we just don't understand exactly what it is that we need to build yet.

That's OK.

And as part of my like, what should we build here, we made this, we may come up with hypotheses about these items that we think that this will actually help users in a certain way.

We may go out and talk to customers to learn about.

You know, what, what do they do? How much do, how far do they drive their bikes, right? Um, do they run out of charge, um, during, uh, uh, an outdoor trail session, that and, and that they're sad about that, right? Do they run out of gas, you know, one of those, those pieces that they're running into, um, that then we can come back to in another refinement session, uh, you know, a week later, a day or two later, and say, hey, we actually have some data which can feed.

Some of the information about this product backloggram, and even though we're not building it yet.

We are doing active discovery work on it, um, that informs where it goes, that helps us prepare for that solution, um, and ties us closer to the outcome that we're after.

So it, that's a key aspect to me is that discovery and doesn't have to happen at the moment you take that item on.

Sometimes it's in the refinement.

There are two things that you didn't mention here, Rich.

I, I, I wanna check with you.

Um, so one of the other things we have tried is to integrate the QA or the testing folks into these teams too.

So as the designers and developers are working on something, they also immediately tested and they give feedback immediately the same day and so on.

They're not the same as the people who are certifying the bill and saying this is ready to hit the, you know, go to the customer, but we're doing some initial testing, so that is working great for us.

One of the things that we're, we've missed doing in this pattern is to have some kind of release management integrated into this.

So as, as we come up with them um.

They come up with a small.

Um Batch of work We know it can be shipped But we've not released it because we either don't have the bandwidth to um set up the CICD we need to do like a continuous thing or in some cases it's because we actually need to test the whole bill which takes like in the case of firmware.

They might do some small scale testing for a week, but then by the end of the week we can't just deliver that firmware without testing it without doing some integration testing to our customers, right? So if we have to do that, how do we while they're testing.

We, we still end up with some convergence time, right? We, we're still looking at some baking time at the end of the week.

So in, in a 5 day week, maybe a day or a day and a half is dedicated to just that system integration.

How do we, do you have any advice on how we can go about doing that in an integrated fashion? Yeah, um, so there are a couple things that come to mind.

Uh, I love that you're trying to get the teams integrated a little bit more together, and it sounds like right now, You don't have any kind of CICD and release pipeline.

Is that correct? No, no, we do, we do have CICD for one of the projects that the, the one that we were working on.

It, it was one of the cornerstones for our uh.

Uh, swiftness, but, but I think we can, it can have, uh, It can be far more integrated in even this project and then there are other projects where, you know, there might not be as much penetration.

Yeah, and um and when you integrate that work more and you have teams that have more that blend of skills, two things tend to emerge, right? Um, uh, one is more people will be sitting on their hands.

Right, if they don't know how to do other types of work, right, which that will also happen eventually they might learn like you're testing people might understand more about how to release, uh, or they might understand more about discovery, right? So if I'm going to be testing these things, how might users interact with them and what what kind of discovery work can I do, um, and talk with customers that actually influence the way in which we test the product and also thus design what we're building to be tested in the first place.

Right, so like, people, I think, grow their skills over time, but even if they were only doing the very one thing, right, then at that point, you're optimizing, again, not for the individual and their busy, but for the team.

To actually create more usable results at a quicker pace, right, because there's not waiting and and there's not that those handoffs to another team.

Um, and as a team, then you're deciding when do we do different levels of validation.

Unit tests can be done on a developer's computer and on the build server, right? And they're very fast feedback that happen often.

But running your full integration test suite, if that takes a day, you said right now, or a day and a half, I don't want to do that every day, cause I, that's, that's too costly.

So now the questions come in to, is it the right way to do it? Right, and, and how might we increase the speed of our delivery, uh, you know, and like of our, of our integration testing, right? Um, and there's, there could be lots of answers in there.

I've seen times where maybe there's things we can directly automate, maybe it's that we need to actually push our testing left more, right, more into the unit level or or other ways that we can automate it.

Um, versus, um, you know, when it's, I mean, at some point you said you have to deploy this to a bike and a phone and try to get everything to connect, and that's cool, but like where does that happen? And do we have a bike that's just sitting there ready for the build server to deploy to, right? And the discovery of that, of how we do that, um, is a team capacity issue, right? they maybe need some time to work on that.

But the question then becomes, why do it? Do I do what? Why, why try to speed that up? Why try to make it, you know, what if you had an integrated team that could do this kind of work, so that, um, why would you Give them the space to not build new features that could go and create more value for a customer, but instead go and speed up their integration testing.

In my mind, I, I don't even want to prescribe that.

That's up to them to figure out what is the best use of their time, right? If, if they're waiting.

So I'm, I'm thinking about it because there are some, there are like 3 or 4 teams and each of them have different needs.

I, I personally can't prescribe one way of doing uh to them.

So I'm I'm.

I, I don't have a good answer for you uh for, for the question that you asked me.

I, I think You, you, you answered half of it already, um, right of you shouldn't choose how.

Right? So all the features, solutions, if we did, if we did this kind of thing, it would make us, you know, integrate quicker, like, and that's cool, right? Um, and I want the teams to do this because this is important.

So but the why still matters.

What's the outcome? It, it's, it's the same thing that we were talking about earlier as far as I can see, fast was quick.

If they can deliver faster and they're not waiting, which is frustrating for any kind of uh creative person to just wait.

Uh, and not deliver value.

The, the, we break the flow of things by so uh that is one thing I can think of.

And and so that's an interesting thing, right? Like if, if we had something like, you know, how might we uh let's go back to business problems, right? And but look at it internally.

We have internal customers that we're trying to go and help.

They demand lots of new features and capabilities on our stuff all the time, um, and we're actually kind of, we've noticed that we have a problem in which our designers get out of flow.

Um, when we have to go and go through these testing cycles and we're waiting to find the results, how might we improve the speed of delivery, um, in, in our build processes so that we can test and learn quicker.

To deliver the most value to our clients, right? So again, no, no solution prescribed, but you know, the same concepts we've been talking about throughout this whole class, right, of focusing on the problem, trying to understand the outcomes that would would say, are we actually doing that and then letting our teams discover the right solution to get there can be applied here too.

So I, that was, that was what I was just like thinking I'm like, yeah, we can do the same things here.

So I've seen it done in lots of different ways.

I didn't want to prescribe a solution other than give you ideas of this could be done, um, but yeah, when you, I, I think, you know, another key value there is um.

When teams can integrate their work a lot quicker, when they can get something done faster, right? When integration takes minutes instead of days.

That increases our ability to learn.

Alright about our customers and learn if our solution creates the value that we want, it increases our overall uh quality of our product, cause I'm not like trying to do something while this test is happening and then like, uh oh, that didn't work.

What did I do over here? How does, how do these all affect each other? Um, it increases our organization's agility, right? So we can say, yeah, I mean anything that we work on, once we're like, you know, we wanna make sure that this is something that can ship.

I can have that done in 30 minutes versus like, yeah, we'll do that and then we have to take a whole day to test.

It gives us more options to test our market, to create better outcomes for our customers, and to create better impacts and results for our business.

So those kinds of things are are really key and that that's the kind of stuff I lean into when I'm saying we're gonna change the way of how we work.

We don't maybe know what that is yet.

And to resist perhaps upper management, that's like, but your teams, we got so much stuff we gotta work on and do, why would we have any time for that? By making this time, we expect it creates this kind of outcome, right? And I can I can try to instead focus them on the outcome about something that they may care about.

And then that's the story that gives me the opportunity to work on it.

Right.

I need to hop off now.

Yeah.

So, but, but Aaron can stay, right Alan? Yeah.

So we, yes, we are literally right here.

I mean, this is just wrap ups, right? So I can, I, I can go through any of the wrap ups and questions that you want, Erin, but we, yes, we are, I mean we we just talked about a lot of those right there, right? Outcomes over outputs, OKRs drive everything.

It's all hypothesis, tell those good stories, right? That's super, super key.

And then, yeah, we get to.

Cheers, have a good time.

Um.

So, let me pause and say, Che, thank you for bringing Aaron and Matthew along, and this is really cool and fun for me, and I'm glad you got a lot out of it.

Um, let's you and I catch up sometime in the near future.

OK? I will, I'll I'll, I'll, I'll, uh, send you an email and then maybe we can do a quick chat and figure out what, what our next steps can be.

I, I can't thank you enough.

This has been great and I'm truly thankful that Aaron joined every single day.

Even when it was 6 a.

m.

in the morning, which she absolutely, absolutely hates.

Thank you so much.

Oh yeah, I appreciate the opportunity cause, you know, again like when we first introduced yourself, right, this is, you know, Shay is saying, hey, you know, I think this is a good, you know, this fits the lines of your interests.

You should attend this, see if it interests you, continue down this path and learn more, you know, there's a lot of new concepts here that, especially since I don't necessarily have the existing management context and background it's kind of like, oh these are whole new foundations I can build off of, which is a lot of fun so far.

It was awesome to watch your engagement with it too, right? And like redoing things and coming back in.

I think he made a good call bringing you guys in.

Right, so, uh, thank you, Aaron for like the, the passion that you seem to show in the class too.

I thought that was really, really awesome.

No, just it's a good teacher, right.

I appreciate that.

Thank you so much.

Um, OK, so, yeah, we are, we are like I said, we're done with the class.

I've got some wrap up things that, you know, that I need to prep to send out to you.

She, I know you gotta run.

Erin, if you want to hang out and talk, you know, anything else, other questions, I'm willing to, to, to chat.

Otherwise, we can all say our goodbyes now.

That's, that's totally cool with me too.

But she, at least minimum.

For you.

Goodbye and take care.

Bye for now.

I'm, I'm, I'm definitely gonna get back in touch with you.

I, I, I loved it.

I love the course.

I loved your pacing.

I love the, the, the, the presentation style and, and the, the thought you put into every single slide and, and flow of, uh, you know, the sessions.

I, I, I really enjoyed it.

I really, really enjoyed it.

I, I actually wish we could do it like a 2 day, hands-on, do nothing else, 2 full days, just do this.

Oh, I wish I could do that maybe 2026, right? We fly me out to, I don't know, one of your locations, one of your dealers, we could do it there or something like that, go with that.

I'm keeping my fingers crossed.

Thank you.

Thank you.

Bye-bye.

Aaron.

What about you, man? Anything else you wanna dive into? Any questions? I know, I know we were going quick there in the end, I let him drive a little bit.

So, I mean, you know, my time is yours here.

Uh, so what can I help you with? Yeah, it's kind of, you know, one of the things when Treas first started, um, almost became his direct reports that, you know, he handed me a list of books and said, here's some reading materials, I think, you know, we'll try to gauge your interest and that's that was the fun part, at least for me it's also in the think of it because it's like, right, it lists some recommended reading materials and all the supplementals like, oh, you know, watch this TED Talk cause, you know, have like this how to tell a story TED Talk, right? um yeah and so it's like, you know what.

What else can I, you know, get my hands on to try to read about and figure out that ties into all this cause I know, right, you know, there's the entire sense and respond, you know, book series that you pointed out in the first session that Tres has on his bookshelf, yeah, outside of that it's like, what else? So, oh, OK, you, you're looking for some specific recommendations on other things to go and look into, read and get and go further.

OK, um, well, I mean, like I said, I gave one, right? I, I think it's really good book there, um.

Yeah, another good TED Talk, um, Astro Teller, uh, he is, uh, he was, um, Google's X Labs, uh, and he led that whole initiative for a while, um, and he talks about, um, Uh, that's like the surprising like the truth of failure.

I'm trying to think, um, the unexpected celebrating failure.

Yes, the unexpected benefit of celebrating failure.

Uh, really cool TED Talk, and he gets into like, again, some of these discovery things and, and for them it was, um, as much about learning about users as it was about learning like tech technically is this feasible and what, what does that mean for us, right? Cause he didn't, he's like, we don't have infinite budgets, we can't just do anything, right? It doesn't matter how much the sales guy wants it, it's um we're trying to solve a problem and and and we say, what's the Best way that we can kill this effort right now, right, uh, to say that like if we couldn't get this done, we, um, we might as well just go work on something else.

And so they actually, he talks about succeeding with a lot of those for a while and some of those projects that have since also gone away, and some other really cool things that they that they were like, this is still viable, but it's not viable for us.

We just don't have the funds, you know, to continue with it.

Um, so yeah, check that out.

Um, books.

I have a lot of books, uh, check, just pull over one of the key ones that I wanna dive to, um, hm.

Yeah, was good.

Mhm When When you work with leadership groups, um, so like I read this as in a um a leadership team at an organization.

I, I eventually became part of the the core management, um, the advantage, I thought it was a really cool book.

Lai's good, uh, but he gets into a lot here into, um, essentially establishing clarity.

Right, so leading with leading with goals, leading with the objectives, leading with values.

Uh, this was a really cool book to help frame that so that when you're going and and trying to work with your directs, right, it's, um, and actually I don't get it.

Why am I doing this? And if you've ever felt that, it's maybe because they're missing some of the pieces that are in that book, um.

Antifragile is a really really neat read, um.

So this book um was The whole key thing here and it's, it's, it's dense, it's like it's a, it's a very, very dense book, um, but the whole concept is in studying and understanding systems that improve under stress, rather than ones that fracture, right? So you, um, if you think of fragility, right, you take a, a glass, you drop it, right, it breaks and shatters, um, you know, it's fragile.

Uh, this cup.

is not anti-fragile, it's resilient, right? It's robust.

So like I can drop it and it's not gonna break, but I might dent it, and that dent's not gonna go away, and eventually that might affect its performance, and I can crush it and it'll go and it'll be, you know, ruined.

But so anti-fragile systems are things that improve where like um Your immune system.

You get sick, you're ill for a while, but now your body knows how to resist that in the future.

Um, a restaurant is fragile, but all of the restaurants in an area are anti-fragile.

If one of them undergoes like a health code violation, they all benefit in some other way from it and learn how to improve their quality because people are gonna be looking for that, right? Um, taxi drivers, right? Uh, antifragile, right? It's like, oh, I'm not getting a lot of businesses in this area.

Where do I go? How, what else can I do? I don't have to sit here and wait, right? So I think that those concepts apply really well in How you work Right, and try to figure out like how do I develop antifragile systems around me to learn and improve myself, um, and, and towards what I work on, um, as well as in business, right? What can we do to Apply stress to our business in order to help us learn and improve.

Even a lot of the concepts we've been talking about here in this class are mechanisms that can introduce that stress and help you actually improve and grow from it.

Sure, I'll definitely check these out.

Yeah, um, yeah, I mean like data books, facilitation books, like I, you know, I've got other things to get to dive into.

I'm just trying to think of what's the best.

What's the best thing for you? Where, where, what are you interested in? What, what do you feel like is a gap that you're discovering? I mean, part of with, you know, at least of the entire training is kind of showing that, you know, at least right, it's the hard part is always figuring out how they can immediately apply it to the projects, right, at least, you know, thankfully since, you know, the class was just people from zero we're able, you know, directly tie into, you know, the, you know, on the mural board what we've done.

And then for now for me it's also thinking about, OK, you know, we've done this one vertical slice, you know, how is this, how can I start doing the same to you know the other projects like Dres I was talking about the diagnostic tool we've been working on.

Um, yeah, and of course, as you pointed out, you know, at what point do you finally have the time to stop firefighting and do something else? Yeah, yeah, and, um, and making that space for your team, right, to learn and, and grow, and um I have a feeling he, he will try to give it to you at some point, right? Um, and, and it's something that I, I envision in the future, you will have teams that you're gonna be, that are gonna be reporting to you and like thinking of how do you, how do you give them an objective, right? Give them a, a, a purpose, something, give them a hill to climb.

Um, give them some constraints about how to get there, and then like let them run towards it and figure out how do you support them to do it.

Those are the best teams, right? Um, and they're the ones that are gonna keep coming back and finding something new and, and, and, and want to work with you, versus the ones that are like, yeah, I'd rather be doing this other thing, I think I know better, you know, whatever.

If you do, prove it.

Go ahead, give it a try, um.

So that that would be kind of interesting towards uh my two of my favorite books on teams and and that uh you go to my library here.

Yeah Um, work rules, Lasso Bach, he was, um, senior VP of people operations at Google.

Um, This is a really cool management book and like all like at least a third of it, if not half of it is about hiring people, um, and like how how so many people do it wrong.

No, definitely, yeah, it's pretty easy to see how hiring.

The wrong people can definitely tank a project or just team morale in general.

Mhm.

Yeah, but they, he also talks like about some cool things about, you know, like letting your teams run and learn in different ways and trying to um like how you How you find out where you like where you invest for performance, right, um, that like you're gonna have some people that are just, they, they do substantially better than anybody else.

Great.

Pay them substantially more than anybody else and like let it run, right? And, and then like don't improve the things that are like, oh this is like not working that well, we have to get it better.

Maybe don't, maybe actually it's not a key factor and need to find something that is working really well and make it even better, right, so that you can like get to that level of what's working versus is that worth investing to bring this up to speed.

Yeah, yeah, so like that that's a cool one.

And then when it comes down to like what do you double down on and how do you grow that over time, um, it's, there's a lot of ways in which you can apply like the stuff that we've learned or, you know, uh, actual patterns or anything to every aspect of your business, every team, and not every team really needs that, or not all of it.

Um, and so, uh, scaling up excellence was a really neat read about that.

Um, So the cool thing here was like like really finding excellence and and understanding how it gets copied, right, is, is what I I pulled out of this book and like the very first chapter gets into that, I really think.

So this is um uh Rob Sutton and Huggy Reyo.

I just uh feel like at scaling up excellence.

The, um, we talk about setting the stage, right, and the, the, the, the first thing that they that they cover is.

Uh, Buddhism versus Catholicism.

Right, and it's like what, imagine applying like a religious view to scaling, right? And the um Uh, the, you know, the more the Buddhist way, like being in harmony with the environment and trying to like, you know, figure out how to grow from there, um, and letting people work with it versus like the Catholic, like, this is the dogma, we will do, you know, this is how you run a mass, all that kind of stuff like that, that creates excellence in a way, and it also stifles it in ways like they both some, you need a little bit of each of them in your workflow.

So, as you're trying to find out like what works.

You know, the thing that I, I got out of this is like recognizing, it works now.

For this thing, it may work again in the future, but it may not, and being open to identifying some of those core elements that lift up your teams and and help you achieve like more excellence at scale, um, it was, it was a cool read for that.

Nice.

Yeah, yeah.

So hopefully I, I at least, you know, made your bookshelf thicker by that much.

Definitely.

Yeah, at some point I can imagine at some point, yeah, I'll get to the point when Shrey's bookshelf.

Yeah, oh God, I know.

I'm, I'm looking at mine, it's not even like that, that thick, it's like it says, um, but yeah, the, uh, oh, the goal, the goal was a good read.

The Phoenix Project is maybe a better read um for you actually.

Yeah, that was one of the first things she has handed me was the Phoenix Project.

OK, good, good, good.

So yeah, you understand that then theory constraints, I mean, I work the automotive parts manufacturer versus an actual like.

Automotive manufacturer.

That's yeah, no, that was that that was definitely part of the reason why tras was just like read this and it's like tell me how you feel after it.

Mhm Yeah, that that was basically his first intro like, all right, here's a taste of what I'm gonna have you start working on, start learning about, and I'm like, cool.

It's definitely a fun thing to read on the flight.

That's that's cool, yeah, oh God, I, I remember the first time I read it.

I got a couple chapters in and I had nightmares.

Like I, I just, I was like it triggered previous things like, you know, that I've done.

I'm like, oh God, like I, I like PTSD got results.

Yeah, it's just like, you know, the, you know, right, starting off the premise is like, oh, I'm this guy and all of a sudden I discovered I'm in charge now.

It's like, uh, No, I know.

What do I do? Yeah, yeah.

No.

OK.

Um, what else can I help you with? curious about.

I think it's just for right with the entire, this entire training it's.

What are some easy ways, you know, the small slices and begin, you know, chipping away at it, like, right, you know, definitely we've seen like how, you know, we spent a good amount of time trying to refine the business problem statement and figure out what are the OKRs that we can create from that.

And it's like, what are some What is a way, you know, start taking maybe a smaller problem, you know, sca reducing the scale of the problem maybe or it's like.

at least we approached this as a business, a business problem statement is that do I just abstract that to like, OK, is my business now this specific application I'm working on and You know, these are the things around that, or is that still, you know, Yeah, trays would describe it, you know, the tactical level versus strategic level, you know, that level, that kind of thinking, yeah.

Uh, good question.

I think, um, not, not everything needs a business strategy, right, associated with it.

You don't need to understand the way we're gonna play we're gonna win all that kind of stuff and, and, um.

Even the business problem um is overkill for some situations, but I think a key piece, you know, to mention is there's always a problem.

There's always a problem that's trying to be solved, and discovering what that problem is.

And and figuring out how we might know that it's been solved, those are good techniques to use, even if you're not trying to craft a business problem statement.

So, when you mentioned before about like creating this application.

Um, that's an output, right? It is a, it's an assumption, it's a guess, it's a way to do it.

So when you're, every time I'm prescribed a solution to build, um, and I'm, and I, it's happened a lot.

I was a consultant, so people hire me and if I don't do what they want, I might get fired.

Um, but every time that I saw it, I'm like, OK, to build the best solution here and to really make sure that I am as effective as I can be on these things, can you take, you know, explain to me what are the the context, right? Yeah.

Give me the context, let me know.

Like yeah, what led you down this path to, you know, tell me to do this, right? Yes, yeah, so I'm asking, getting curious about that, right, and, and not in a like, um, I wanna know if you're right, right, you know, but like I, I just, I just need to know, like the more I know the better I can do, right? So just help me, teach me.

Right, if the context is like, oh, you know, you, this is the specific problem that the customer is facing, and you want to make sure that the solution absolutely focuses on that, right? That's important information for me than that, but yeah, OK, when I, you know, build a solution that you're dictating that I do put an emphasis on, right, whatever.

The pain point that you've had already identified and you've communicated to me.

You know that I'm addressing it.

Yeah.

Then the next part is finding out how will they know, like how do you, how do you know that this pain point has been eliminated, um, because usually like out of those three things, the solutions easy one to come up with.

Finding the problem is probably the the next easiest thing to come up with.

It's the, the more obvious somebody's yelling you about, you know, yelling at you about it.

And then the thing I see missing the most is how do we know that we actually did the right thing.

Um, but when you start asking these questions, if one of those things is missing, whoever you're talking to might feel uncomfortable with it, right? Um, and so help them discover, right? So it's like, oh, well, here's the problem, like, so that's, that's why we're doing this thing.

Cool.

And just to make sure, like, so that we maybe we can even develop our solution to give us the the the results and like how will we know this is actually like working.

And we made the right investment or that we should keep investing in this or whatever.

If they don't know, like, it's like, well, well, cool, let's like, let's come up with a couple of ideas and lean on these techniques of like how what what behaviors will we see change, right? Uh, are there any leading behaviors that might lead to that, right? And start coming with the, you know, with, uh, well, if this changed by 30%, if we start to seeing, you know, people actually lying on the mattress, you know, whatever, um, then we can do that and like.

Actually, uh, someone I'm working on tomorrow is, um, next week, is it next, oh my God, it's next week already.

Next week, I'm giving a talk with um Jeff and Josh on the combination of OKRs and nudges.

Uh, do you know the concept of a nudge? Not familiar with that.

OK, so, um, one other book and uh I don't have it on my bookshelf right now, um, Thinking fast and slow, it's a dense read, but it's a good read.

Um, it's got my copy.

That's that's something I gotta figure out where the hell is my copy.

Um, so thinking fast and slow, uh, cover talks about the concept of, um, humans, humans process information in two different speeds.

Some things you think about really fast, and they react to it without sometimes without even recognizing that they reacted to it.

That type of information processing is, um, they call it like level one thinking, and it is uh cheap for the body to go through.

Right? So that you're, you expend very little energy making that decision.

Um, there are, when you have to think about it though, and actually like, well, do I want this and what do I do? And, you know, is this the right choice? Um, that, that's a slower, deeper level of thinking.

Something we're obviously capable of doing, but research shows actually causes physical pain in people, like, like it's, it's uh You, you, the agony of that choice.

Right, that word is is chosen on purpose cause it feels painful at times to go through it, and it takes a lot of energy, right? So, um, if you want behavior to change and somebody has to choose to do something out of the ordinary for them, and let's think about it, then that's a painful choice for them to make, and they're most likely to not do it.

Right? So you want them to get to buy bikes and you start giving them all this stuff to think about and choose.

No, right? They're they're not going to.

So instead, the idea is you create a nudge and a nudge is also a choice and um uh work rules talks about nudges a lot too.

There's some really cool stuff in here that and the concept is, um, you want to, you wanna do a nudge is an experiment or an a and an opportunity.

To provide choice to somebody, but in such a way that the choice that you want them to make is now a little bit easier than the choice that they would have made before, and the choice that they would have made before is just a little bit harder to make, OK? So, uh, the, a, a simple example of that is, um, we want the outcome of people um living healthier lives through healthier eating.

OK, um, and we know that in our cafeteria right before they go to check out, there's the rack of candy bars that I could pick up and buy, right? And so it, it's a very easy and obvious choice of man, that, that, you know, pack of skills.

It's delicious, like it's pretty good, yeah, right there I can just grab it, right? And, and off I go, um.

And so, you know, people won't eat as healthy because the the simple choice put right in front of people is a candy bar that's right within reach, right at the moment of checkout.

I don't even, I barely think about it.

I just grab it and go, right? It's um, and that's why they go away, then it's a more of a mental, it's like, oh, do I wanna actually walk over there, grab it, come back, stand.

Yeah, so maybe we actually keep the same racks of things, but we move the candy bars, like you said, somewhere else.

We, we put them just out of reach.

We move them to a different spot, so you have to like, oh, hold on, I need to go back, I need to go over there and do something to get my candy bar.

But what's right at checkout? A whole bunch of fruit.

Right, I can just pick up an apple and be done, right? So the key here is that there's still choice.

That's being made.

I can still choose the thing that I want, but because it's a little harder than it was before, and the thing that I'm, that somebody wants me to do instead, it's just a little easier than it was before.

It's a little bit more like, I, I don't even have to think about it.

I can just grab this apple and go.

I start to nudge behaviors in the right direction, but I can't, like, remove the choice.

I can't say like, well, we're just not gonna stock candy bars anymore.

Right, people get angry, they won't do it, they'll, you know, they'll they'll go out and get them anyway, they'll stock them in their drawers, right? We will not change the behavior.

Um, and we can do that in a lot of things like with our products too, like we can make it easier, and I think, um.

We're talking about that with the bikes, like the demo application, then trying some things out and like just be like, hey, um, why don't you, why don't you try these settings on our local bike, right? And just like hit a button and we'll schedule a demo for you, right? And um that's like, oh, I, I don't.

Do I want it? Oh, I mean, it's right here.

I just but I it's like I'm already here.

It's already set up, might as well, right? Yeah, and like versus like coming in and like having somebody say like if they come into a store, how do you ride and what's going on? What do you do? All right, well, let's like, you know, configure this thing, you know, a certain kind of way versus like putting.

You could ask them that, but you can also put it right in front of them and be like, here, play with this, and configure it be like, hey, you just configured the bike, you want to go ride it? Like it's it's right here, it's ready to go.

And like that choice, and they could say no, they be like, well that's it I you mean how I set it up? Yeah.

Sure.

Right, and now we lead them towards the things that we want.

Um, so those are, those are cool concepts.

And so thinking fast and slow gets a lot more into the brain science about why, if you're curious about that, and it can really help you think of like leading towards those those good outcomes and and getting people to If you, you know, what I try to do is I try to nudge people to think about problems and outcomes.

Over solutions, like, yes, it's easy to to come up with a solution, right? If I make it harder, like, I'm not gonna do this until we understand why and like we can do it, but like we, you know, it's still we, we gotta, I gotta like this up, I gotta do that.

If you just tell me what it is that you're after, we just get going on it right away.

And that might shift them for a smaller thing.

And last thing I ask, is that right? And then the other part that, especially early on, we focused really on.

Avoiding the use of placeholders or You know, making sure that we're producing numbers, you know, it's backed by data or some analysis of Some proof of indicating we currently do that, we are we have X Y Z, you know, we have 30% fewer sales than the plan of record, right? I think, you know, you highlighting the strength of directly indicating that no, you know, we don't need to say we have why models being sold every year.

It's like if we already have the data and we know that's a problem or you know, we know that this is what we're looking at, just toss them in there like what's the point of hiding it.

Yeah.

What do you wanna know about that? Or where, where are you going with that? And I think I realized I started that as a question and I started expanding this and I'm trying to get back to the original.

Um, At what point does, you know, if you don't immediately have that information in front of you, at what point does it, you know, the, the value of actually going through and digging and collecting all that data to produce these numbers at what point does that become prohibitive to, you know, that's kind of like, you know, OK, the output is I'm producing in, you know, a statement with hard numbers versus I just need to say this is the problem I want to solve.

I don't quite have all the data I need to justify this yet, but, you know, at what point is it like I need to do one or the other, no, or I need to give up on.

The data collection and analysis.

Um, OK, I, I see where you're going there.

So my thoughts are, um, the data is there to make the story more compelling.

And to give us some guardrails, right? So, um, the, the, the assumptions about we're gonna improve it by 30%, right? That's, that's a guess, right? And it's just, it's something to show that we're gonna make a movement like you said with the example of we improved by 58%, like that's great when it was at 14, like, no, no question this was worth the time.

Um, so having data.

Helps make that story more compelling.

Um, and, and having a compelling story is important if the cost of investment.

is going to be higher, right? So, um, ah, yeah, I mean, I, obviously it's the problem.

I mean, like, haven't, have you heard anybody talk about it? Yeah, OK, at least one person? Cool.

I want to go and spend 4 months working on this.

Oh, it's a lot of money.

Like, now I'm curious, should we really do it? When I can make a direct connection for somebody like, yeah, we've seen this problem, here's evidence of it, and here's also what's happening because of it for a business like Data about the customer, data about our business, um, people care about that, like, they start to, it's like, it's hard to refute data.

Um, I'm not saying people don't do it every freaking day, but like, it's like, here it is, oh, you don't have data, go get me your data.

Right, like, show, you know, you don't think that this is true? Cool, go get, go, go, go get me data.

You're telling me this is a problem I need to solve? Well, show me the impact on this, is this actually, you know, we're on the value chart and say, you know, this high effort low value, yeah.

Yeah, um, something we didn't cover here, um.

In the class, they used to have a slide and it's uh Missing, um.

Truth curve.

Have you ever seen this before? No, I'm not.

OK, so, um, I, I'm gonna find out why this got taken out of the course, cause, um, I know it's in a different course, but so the idea is really about evidence, and you're saying like how do we, what do we need for this evidence and this data, and if we're just, all we're doing is talking about is this a problem? Like, like I think there's a problem here.

OK, well, like, you got any proof of that? I saw somebody that has this problem.

I don't need a lot of evidence yet, right? But also my investment is just a conversation.

Like if we're like, no, I don't, I don't agree, don't go any further, right? But if we're actually getting to the point of like, we're gonna get something out there, we're gonna release, right? Um, we need more evidence that says like, yes, this is actually worth and, you know, doing the, you know, the work, that kind of thing.

Um, and so that the whole concept here is like, if I don't have enough evidence for the things I'm working on that I'm going to be putting out in the market, that is very risky.

It's, it's unnecessarily stupid, right? I can also collect way more evidence.

That is necessary for the thing that I'm doing, like, if I'm just trying to have that very first conversation, Um, mountains of evidence are great.

If it was cheap to get it because I'm already collecting that data, I've already instrumented my products to have it available to me, and it took me one quick question to an AI tool that already has access to all that data to spit out an answer or like I'm looking at uh Tableau or whatever right and I didn't have to invest much.

Cool.

Now I've, I've already got the data, like it's, it's actually I've got loads of evidence and very little effort to have that conversation move ahead, and I can make me kind of jump to some of these higher states, right? So, um, so you're, you're really trying to get, you know, like in this case, we're trying to make two things happen.

One, Um, that we give evidence to create a compelling story, to decide to actually pursue this thing a little bit further.

Um, and then afterwards we also want to use it to build, change the way in which we build systems like our products and the processes that we put around products to make the data that would help drive these decisions available to us cheaper so far, yeah.

Easier and cheaper to acquire.

When you don't have it, it's gonna take time, and it's gonna and the effort is bigger, but the more that you put that in, the easier it is to put that data in there in the first place of like, we actually, it's not just anecdotal, like people, you know, we have clear data that's showing there's a problem here.

Um, From another perspective, like, uh, I, I think I maybe mentioned Etsy's monitoring and data-driven solutions in the class or or no? I don't think so.

OK.

Do you know Etsy, the website? Etsy? OK, so Etsy is their masters of continuous deployment, um, hundreds of times per day, but I, I like one time, um, they, they do, they monitor some really key things, uh, the modern stuff like, um, the frequency of logins.

They monitor um volume of of posts in their forms and support, you know, channels.

Uh, they measure volume of sales, um, and there was one other key metric that they're getting into.

I can't remember what it was, uh, but like these are things that are just on like up in a, you know, uh, an operations room and Etsy all the time, and they're just like constant graphs, they're just, you know, flux.

And they also have, because they have so much data, they have data about like where, what's the range? That they should kind of be in cause it's never like, you know, exact, right? But they know like, what's a typical range on a 2 p.

m.

on a Friday, you know, kind of thing, right? And so they monitor all this stuff, and because they, they developers have the agency to release anything at any time, like there's constantly stuff that's going out there and changing.

They wanna know like what's what's moving and how do they correlate that to the changes that are coming back on.

Uh, so they had a problem that they didn't write a business problem statement, they just, they came through it much quicker than this, um, where they noticed what, you know, all of a sudden once uh a the the volume of sales.

Started dipping down, right, like just it's unnaturally.

The volume of logins.

Dipped down, like people were just not coming in and buying anything anymore.

And then, but also the number of support and for posts rose.

Right, and it's like, wait a minute, this is people are engaged.

But they're not buying, and like, what, what's going on here? What happened? So they started looking at data to find out what's the problem that we're trying to solve.

We have evidence and the evidence in this case was that um people are saying that they cannot log in to complete their purchases over and over and over and over again, right? What do you mean they can't log in? Can we, do we have logins? They run all their tests.

Yes, login works perfectly fine, right? Somebody get in there and actually log in and go buy something and let's see what the heck is going on.

Um, and they found out the problem.

Somebody made a change to a CSS style sheet.

Where they were modifying the color of uh something that was blue in one spot and they wanted to make it white.

It's totally different, right? But what that meant was, is that the blue login button.

That had white text on it and that was on a white background, also used this style and now the login button just disappear.

So people didn't know, like they didn't think I click enter to log in.

They, I have to click the button and the button's gone, or it wasn't.

You just couldn't see it, so nobody knew how to log in anymore.

And thus I was trapped, right? So, OK, we can find that, revert that change of roll forward and I wanna go.

Um, but they, you know, they, they designed for this information that helped them identify potential problems from the very beginning, and it, it's part of everything that they do so that they still designed to get this information and they can still release things hundreds of times per day.

And get it there, right? So I think like.

It can be a journey, but that's, that's also why you want those numbers.

It's like, if you don't have them.

You know, and your stories are compelling, you need to figure out how to get them, and you wanna make sure that that is as easy for you as possible, so that we can actually focus on the problem, find the problem, focus on the problem, solve the problem.

That's sound good? Or any, any more questions around that? This one It's gonna, you know, try to start seeing where I can put these in the practice, right? Yeah, um.

You know, a, a, uh, a political variant of the story, right? Not having the data right there, right, um, obviously compelling action, but also compelling lots of different kinds of action.

Um, so I think about all of the troops getting deployed to support violent crime in Chicago right now.

Right, um, OK, obviously, you know, the president can just choose to paralyze troops and send them in and not, you know, there's a change otherwise, right? But there's no like, you know, we see violent crime as evidenced by blah blah blah blah blah.

And in fact, the people who had the data created different problem statement with we have all of this evidence that says this is not a problem.

And the more compelling story is that, right? And so that's creating this distance, right? And if, if everyone could see the same data, it's like, yeah, actually there is a problem that we would need to do something with.

Um, so let's go do it, or, you know, we all see that there is no problem, so we're gonna put our energy somehow, so.

Data helps tell the story.

It just makes it that much more believable when you go and get there.

Right.

Um, we are also, so another one, I just got a preview of a storytelling class that we're building with Sensorypa.

Um, so that was fun.

We did like, um, ethos, pathos, and logos arguments, right, like a appealing to authority, uh, appealing to logic or appealing to emotion, um, not in that order, right, but, uh, those things, and then other, other formats and stuff, so.

It's another cool way of like taking these kinds of information and formatting it in different techniques and using those things to, I said, engage people and say, yep, let's go work on this problem, and let's focus more on that, rather than your luxury list of ideas on how to solve it.

Or it's like, yeah, you know, let's go.

If you got a laundry list of ideas, let's try and see which one of those we actually need to focus on, right? Exactly, exactly.

All right.

Well, um, Erin, I just realized I have a call in 20 minutes and I have got to figure out lunch.

Um, so I'm, I'm gonna get rolling, but dude, I, what I said before, I really meant it.

I really appreciated like seeing what you're coming up with and your engagement and like re-engagement with all the material.

It, it was cool to watch you think about it right on there and, and to your questions got more intricate as we went through the class.

So like that was, that was cool for me to see.

I'm at your disposal, man, like if you ever want to talk about any more of this stuff, and if I can help you out in any way as you continue to, to learn, grow and advance in your career, please let me know, OK? Thank you, it means a lot.

Yeah, you know, just in Patras, you know, I really appreciate, you know, the way you've, you know, set all this up, you know, tying, you know, slide deck content, all the different experiments and giving us, you know, the space to be like, oh, OK, let's go ahead and actually try to do what we just talked about, you know, I really appreciate that.

Yeah, you being so willing to provide feedback.

Cool.

Well, I will try to keep that up and I really appreciate that that feedback there.

That just, that means a lot to me, you know, I makes me feel good, makes me want to keep doing more of it.

Yeah, thank you.

I appreciate it.

Thanks, Erin.

All right, take care out there and um yeah, I will, like I said, you'll hear from me again soon, but you can always reach out any time, OK? Have a good day.

Enjoy your lunch.

All right.

You too.

Take care.

Take care

OK, so welcome back.

Day 3, being management.

Um, I had a question for you around like we just did a little bit of review on some of the OKRs and business problem statements, so sorry that wasn't recorded here, um, but getting a lot more user focused here, starting to understand a little bit more of the connectivity of what are we looking for in behaviors and changes to the user focused, especially as um we ran into and highlighted.

Some things here on um uh your XC XP bike and um these weekend trail riders who might be going and purchasing them.

Um, so the question that came up for me related to homework like you flushed out uh your OKRs a little bit more.

You actually came back and revisited your business problem statement to try to get a little bit closer.

We even came up with an additional persona there, there's more than just an urban commuter who's riding your bike.

Um, did you talk with anybody in your organization? About any of these business problem statements or personas or any of these au cares.

Not yet.

Yeah, I would only say the persona is based off of, well, obviously the company we got plenty of weekend trail Raiders, so the persona is based on people I know, but no, I didn't directly talk to anyone.

Matthew, No, OK, um, so it's, it's good to dive into, right? Cause as well, these start to start to navigate the story, and they help us make sense of a little more of what we're gonna get into today, uh, because if, if you're If other people in your company see these problems as well, right, the OKRs start to make a little bit more sense, right, of like, is this, um, are these Are these things that they can see being measured? Are they results that even seem to make sense? Are we, um, are we shooting too low, and we should be more ambitious? Are we overly ambitious? We're not even sure that there's even a chance we can hit that yet.

Um, we might discover more of that today, but, you know, those kinds of things, and just that conversation can invite, um, new insights into where you might want to take these other missing.

OKRs perhaps other missing aspects of the persona, uh, that kind of bit.

Earn, you said you have people that you know who may fulfill this this persona of the weekend writer.

OK, so that was actually the other part of it is, where would you find these people, right? Does this persona even exist? You already know a few that are out there, um.

I would encourage you in the future.

Talk to them.

Maybe you think about like, can you talk to them about some of these items and and like how would they buy your bikes? What do they care about in terms of their ride experience? Um, you can really start to learn more about maybe some of the missing obstacles that are out there, or missing needs, things that they have or want, or like, you know, is it, do they actually care if it's easy to clean and maintain, right? Cause like it's gonna get dirty all the time.

It just needs to work while it's dirty.

Right, I don't know, um, right, the validating the assumptions, right? Validating the assumptions, yeah, yeah, trying to get through that, um, because that's, that's really where we're at here in this class.

We have, we've been going through lots of assumptions, right? Staring at some very high level of assumptions and starting to constrain that problem a little bit more at a time, over and over again, um, to a point where we can now start focusing our discussion here.

It might make it, it might be difficult.

You know, to prove these things out, um, but definitely more focused in terms of, is this really a problem? Is this the right problem? And how and who could we talk to to find out? And how would we know if um if this is really a problem with them, like, will they take this action? Could we see this key result happening? Does this outcome, uh, actually exist and matter, right? Or is there something else that's going on? So, You know, we are trying to help figure out as well, like, if we're gonna make all these guesses, there's so many different ways in which we can go and solve the problem.

Right? Let's not waste our time working on too many different things that could just be wrong.

OK.

So with that, let me bring this over today.

Um, so here's what we're gonna talk a little bit more about today about how do we know that we're right? Um, so we just did some of the review, uh, and some of our thoughts and questions.

Uh, we're gonna start today by talking about solution hypotheses, right? Um, that again, these are, we're still going to make some assumptions, assumptions that this thing that we could go do, this solution to the problem, uh, this, this output, this activity, whatever, actually helps us, uh, drive the desired behaviors that we're looking for with our customers.

Right, to create the appropriate outcomes and impacts for both them and our business that we're driving towards.

All right, so, um, hypothesis statement, we're going to focus on a specific format here in this class.

We've done that a couple of times, right? Just an opinionated way on how to write a hypothesis statement.

OK.

And so in this case, um, it's just a we believe, right? We, we think, we think it's going to happen.

It's still our assumption.

We believe that this outcome will be achieved if these users successfully obtain this user outcome or benefit with this feature solution, OK.

Uh, so in this case we're talking, oops, it's funny how that disappeared.

I'll have to pull that, there we go.

Um, we're focused on the key result, right, the outcome, the thing that we're looking to see happen, the, the, the, the activity we could measure.

Right, um, will be achieved if these users, what's our persona, who's involved here, uh, attain this user outcome or benefit.

What is the, the need or obstacle.

That we saw in our personas, right? Their, their goals, their objectives, their job to be done.

What are they trying to go and achieve there, right? With this feature solution, um, the feature solution is the only thing we haven't talked about yet here in this class on purpose, right? Um, we, we've maybe had lots of ideas and I'm sure, you know, you've got many things going on.

I could do this, I can make this so we could we could do these things to get this to happen.

Uh, but we haven't really codified any of it yet.

This is where we're gonna start to get there.

So we should, we already have the cure results, the persona, and the persona's goal, with which to start filling this information.

OK.

Um, I'm going to show you another place in which we can put and use that here in a moment.

Um, so as an example, right, we believe that the number of accurately completed and submitted mortgage applications will increase by 35%.

We had that the other day in, um, as an example in the business problem statement, right, that we want to see an increase by 35% in submitted mortgage applications, or I think the, actually, maybe we need to make a tune to this, a tweak to this because we had 20% increase in submissions.

Uh, if repeat home buyers, that was our persona, right, feel prepared and confident in their application, what they want out of it with an email-based mortgage application process.

Ah, versus the solution.

All we said before was a simpler application process.

Now we're talking about maybe it's an email-based mortgage application.

OK, um, so this again fills in all of those key pieces there.

What's the objective that we're looking for, right, the, the, um, the key result that we want to see happen with with whatever outcome there, who's involved, what matters to them, and how could we go about actually starting to solve that problem.

Right.

Um, now, last little bit here and then I want to go back over to you all for some questions and such.

Um, you already have, like I said, most of this information, but I'm going to point out something else in our, um, on the mural board and another tool that you can use to help you organize it a little bit more.

Uh, so what this is showing uh for you is, um, the lean product canvas, uh, which we do have a copy on our board, we just haven't put our information there yet.

Um, and it, and it highlights some really key things, right? The users or customers, right, that's one part of it.

Here's our persona, Michael, we could see Michael, right, the ops manager, Michael, the ops manager, Michael, right, we just grab that and repeat that sticky over and over again.

Um, what are the things that they're trying to do here, accuracy of uh rate of post interactions, things of that kind of nature, those are a little bit more up in here, right? What are we trying to get out of it? Um, we can see the user benefits, right? Reporting tools show accuracy, these could come from the user benefits side or, oh, I'm sorry, um, they assure that our data is accurate, um, I can improve my cost, etc.

those who come from these user benefits section, right? So what are Michaels, needs, and obstacles, we could start seeing the user benefits, and then we start crafting solutions, which is another box for as well.

So what that looks like on your board is over here, the lean product canvas.

Let me just highlight into this, um, most of the things that we've come up with so far fit in here, right? So the the business problem statements that we've crafted go into box one, right, the most important problem that the business needs to focus on as it relates to what matters towards our customers, etc.

um, what are the outcomes that we're looking for that's also, uh, as well, um, it, it stems from our business problem statement but starts to come into the ORs, right? What is the main thing that we're looking to see in actions and measures? Uh, and measurable actions we take.

So, um, box 2 is, is that first, we believe that we can achieve this business outcome, this part, right up here.

The users, our personas go here in box 3, right, who's doing it? Um, the user outcomes and benefits, the jobs to be done, what are they seeking out of it? What's their need or obstacle? What do they want, uh, comes in box 4.

Why do they care? Our solutions will start to fill in box 3.

What are the potential ways we can get there? And then we're going to focus on filling in essentially box 56, turning those things from 234, and 5 into hypothesis statements.

And then how do we go learn, right? What do we need to learn, how are we gonna go and do it? We're focused on the bottom part of this canvas here today.

OK, so I'm gonna pause there before we get into our next little bit.

You, what can I help you with now? Questions or insights into uh the the solution hypotheses, their structure.

I think I'm kind of just uh waiting to see how we start applying it for any questions come to mind.

OK.

I think Nope, no questions here.

Sorry, I was on muting.

OK, now I just sometimes it's an indicator you got something that you're ready to talk about, so that's why I called you.

Shall I see you good? or you have anything you want to chat up.

OK.

Well then, let's do that.

All right.

So here's what we're gonna do next.

Um, we're gonna write some uh some solution hypotheses.

Uh, what I'd like to do is give you all a bit of time uh to focus on, on bringing that together.

So, um, And let's see, where is it on the mural? It is number 6, feature hypotheses.

We're going to start to figure out what potential features, things you can go and build, and then I want you to specifically use that format.

We believe that our business outcome, what is the thing that you're looking for, right, um, can be achieved if these users pick your persona, can achieve this goal or outcome, what matters to them, right, with this feature by take some of the feature ideas we have and start putting them in there.

Um, so just give that that structure a bit of a whirl and start seeing what you can come up with, right? How many different hypotheses might you develop on that either leverage different features or focus on maybe different goals or outcomes for your users, those kinds of things to start crafting hypothesis statements, OK? Um, I'm gonna give you.

15 minutes to work on these.

We'll come back and talk a little bit about what you've built.

Um, if you need me, call me, I'll be here, but otherwise I'm gonna let you all get to work, uh, and work with each other before I go and contribute any further.

That sound OK? All right, so, uh, I'll be here like I said, muted and off camera, but 15 minutes for you all starts now.

Well, we want to talk about our XC and XV features.

Yes.

It's an easy one.

Just kind of started thinking about what's currently on the XC and XP platform.

Replaceable batteries is one thing, I guess, uh, you should specify.

It's not just replaceable batteries, it's Hot solvable battery is the, yeah.

Plug and play.

Yeah.

Um, And I guess we do advertise it's kind of like our low power, it's it's our X lineups, that's like the.

Catered towards, I guess not say lower power, but you know we're not talking like full on adventure.

Is that something that we consider a feature? Yeah, I mean we could might say, you know, lower power supports newer writers, right, they're less likely to send themselves into the surrounding bush.

Ma.

We can say, does XCXP have configurable renas? No, right? No, I don't think it's a different, yeah, they're just the different r modes.

Yes Um, I'm looking at the copy from.

From our website purpose built for maximum off-road fun and most affordable in its class.

Category leading performance in a compact lightweight design.

Intuitive throttle response easily swappable battery.

It's perfect for riders seeking thrills without.

Shifting gears and delivers a simplicity, reliability, and fun factor you've come to expect from zero motorcycles.

Yeah, that's definitely feature no gearbox and clutch.

Let's see, I should probably open that page too.

Compact lightweight design.

So we're talking about design, affordable in its class.

I mean yeah, when we think about lightweight, you know.

In the event, the writer does mess up it's a lot.

Less painful to recover, you know, if your bike is down in a ditch somewhere and you need to go fish it out yourself.

Mhm.

Laptops touch pads with you.

Factory tuned suspensions switchable traction control.

It's not All right, let's try making some hypothesis based on Features we have.

So, We believe that's.

What do we believe? what kind of business outcome we're trying to achieve? My gosh.

With something like saying um You know, we believe that our You know, our pre-order conversion rate.

You know, an increase in pre-order conversion rate can be achieved if The weekend trail rider.

Can use hot swappable batteries to mitigate concerns over low range.

When they're out on the trail.

How does that sound? I, I didn't get the train of thought, so your logic is that.

If they get to experience it.

Then We'll have better conversion.

Yes, right, they, they can use hot so they have a feature that allows them to.

Have that outcome or our business outcome.

OK.

Perfect.

Would that extend to Everything else too.

I'd say and I, I, I think that that is even more true with uh ride mode.

And lightweight design, because that's something that we can't convey.

In ads in any way or or online in any way.

Right.

But I think we should type that at least someone's already typing.

that up if you want to modify that box.

What I was thinking was we believe that.

Alright Yeah, cause as I say, it's not quite a business outcome, but right, mitigating the You know, the, the customer's concerned over low range.

Yeah, it's not quite, is it? Is that considered a business outcome? Or is that just a function of Something.

Hey guys, I'm gonna pause timer for a sec.

I think you are, the question you just asked, Erin, is important.

Um, I think you're missing a piece of the format.

Take a look at the format.

Take a look at what you wrote, and tell me what part is missing.

Is the goal or outcome? Yes, right, for the user's goal or outcome, right? So, um, you, you, it's more like we believe higher conversion rates can be achieved, we believe that we can capture more of the e-bike market.

Um, can be achieved if we can show riders.

Um, can reduce their anxiety over or their their range anxiety.

Uh, by leveraging the hot swabble batteries.

That makes sense? Yes, OK, so yeah, so like just you know, slide that other portion that you have in the middle up a little bit and then like you can achieve it um for the we can show writer by, you know, um, so they can achieve what? What did what did they achieve, right? So we, we believe like.

Our business outcome can be achieved if those people can do what, like what if they, if they can achieve what outcome for them through your solution, OK? Um, so with that bit of feedback, I'm gonna go back off camera and restart the timer.

Good luck.

Thank you.

Adam take over, so I, so I can watch.

Sure, right, so.

posted right now the text, the text reference.

Right, we achieve our conversion rates of 2, right, so the user behavior outcome we want we're concerned about is, yes, Rich just said.

Can mitigate um range anxiety.

And we can say, you know.

By experiencing the convenience of hot swappable batteries while out on the trail.

Cool, so now we have something for hot swappable patterns.

I guess didn't he mention that we should like the business outcome isn't necessarily the conversion rate, but I guess being more prevalent in the dirt bike market.

Less so about the conversion rate themselves.

I mean we could tie the two together, right, cause Yeah, at least in my mind, you know, the understanding of, you know, achieving a stronger position than the dirt bike market.

Can be done through con you know, increased conversion rate.

Cause at least in in my mind, right, just saying we believe that we can be.

You know, the Well, I guess that works too, you know, saying like we could be the best selling dirt bike.

Um, If the weakened trail rider can mitigate the range anxiety through the feature that we just explained.

That definitely works.

It's No one dumped one.

I guess the other thing is that is it a problem just continuous to reuse the same business outcome, or should they be different? at least in this case, you know, I've narrowed in on the business outcome of the conversion rate.

Um, so I guess that's kind of where Matthew is more.

At least in my mind, the more abstract goal of becoming the best selling me dirt bike.

Could be better used.

Yeah, I'm not sure if it's a business, but what I was trying to type up in that post to the side is, is kind of.

I guess one thing I was looking into is like as like an outcome is that we're like a leader in.

Um, I guess like the casual ride experience, um, with like the XBXC lineup, not necessarily dirt bike related, but more so catering towards the.

The the weekend writer like the like in the casual writer.

But I don't know if that melts too much into like the users themselves and kind of.

Well, so you, you're you saying that gave me, you know, a good idea to say, right? We believe we can.

We can achieve the best IC to EV conversion rates.

By highlighting, you know, by advertising to newer writers that, you know, the convenience of the lot, you know, no gearbox and clutch.

Or the experience of not having a gearbox and clutch while out on the trail or something like that.

OK, that's IC to EV.

Conversion.

I guess.

Well, one thing I'm not sure, 11 thing I'm not clear about here.

If we make a hypothesis, are we making the hypothesis about a lagging indicator or a leading indicator? The reason is with a leading indicator, we can actually test the hypothesis.

If we do this, something might happen.

But if it's a lagging indicator.

We can't test it, but it's a true hypothesis in the sense that we'll do something and then the market will respond and after the event, we know if it's true or not.

I'm not clear about that.

So, uh, like the conversion rates.

that we are the best in class in Uh, in, in the dirt bike market, in the dirt bike segment is A lagging indicator, yeah.

But conversion rates, we have some control over.

Uh, wait, even the conversion rates are lagging in a career.

Yeah, cause then I'll determine if the feature where advertising actually matters to.

Yeah, leading indicator would be, let's say the number of demo rights.

Oh, no, now I'm confused.

Is the hypothesis.

OK, so this one says the format says.

The business outcome.

So if it's the business outcome, then we, we are essentially looking at a lagging indicator.

So both of those, the, the ones that you said just now, both of you said, I think are valid.

Although the ice to craft one trying to come up with.

OK, so we have a a little bit of questions there at the end.

You wanna dive into that a little further? Yes.

OK.

So, um, you, you highlight an interesting point.

The things that you're currently focused on as business outcomes are more lagging, lagging indicators.

In fact, what you're really looking for, what you're sort of putting there about higher conversion rates or like becoming the best selling e-dirt bike is more of a business impact.

Right, like, you know, it's, it's the if we create, if we increase higher conversions, right, if more people buy our bikes, that's a lagging indicator, we'll become the best selling e dirt bike in the market.

That's an impact that we are now, we are now the leader in the space.

Um, there are many other ways to become leaders.

We can buy out all of our competition.

Right, we, they could all just go under, right? There, there are other things that could create that, but, um, so I think you've got a little bit of both of them showing up here.

It's not horrible though, right? It's showing why uh that matters, but you are talking about, um, conversion rates is the business outcome, which is a bit more of a lagging one, and that's not again necessarily a bad thing here in this case.

Um, so like, You, let's talk about leading ones here in a second, but like if we talk about uh the, the user or the the example before it was that we believe the number of accurately completed and submitted mortgage applications will increase by 35%, right? So it's the people actually doing the thing.

So, uh, not just that, and, and the case in, uh, the one here is that um we can showers will buy our bikes, right? So conversion rate is, is the important aspect, but like that will increase the, the number of pre-registered.

Um, uh, you know, the, the number of pre-registrations into actual sales, right, the conversion rate by X, um, if, uh, if we do these things.

So the only thing that maybe is missing here is like what's the, what's the number that you're aiming for? Like what's the, if we're thinking about the outcome there, it should be a measurable type of, you know, something measurable that we can look at, um, and what's happening, so we know that they're converting by how much.

We believe that we can achieve a conversion rate of 60%.

And become the best selling e-bike, you know, there, uh, if we can show riders can mitigate the range anxiety by experiencing the convenience of hospital batteries while on the show.

Um, so let me, before I go into reading, um, Kind of story does that tell you? Well, yeah, it's telling the telling the story of No, perspective but the right, the weekend trail rider.

Maybe he's not familiar with EV's who, yeah, is legitimately concerned like how long can the battery last if I'm out and about, you know, they don't want to become stranded out in the middle of a trail or cut their day short.

It's in my mind.

Good, Matthew.

What about you? Um, Sorry, I'm still trying to, I'm not the best at this still.

um.

I'm sorry, is this question catering towards the, the hypothesis we wrote or is this the? The one you wrote, yes.

I'm sorry, my brain is just not working right now for some reason.

OK.

Um, um.

So, I, I ask like, what, what do you get out of that story, right? Cause I'm kind of curious, like, here's a, here's a quick thumbs up thumbs down.

Does this feel like a good story to you? I just feel like, like something that's compelling.

Yeah, I I mean like this looking from like the experience that we the looking at the future.

Um, I guess mainly counter about the whole I this maybe just my.

I guess my experience in with our with bikes and these swappable batteries in the past is that there's still kind of like, you know, the, there's still a concern about.

Yes, I can swap out the batteries, but for the batteries, I swap out, how long does that take to recharge it? Um, so I guess there's still some.

And there's like gaps, but like it seems like there's other not not a complete solution, yeah, it's not a complete solution.

Sure, that's the main thing I'm seeing from this.

It's, it's part of this, it's, it's a way to solve the problem, right? Yes.

Right, um, and that's fine, where we have, I think you, you listed up multiple different features.

That was just the one that you focused on here, right? Um, so hot swap batteries, different ride modes, perhaps also changes, um, the, the, the quickness of which you consume that battery.

Uh I like that you talked about, um, Uh, when you're talking about different ride modes, you're saying like, do, do we really need this to be like the torkiest thing that's out there because like if you're a new, you know, uh trail rider and the bike only just jumps away from you, like how turned off for you, right? And that ability to one also, um, you know, like different ride modes highlights some interesting things.

It, it makes it easier for new riders to get on, right? It also uh perhaps changes the amount of um of, of energy that you consume.

When you're writing, which also helps with, you know, range anxiety.

Both actually probably go to the problem, you know, like the job to be done of um a user wants to go and have fun in nature.

Right, and, and providing a way of like zipping through on a on a motorcycle on a trail, it is fun, right? Or it's a way to create a thrill in nature, right? We might have the right product for that job to be done, and the different ride modes actually kind of helps solve that in different ways and and target some different kinds of things, both range anxiety and um I'm, I'm a novice.

I don't know what the heck I'm doing.

Uh, you also talked about that in terms of gearbox and clutch, which I thought was really cool too, right? It's, uh, I want to just go out and have fun, um, and without having to worry about like, how do I ride this thing, right? If you can, if we can make it easier for them to get on and actually start writing because we have no gearbox and clutch, that makes it easier for them to achieve their objective of going out and having a good time.

Right, so I think what what you honed in on here as well is actually the much more specific like I'm, I do this thing a lot.

Like I'm a trail rider who goes out a lot and I've got anxiety like about giving up my my ice bike for an e-bike.

Which I think there's I think there are two, sorry, go ahead.

I was gonna say is, is it kind of in a sense kind of like at least in the framing that we currently have, we've kind of more focused directly onto like a user problem versus what their desired outcome is.

Yeah.

Yeah, like the, the, the user problem is perhaps range anxiety or lack of control, right? That's a problem that they have, um, but, but the, the thing that they're actually trying to achieve is different, right? They want to stay out for a long time and they want to go have fun, right? And I want to, I want to, I want to come back out.

I want to have enough fun that I want to come back out or stay out longer, right? Those kinds of things.

Uh, so I think you stumbled upon that in a good way.

Um, sure.

So there are, uh, so if I just look at the one that, um, we wrote the, we believe that our we can achieve our conversion rates, uh.

What, what I'm pointing at right now.

So there are two parts to this that I can see.

There's the There's the content and there's efficacy of the content of the idea, and then there's the efficacy of the hypothesis, the way it's been framed, right? So the, the content of the idea that uh we can affect the conversion rate by letting people experience the hot swap of the battery.

That might be 10 of us based on our experience so far.

But we can't be sure, right? That's, that's not, we're just going off of what our opinions are.

But if you look at the framing of this, every single part of this can actually be tested.

It's a valid hypothesis and every single we can, can it affect our conversion rate.

We can check correlation.

We can do a survey before and after to check if, uh, check how people are feeling with um their range anxiety and, and the fun they had on the right.

And third, we can see how many people actually showed up to, you know, experience it.

So almost every part of actually every part of it can be tested.

So from a, from a framing of the hypothesis, I, it looks like a great framing whether or not it's a result that is in aligned with what we expect.

I, I think we should leave that to the experiment so it doesn't buy us our experiments.

Yes, absolutely.

Right, they, and the, the, you're going to get to some interesting experiments in a minute, um, but you're right, like the fact that it's testable.

Um, makes it easier to connect to, right, and makes it easier to like to find out what's this the right idea.

And it might point out that actually they don't have range anxiety.

There's something else that's going on, right, wrong, and we can learn more about that.

We can tweak our proto persona a little bit further based off of the experiment that we run for the solution that we think will get the job done.

Right, and realize, actually there's a different job that seems to be emerging as a theme that we need to keep that we need to come back to.

Um, understand how we measure that kind of outcome, like what would, what would it look like to solve that problem and then come up with a new hypothesis.

So, so if I'm understanding this correctly.

Uh A bad hypothesis is the one, is, is one that we can't answer yes or no to.

If we can if we can figure out a way to test it and then get to uh to a yes or no uh answer for every hypothesis, then it's a Then it's a good start.

And, and then after that, it's about optimization.

Do we have all the resources to try every single hypothesis? Yeah, obviously not.

So we, is that fair? It's pretty close.

Um, my, my thoughts are, uh, on that are two things.

One, the more, the more specificity you put into this to derive that yes or no result, right, um, helps you understand more about the hypothesis.

And so I think the one thing that I, I see missing in your hypothesis is what's the conversion rate you're looking for.

Right, because that uh we can our conversion rates of 2%, 40%, 90%, like what that that adds that specificity makes this more testable and it makes it more of a definitive yes or no kind of answer.

And we can put in 90% and get to 87%.

And then the result is like did we prove that to that hypothesis to be true? No, but we also saw a lot of good interactions, and a lot of good data, and maybe are just our assumptions on the fact that we'd hit 90% were wrong, but we weren't wrong by a lot, right? So it was a it was a no, but a good no.

In that case, and sometimes the no is like, you know, also a good no is no, like, in fact, like we're only getting to 12%, we really um we need to get conversion rates of 90% to stay in business.

We need a shift Right now, right? So when you have that level of specificity, and you can get to those yes or nos, um, it helps you make other decisions within your business.

All right, so that's one part of my two part answer.

Questions on that? Uh, I, I think I'll hold my question for now.

Um, OK.

So the other piece goes to what you were saying or what uh I think we're just talking about of maybe what we find out is that our solution.

Isn't the right solution.

Right, that the feature that we came up with isn't it isn't the thing that's actually doing it, that the outcome that we're looking for, um, still matters to us and to our writers, uh, and we, and we know that this is we're actually holding on the right people, but we didn't find the right thing, um, and so it it's, it's like a, this is, this is good.

We don't know.

Like people aren't saying yes or no, like they're, we haven't seen any, any shifts or whatnot.

Um, it doesn't, it might mean that the feature was wrong, right? And so it could be towards your, your understanding, is it a yes no kind of thing.

It could be that the answer is no, right? Like we've we've disproven this hypothesis.

It doesn't mean that we also built a a bad thing.

It just means that we also haven't stumbled on the right ones yet.

So it's, it's almost more of like an inconclusive.

Right, like this thing maybe made some shifts, but there could be other factors.

So again, probably specificity helps um eliminate that, that if there are other things that could influence conversion rates, um, like it's they actually they don't care that it's, you know, hot swivel bikes, it's the fact that, you know, you're charging $90,000 for a motorcycle that people use for fun, and that's just way too much money for them to your bikes don't cost that much if I recall, right? I think they're like a lot less than that.

But it was funny.

I was, I was talking with a friend of mine, um, on Monday, who, uh, I think I said, you know, his, he has a friend who has an e-bike who wants a zero motorcycle.

I just, this is, I think the way he put it it was like you're the gold standard.

Then I'm like, well, and I'm like, why did she have it and they like he was like, oh no, I just wonder if it's too expensive.

It looks, no, actually that's pretty comparable.

It feels like that feels right.

It's just, I guess is that the toy that she had or she just didn't know at the time, or like what was the problem? And so there again, there are other things that could influence your conversion rate, that maybe you're seeing, well, there's movement here, but that's not the only thing.

So it's inconclusive in that we We don't, we can't just test one hypothesis, perhaps to meet the objective.

We need to actually test numerous hypotheses, um, and figure out which ones we need to test in which way.

Yeah, that, that was a question I was gonna ask.

I uh what I was gonna ask is it, it doesn't.

It seems like we need to have a first we need to have a whole bunch of hypotheses listed out, framed well enough that we can test them, and then we, we probably need to prune that down to a set that we actually want to test.

Not all of those can be testable or even if they are testable, we might not want to do, or we might not want to test all of them.

Um, so there's some pruning.

Based on our previous experience that needs to happen.

That was the question I.

In fact, that's the next part of our content.

So, uh, I'm gonna pause, uh, I just, I'm gonna jump into this for a second, and then, um, we're, we're gonna pick a hypothesis to work from in the next section.

Uh, so I'm gonna give you just a few more moments as well to see like is there any, after, you know, all this discussion, are there any other hypotheses you want to work on, because you're gonna pick one of them, right? And if you like the one you have, great.

And if not, I'll give you a few minutes at least to, to see what else is there.

Um, so we also have the hypothesis hypothesis prioritization canvas.

So it's just a way to look at, um, and actually that URL at the bottom is wrong.

I told Jeff, but I, I'm like, it's not question marks.

You just search for it and you can get there.

Um, I can give you copies of this too.

I'll put them in our uh authority flow.

Um, so the Rich on that note, so the slides that we have in the, in the link that you sent us.

Those are, I think, much older.

They don't, they look nothing like this and there's also some content that's actually missing.

So let me tell you a little bit about that.

Um, yeah, I will give you a copy of the current slides, um, post class.

I'll make sure that they get up to, um, you know, to clarity flow there, um, but they just rebranded Sense and Respond with like a whole new feel and look to it and made changes to that material this summer.

Um, Josh just flew from Brooklyn to Barcelona to meet up with Jeff and do some work, including the face to face that I'm going to this weekend.

Um, so part of their, their other thoughts is they, they said, yeah, we know the videos are up to date.

We probably need to re-record them and reupload that material and, and do it.

Um, I think they've, they've just gone through enough change that that had been on the back burner, and now it's getting to a point where it's like, oh, this is a problem.

The content's still right, but like good, but it's not up to date.

And so, um, yes, that I will bring that feedback to say we have real customers who are noticing that there's a drift.

Can you guys please re-record the videos? Um, so, but, and then, yes, I will definitely give you a, a copy of this material, uh, for, for post class, yeah.

So, um, on this prioritization canvas, there's a couple of key things you want to look at, right? Um, what's the perceived value and what's the perceived risk, right? And again, these are, these are guesses in some sense, but they help us understand some things and as well, the reason I asked about the story, like that compelling story is because imagine you're telling us to your, to an executive, because you want to get one of your hypotheses funded to go learn to actually create an impact in your organization.

Um, if, if we believe it, they might believe it too.

If we look at our our hypotheses and go like this doesn't quite make sense, it's not gonna make sense to anybody, OK? And part of it as well as understanding a little bit about like, well, what's the value out of this? Like, what could it actually do for us? And then also understanding like how risky is this? Like is it risky to do hot sopable batteries? Um, maybe it's probably more risk than, um, you know, uh, configuring a ride mode, but again, I don't know.

I, I just software to me seems easier.

Um, maybe it's the opposite direction.

Uh, and, but they could still both create some high value if they can create that conversion, right? So if it's, if it's low risk, high value, just put it out there, right? Like just go for it, um, because if you're correct, you're going to create a a a really big business impact there.

You're gonna create a lot of value for both your customers and your organization.

Um, and that's great, like we don't really need to discover, just, just go do it, because if it's wrong and it doesn't create the value that we're looking for, at least it was cheap and it was low risk and like either, either like it was low risk that we didn't have to invest much into this or as low risk that it didn't create a a big PR nightmare for us, right? So who cares? We learned something, move on, try the next thing.

OK, um, the ones that are high risk, high value, um, are the things that we, we want to get more data on.

And like, we're not sure yet.

We, this could be a challenge to us in feasibility, this could be a challenge in terms of, um, like the actual markets and what's going on there.

There's a lot of various risks on that.

We have to probably try other experimentation techniques, uh, some other prototyping.

Other types of um of interviews and, and interactions with customers to learn more about if this is, if this is worth pursuing further before we really just put something out there, right? And then when it's low perceived value, this is like when we're not really sure if we should actually do the thing, right? Uh, low risk, low perceived value, um, we don't again need to test much, we could probably just go do it, um, but they might not.

Then it might not matter to our customers or to our business.

And so it's really a question of do we need to do this? Why is this an option on the table? Is it a compliance piece? Is it to make one shareholder happy? Uh, is it, you know, a um a very noisy customer talks about it, and they could influence us, but then again, they might not like.

You know, we just, we want to learn a little bit more there and so it's again this thing of, we could go try something, but we should really ask, is it worth it before we do, or just, you know, say like have a conversation with that noisy person be like, I, I see, I see the problem, but what else could we be missing, right? Or, or inform them more about the broader market that you're trying to, to help.

And then the things that are low, you know, low value, high risk, just don't, don't do them.

If it's not going to create a big impact for your customers.

Uh, or your business, and it's going, and it could potentially cost you a lot.

That's not a bet worth placing.

All right.

So, uh, let me then come back here for a moment.

Um, so yeah, here's what I'd like to do.

I, I like to say, you said you've got to, um, I'm just gonna give you a moment to like another 5 minutes to craft any other additional hypotheses you can think of, uh, because, um, after we get through our next little bit, we're going to pick one of them to expand on further.

OK, so try again, think about, um, I really liked how testable are these? how specific could we, could they be, how yes no of a decision could they lead us to.

Those are all really good points to lean into as we start thinking about what is the overall business outcome and and and measurable outcome that we want for us that impacts our users, um, what is the thing that they are trying to achieve, um, and what are the various features.

You've got a lot of them.

That uh perhaps you want to lean into some other different ones to see how they go, OK? So, uh, 5 minutes before that starts now.

Well, first, what kind of hypothesis would you like to see now then? No, I, I added one on the thrill of the ride mode thing that we discussed earlier.

I was thinking.

I'm We could add one more on affordability because like what he said just now, somebody didn't even know, right? So I'm.

Do people even know that these are affordable, or do they see it as something else, or do they even know that it, yeah, do they just assume that it's electric, therefore expensive, like the rest of our products? Yeah, yeah.

So what, because all of these are about demo rides that will need something else.

So I, I wanna just add that.

So I added this one for.

Um, What This one was earlier.

Yeah, that one was just a copy.

Yeah, I wanted the copy for the affordable one.

So, OK, now I can go and get for that.

Not we can trail riders still can achieve.

They're trying to make one about the the sale, the sale price.

Yeah.

What is the feature there, that it's.

With its price competitive, I don't.

Technically it's a feature, right? It's comparable to the rest of the market.

Comparable or cheaper then? I just wrote.

Something in A non-English first language.

You, you can read it.

It's, it's well.

I'm gonna put it in some other color.

Here Whoops, somebody locked it.

OK.

Well it was locked it right now.

What does it Um, achieved if.

So this has to be a weekend trail rider, right? Because we're only talking about XCXP.

Can we use it for urban commuter? Um, well, actually, no, we can't in the EU cause it's street legal in the EU.

OK.

Urban commuter.

Yeah, I use.

I mean, I guess with the With the user outcome in this case be that, you know, They feel like they're getting, you know, they want a You know, whatever the value proposition, you know, a good value proposition for.

You know, if if a user is cost sensitive, what is their Was that was their desired outcome when they buy something that it's worth it, right? They need to have an understanding of what they get for the value or All they need to, uh, or we need to create enough desire.

That It overrides their their budgeting aspect.

Yeah.

I, I'm just gonna put something here uh so we can decide which one we wanna do.

You, you both can decide which of the other three.

I just want to complete this one.

OK.

Um, so here's what I'd like to do next.

Um, let's break until the top of the hour.

It's about 10 minutes.

When we come back, we're gonna start figuring out how do we learn about our hypotheses and what experiments might we try.

To see where these things go, OK? Um, so, uh, feel free to take a break, keep writing if you want, but, um, I'm gonna pause the records.

OK.

So now we're back, uh, to talk a little bit more about testing your hypotheses.

Um, as we looked a bit before, we've got 4 different hypotheses listed here.

We're gonna start highlighting and honing in on, on, let's say one of these to start testing out, uh, here in a moment.

Um, maybe we can, you know, think about which maybe is, it gives us some of the biggest impact, uh, maybe has the right amount of risk, uh, to get in there so we can start figuring out how we can go and test this out.

OK? Um, so, my next question for you here is, um, MVPs and experiments, um, MVP, have you heard that term before? OK, cool.

Um, what is an MVP? Minimum viable product.

Uh-huh.

Minimum viable product.

So here's what I'd like to do next, um, where I take 20 seconds.

I'd like everyone to pull up chat in Zoom.

And you're gonna type, but don't submit.

Uh, until I tell you to, um, over the next 20 seconds, what does a minimum viable product mean to you? OK, so start typing your answer, but do not send it.

There it says, what does minimum viable product mean? Bye.

321.

What you got? And then are you still typing? Yeah, I'm just finishing up my, my thought here.

OK.

So what, what you got? Let's see what she, what's there so far.

Um, we gonna build to test their core hypothesis, uh, early representation.

Matthew to you, what does MVP mean? Initial version of prose initial feedback.

OK.

Did not finish that grammarly.

So we have um We have a couple of different directions, right, that we go in here, um, but they, they both start, they, they talk about something small here, right, um, an early representation of our product, an initial version of our product, um, something that we can observe that helps us with the direction, and then she he said like this, the least we need to build, right? I just, not just an early representation, but the least thing that we need to build.

In order to test our core hypothesis, and that is kind of really where we where we need to hone into, right? The smallest thing that we can do or make to test our hypothesis, that's an MVP that that goes straight back to um the lean startup definition, right, which is the smallest thing that you can do or build to create the maximum amount of validated learning.

Right, and validated learning in this case is, is that sense of when do we see users taking actions, uh, often, you know, thinking of outcomes and behavioral changes, right? When do we see them taking actions and making an exchange of value, uh, for us and the thing that we think is valuable, right? So when we start to see that, that interaction, that validates the learning that we're going for.

So the smallest thing that we can do, uh, make or do to test our hypothesis, share us your.

Like I said, pretty much right there in the money, uh, answer is the question or the answer to that question is our MVP.

Um, I've heard so many different definitions of MVP over the course of my career, um, like it's the smallest thing that we'll build before we're ready for the market.

That's probably gigantic.

So, um, it doesn't tell us anything about just an early representation.

It's not about observing direction.

It's like, it's again a lot of assumptions on where we go.

So, um, You're closer than most of the people I work with that on on the term MVP.

We're going to focus more on the experiment side of things here, um, but those are again like that, that core definition of the smallest thing I can do uh to test our hypothesis.

That's what we're headed off to here, then we're going to figure out what is that experiment.

OK, um, so we're, um, I'm gonna walk you through some of the, the pieces here and then we're gonna get into the next exercise.

So when we're designing an experiment, we want to understand what is it that we need to learn first, right? What's the most important things that we that we need to be leaning into to learn, um.

Alright, remind me again, uh, Shay, you've read Lean Startup, Aaron and Matthew, you have or have not? I'm not.

I have not.

That look and sure you've read this.

Yeah, OK, that's a, um, so.

There's an interesting piece in there, um, we don't necessarily have to go quite this far in, uh, in terms of our learning, but when I think about learning first, um, sometimes the things that I want to find, if they, if they exist are the leap of faith assumptions, uh, as Eric put it in the book, he says, like, what are these make or break kind of things that like we have to take a leap of faith, it's until we try this and find out and learn.

Um, and we won't know if this is, if there's even any reason to pursue this any further, right? And so it's that we've really got to get this in front of people.

We've got to see if this is a market that anybody wants.

We got to see if they want, um, uh, an e-bike solution or if they, if they want to even be on a bike versus if they want to be on a four wheeler, right? There's so many different kinds of things that we need to, that we maybe need to learn that's like, there's no point doing all of the other pieces if we can't.

Uh, prove out that that leap of faith assumption.

Um, here we're gonna, like I said, we're going to hone in on some of our hypotheses, and we're trying to figure out what do we need to learn first in relationship to our hypothesis and then what's the least amount of work that we need to do to learn that.

OK.

So, for example, we can just talk to two customers.

It's a pretty easy thing to do, right? The least amount of work.

I at least go talk to two people who ride bikes, who ride our bikes, see what they think, right? That can give us some ideas.

Um, maybe we build a 3 question survey.

And send it out to 10 people.

It's pretty easy.

Google Forms, something like that, right? Just push it out there, um, could be who you send it to is interesting.

Are you sending it to? You know, your personas, do you, do you know those? Are you saying it to like grandma and grandpa because they're they always help you out on stuff and they're willing to take your survey, right? So we could send out to a couple people, uh, maybe we make a paper prototype and show it to people.

So, uh, I have an app and I have an idea about a change to the app that we can make, but rather than building the new app, um, I'm actually just gonna go and create on paper, here's what the screens would look like.

Here's what this would be, how, how would you interact with this? What do you think and try to get that concept out.

So people prototyping is an often cheap way uh to go and do something if, um, if building the actual thing is too costly, um, or we, we actually could go build it, right? Like if, if it's like a software solution, if it's pretty cheap to build, um, we can build and and perhaps even release two versions of it so we can AB test and learn which one drives the behavior that we're after.

That would be good too, right? All these things have different kinds of investments that go into them.

OK.

Um, Questions there.

OK.

OK.

All right, so here's then what we're gonna do next.

Why my slide come up.

Um, we're actually gonna go and get into this and pick one hypothesis test, and then we're gonna start breaking it down to, in what ways could it be wrong? Um, and what's the spons experiment that we could run, OK? Uh, so we have these same kinds of ideas up on the mural, so I'm gonna take us back to the mural, uh, and I'm gonna show you I'm gonna do this live with you.

Here's how we'll run this exercise.

Um, I'm gonna let you all pick the experiment, and then I'm gonna run through the questions and do the typing for you.

You can tell me what you want and I'll get it up on the board, right? So which of these hypotheses Is the one that you want to go and start working with.

Which one seems to be the most important hypothesis for you to go and test? I think the The constant the The EU can have one targeting the EU urban commuters.

Um, cause I think the value of understanding if the price actually matters or not.

Um, That's my opinion.

Yeah, I would say so too, because like the other ones would require like also using some like demo rides at the very least.

Uh, that, but for something like, you know, understanding the pricing and stuff, that, that's just, you know, that can be talking to someone who's on the fence about buying a bike, someone who is just kind of, you know, readily browsing, not necessarily also considering or someone who is actually in the market, like there's a kind of a a wider.

I guess audience that we can uh capture with that, even though we're focused on the urban commuters portion, like there's still kind of like a lot of different uh people that we can talk to in regards to that.

I would go for the opposite.

I'd go for one of the 1st 3 because it's complicated enough that I'd, I'd want to pick pickri's brain on, uh.

One of those, especially the part where how do we test it without building much.

That's a really interesting question.

Um, I'll let you pick whichever one you want.

I, I think the story you're evolving on the ones on the left is a more powerful story than the one on the right personally, right? Like you're, you're, you're talking about business revenue essentially on the right, and do people get it, but there's not a lot else to learn there.

Um, whereas the things that's like, yeah, you can argue, sorry for interrupting, um, but you can argue at the, the last one kind of really is table stakes if, you know, we price ourselves out then there's no point, right? It's kind of table.

Yeah, and but you also, you could learn that just by looking at your competitors pricing versus your pricing and start to see, you know, what's what's an acceptable margin.

Yes or no, right? And then there's not much else to learn there.

Um, But do they care? Do they know? Why do they know? How do they know? That kind of thing is interesting, um.

Towards experiments.

Here's something that I also want you to think of.

I am gonna actually ask you to go run an experiment.

If you're focused on the EU, can you engage with the EU before our next class? Yeah, I mean, we have plenty of.

Uh, I mean, Yeah, we have plenty of co-workers in the EU who write our products, diesel.

All right.

Just asking it.

So, which one, what's the one that we're pulling over? Is it the EU one or are you, are you trying to override and pull them towards you? You're gonna have to, it seems like maybe make a compelling case if you want them to work on one of the other three.

Uh, um, I, I still think it's, it's, it should be one of the first three because it's, there's some content in it that There's enough ambiguity in it that we need to navigate and that we need help navigating.

The last one is more straightforward.

We know what to build even for the last one.

Hm.

The 1st 3, it's not super clear, especially for the case that the situation where we're in, where we're trying to build out everything that we can think of, and we're relying on people to just give us a list of features.

Navigating one of the 1st 3 might be, in my opinion, that's that's the.

Best use of our time, but I'll leave it to the two of you because I'm, I'm gonna be gone in 13 minutes and I, I, I, I don't want to leave you with something that, uh, you know.

You don't want to work.

I'd argue for the first one since that's hardware inherently has in my mind the higher risk than the software, which kind of lead to the ride modes we discuss kind of more on the software side.

There's the hardware aspect of, you know, that design, I think will be interesting to run, you know, this experiment on.

Matthew, how you feel there? Yeah, I can get behind testing the first one.

OK, I'm gonna copy that.

Um, sure, I think you brought up a really great point.

This back one is like you, you actually talked about the experiment, right, that it is the feature, but it's also the experiment like build a web page that lists competitive price points.

That's a, that's a pretty easy and cheap experiment you could run to help them figure out the value for the price.

There's many other experiments we we can run there too, but that is a very specific experiment, not so much of a, a feature of your product.

And the other ones you really talk.

Sorry, sorry, go ahead.

I was gonna say the other ones you're actually talking about features of your product and, and if I recall from last week, things that are more directly related to things that you're doing.

Right, like your, your team maybe is more involved with the middle two than the one on the left, but the right one was more like this would be the marketing product, like the marketing side of it.

And to use your matrix that you had earlier, it's in my mind that's like low risk, medium value.

OK.

All right.

So, here's what we're gonna do.

Let me pop it up here.

That's the hypothesis that we're gonna go and test.

Um, so, first question, how might this fail? Like, how, what are the things that could go wrong here, um, and, and say that this actually, this doesn't work, this isn't this isn't possible, um, people don't even sign up.

People don't sign up.

Good.

What else? And this is in the context of what can go wrong with the experiment.

Uh, with the hypothesis, the hypothesis, OK.

Yeah, we don't have an experiment yet.

We're gonna talk more about the experiment in a bit.

We think the feature that that we want is hot swappable batteries on the trail, um, and that helps mitigate range anxiety.

Um, we don't have hot swappable batteries.

We don't, uh, let's say we don't have the stairs.

Mhm.

Where we run out of spares, spare batteries to swap.

A it's not even developed yet.

And this is even before development, correct? Right, uh, so we don't actually, it's not that don't have hot salt batteries, um, we can't.

Swap batteries.

Quickly.

Right, um, so the, the feature of Being able to do that doesn't exist.

Um That's good.

What else? We don't have a dirt bike.

Ah.

OK.

Let's say or we don't have access to a trail that's long enough.

To get some anxiety.

Uh, Or like the trail doesn't, if it's not long enough, like it's not like there's no elevation change, that would also effectively help us track the range.

Hm, um, so the trail might not be indicative of what users would actually deal with, right? Yeah.

Yeah, or like, yeah, trails like too paved or, you know, too compressed, and that's not.

Like a thrilling black diamond trailer on trying to run down.

OK.

I didn't type that right.

Um, OK, let me, let me do something here for a second.

All these are technical risks.

Can we do it? Actually, this is also a technical risk.

This is the only one you have right now is a market risk.

Should we build this thing? Like people don't sign up, people don't care.

What else might be going on in the should we build it side, like what, what matters to the market, the people that help, they should help you decide if this is even if not so batteries is even something you should bother with.

Yeah, I would have said, you know, that that technology doesn't exist, and I guess that's still a technological risk.

Yeah.

People don't know.

You don't know what.

No about um.

They don't get to know about our um.

Uh, experiment.

Actually, that's about the experiment.

So, uh-huh.

Um, They don't even know about, let's say dirt, electric dirt bikes.

Good.

OK.

What else? It's winter.

Say that one more time.

It's a seasonality, yes, um, ah, um, it's not the right season.

That's just a that's a timing thing, right? We have to wait a long period of time here.

Um, I might give you one that I thought maybe would come up, um, people don't care about range.

We we think they have ranging anxiety.

Yeah.

What, what if, like, regardless of how far they go, you can keep them entertained for two hours, and that's like physically exhausting enough, trying to ride their bike, but that's all they really wanted.

Or maybe you got people that are trying to be out there for like a 10 hour day, right? So range and time are maybe, you know, correlated here, but Uh, what, what are they doing? Are they like they ride around really quick and then they stop? And they hang out for a little while longer and then they go back to it, but that's enough.

Like, what's the, they don't care cause that's actually not.

Something that um uh you already are long enough on range.

You know what I mean? Definitely makes sense.

Yeah, that's.

One of the core assumptions.

Yeah, core assumptions that they get to a scenario where range matters then.

OK.

Yeah, so like these kinds of things, right, um, they don't allow electric bikes is more about the experiment and like, you know, so if we're thinking about this thing, right, we want them to experience it.

If they don't sign up, they won't experience it.

If they don't know about electric bikes, they're not going to experience it, right? Uh, it's not the right season that matters, right? But if they don't care about range, like this could fail because we can mitigate range anxiety, but they're actually, we find out that that's not.

An issue, right? Or um they don't run long enough to exhaust the battery.

That's another thing that that's actually not an issue.

That feature maybe doesn't need to be built.

We, so should we build this thing, right? Should we, should we, you know, when we go to test this hypothesis, should we build this feature here, um, those maybe impact us a little bit more, right? And then afterwards we can find out, well, actually can we build it? Like, is there even a way that we can swap a battery quickly? Um, or not in a bike.

Um, that's another technical thing that if, if they don't care, there's no point doing any of those.

Right, so I'm gonna, I, I'm gonna suggest we, we work on one of those ones and should we build it next, um, but like there's another, I was just listening to Josh talk very recently about the same kind of concept and, and honing in on experiments, and he's like, um, let's let's take the cure for cancer as the feature, right? This is, this is something that is all market risk, no technical risk, right? Um, because there, there's, if, if we had a cure for cancer and people could pay whatever the cost is, they would buy it, right, if they had cancer.

If it was like it took too long, it was too painful to get through, you know, it's only affordable for like billionaires or something, yes, maybe they can't, but it doesn't mean they don't want it.

Now we have to figure out how to make it cheaper, right? Like how do you, the technical part is the hard part there.

So it's, it's really all technical risk of how do we do that, right? So, um, when you're looking at at these kinds of things, the technical risk, can we build it, you know, we just try to figure out some way to build it.

No one's built it before, unless like physics prevents you from doing it, you can probably build it, right? So the question then becomes, do we even bother investing the resources? So should we build this from a market risk is a key aspect.

Um, are there one of those that you want to take further, or do we, with that in mind, of should we build this hot soluble batteries piece for um and getting them to do that while on the trail, um, is there any other should we build this risks that you're curious about, other other things that could make it fail? Erin, you're muted.

Sorry I didn't hear that.

No, even then I was just uh muttering to myself, being question to myself.

Here's, here's maybe another way to look at it too, um.

What would keep you up at night? Like if, if this was like a, gosh, you know, if we can't get this right.

Um, I don't know what what I'm doing here, like, or, or the, the secret fear, the thing that you, you don't want to say to your boss, right? Like, oh, this is a great idea, but you know, it's probably gonna fail in this kind of way and but I'm nervous to to speak that out.

Are there any of those that come up in this regard? Yes, for me, like something I'm thinking of is, um, you know, yeah, you can hot swap a battery pack in this case, or you know, in this scenario, but we don't get the owner would have to buy the the the other pack essentially.

It's not something that's included with the motorcycle, so I guess a concern I would have from that scenario you basically just outlined is.

Is the Is the ability to swap packs, does it improve the experience such that the extra costs required to purchase the.

The, you know, extra packs worth it.

Do people actually want to pay for this feature, essentially.

We don't want to buy a 2nd battery, that's a risk.

Right, I that that's, that's pretty key.

Um, they're not willing to, to pay for the future.

They think it would be really helpful, so they can keep running around having a good time.

But they're not willing to make that investment.

Um, so that would be good too.

Love that.

What else? Any others? At least for me, that's the the big one that.

I thought of.

Cool.

Earn.

Yeah, I was just kind of, I guess my mind just keeps tending towards more technical risks.

You know, or something just like they don't have the ability to carry the battery out there.

Yeah Yeah.

Um Ah, that's an interesting thing.

Uh, people can't bring an extra.

Battery with them.

Mhm.

Um, cause yeah, like, how are they getting there? Are they towing their bike and the battery? Uh, is it like loaded up in a truck, um, or do they riding to the trail with the battery on their On the bike somewhere like I guess like just, you know, chill on the back and If that's the case, how far can they go to get there? Before they almost immediately have to change it up.

And those are interesting questions, which, you know, you're, you're also highlighting another piece like Even, um, I would imagine even gas dirt bikes could have a similar problem.

I don't know what the range of a gas dirt bike is, uh, and how long people would ride it for, but you could exhaust all the gas in there and you've got to have a can to fill it up with, right, and do that on the go and bring like 2 to 5 gallons of gas with you.

Um, to write some more, um, but again, I'm not sure what kind of range they're looking at there and how many people do that, and, and, you know, like, are they gonna bring it anyway, right? Cause if they're gonna bring it anyway.

Yeah, maybe that's actually, who knows if that's really concerned.

Right, so like that's maybe if if I were to prioritize this one I'd say that's a little bit less than Then, you know, things like, like this one and this one, those two on the left feel like some of the Biggest impact things to me about, do we even do this thing, right, if they're never gonna buy it, if they're not gonna ride long enough as well to exhaust it.

Like we, we shouldn't, we should just either make a bigger battery right from the beginning, right, or we're already good to go, right? But, um, the cool thing is all these help us find out many like perhaps many different experiments we could be running, right? So this is good, uh.

I, I'd like to, I, I think it would be good to hone in on one of these two from just what I'm feeling here.

These feel like perhaps some of the, the riskiest things that we could, uh, you know, have to impact our future, um.

Which one do you want to work with? Which one do you think is is the right amount of risk and and the biggest amount of potential value in learning here for us? Yeah, I think in terms of the overall risk, I think determining if people want to buy a second battery is a lot lower risk than Figuring out how well.

I guess it depends on how you create the experiment.

I can see ways you can make these.

Make like writing testing how long people ride, you know, a higher risk than just asking them.

OK.

So like if they, if the battery is big enough that they wouldn't have to exhaust it, they wouldn't want to buy a second battery anyway, or if they're really, really short, they're gonna have to swap them.

How many batteries do they have to buy? Are they gonna need like 5 batteries for the day versus 2, right? Do they want to do that kind of thing? So it's like maybe this is the thing that leads to, will they even buy one, right? It's like, like do they Do they already do we need to do it at all, and then with that, right, would they buy it if we gave it to them? All right, so here's what I'm gonna do.

I'm gonna take this guy.

We expand upon it.

What do we need? To learn here.

Right, so Yeah, go ahead.

Well, I guess just kind of just what we've already kind of said to a degree, well, how long are they actually riding it for? Like what is their actual range requirements, so to say, for a day of dirt bike riding.

Yeah.

Uh, How far do they ride down the shows? How long do people ride? Um, good.

What else? And if we wanted to also think about the other risk about costs, you know, just taking that data, understanding, OK, like for the typical rider from this experiment, how many batteries would they need to purchase, so I guess.

Uh, that's again, that's going off of the other wrist that we're not necessarily zooming in on.

But um let's see.

What if, um, you know, in the case of what you said, right, and if they ride to the trail versus they trailer the bike in, you know, what if they already have You know, some alternative that Doesn't require them or.

How do I phrase that.

Is it kind of like, how did they get to the trail? Yeah, essentially.

Um, that's kind of there's something more there.

I feel like there's something else that you wanted to, to bring up, but I'm not sure what it is.

It it kind of ties into the You know, how If they're writing versus they're trailering it in, right, then the the the amount of range they need to get, you know, the distance to get to the trail, the equipment they bring, all of that.

Um, so that's why it's kind of like, yeah, how do they get there, I, I guess, you know, what do they? What do they normally bring if they're, you know, an existing IC writer, for example, what do they already bring? Mm, would be a good question.

In case like you said, OK, do they already bring cans of gas? If so, how much for the equivalent range? Yeah, yeah, like cause they're already.

They, they already, which is also kind of interesting, right, cause what are they already being with them helps me understand a little bit more about if they want to buy a second battery.

Not not fully, but it gets close like they already expect that they're going to be out long enough to refresh this thing.

Now, is the second battery the right cost for them versus filling a gas can? Well we know that they know the battery is gonna cost a lot more than filling a gas can, but what does it cost to refill that battery? How long, how many times do they do that? And like, you know, those that that's the balance that you are all playing with all the time, right, of cost more here, save more there, and then now also it's a bit about the experience.

So if they're already doing that, then that might actually You know, help with this risk about how it might fail, right? So that's a great thing to learn, uh, what do they already bring with them, um.

You You also you highlight something like how do they get to the trail is I'm gonna just I'm gonna tap it off of this one, How do they get to the child is, is kind of also like, um, how much range do They have when they get to the show.

Mhm.

Right, um, It's not quite right.

I know it's not right, it's like capacity or whatever, but like if they, if they're riding their bike there versus if they're towing it there.

That that matters, right? Um.

OK, we got a lot of good, what do we need to learn, OK? Um, we're gonna pick one of these and we're gonna start figuring out how could we learn that, right? And, and so we're gonna go through this thing of like if we had a day, if we had a week, if we had a month, what are the different things that we could go do to learn, right, about any of these items here that help us figure out how it might fail.

Um, so let's pick one of those to go a little bit further on.

Yeah, then we can, so we could follow the how far or what did they bring with them? OK, Matthew, I think I like how far a little bit more.

OK, so we expand upon that one.

Also, before we go any further, um, Erin, I also, I like this one.

I'm glad that wasn't the one you chose, uh, but I think it's also neat about the learning on this one.

if you were to take this experiment a little further, what they are you bring with them to the trail.

We talked about um not having a gearbox, right, and not having to like change things out, being able to configure the ride mode, if that was the experiment that you that you chose, by being able to configure ride mode, this matters maybe a little bit more, right? Like, what is all the tooling that they're bringing? What do they need to, how, how do they, what do they change on their bike when they get to the trail to create the right ride experience and all that kind of stuff, right? Cause that, that certainly plays in there.

So it it plays a bit into.

How long do they go for to exhaust the battery, um, but it's, it's more tangential versus like how do they change the right experience, right, that this one relates a lot more directly to that experiment or to that hypothesis.

OK, so what are the things we could do to learn? How far are people ride in a day on the chest.

I guess we could just like literally set up, get a list of riders, meet up a trail and just give them a bunch of batteries and see how far they go until they're done.

That's the simplest thing I could off top of my head.

OK, so, uh, track, uh, so I'm gonna see if I, if I, yeah, I'm hearing this one right, uh, so yeah, track, um, uh, rides and used capacity.

Mhm.

OK, good.

What else can you do? Yeah, I'd I'd just show up to a trail and just, you know, when I see riders come back, you know, I just walk up and ask them, hey, you know, how long, how far did you ride, and where do you ride along the trail? How far did you get out? It's kind of more of a simple data collection kind of idea.

OK, I just, I'm just gonna go interview people.

I'm just gonna go ask them, you know, talk to them, OK? Good.

What other kinds of experiments could you run? Let me just get one more before I I go any further here.

Mhm I want you to learn how far people ride.

We learn.

Mentally I'm thinking like, well, we already kind of already said we talked to people.

Um, Hm.

Mm.

Not quite sure.

Not as, not as creative on this.

Matthew got another one? Trying to think on like a one day time scale.

And this one might be better suited for like a one week thing, but if we had like, you know, the, the manpower to do it in one day, um, maybe run a couple tests up side by side at different sites.

Mm.

With different trail conditions.

What kind of tests? Well, I guess similar, just, you know, see how long it can take, how much capacity gets used or.

How long they uh like the rider like spends at a given trail, but the the differences between the two locations is like maybe one trails like.

A little bit longer in length, but you know, there's not as much elevation change, it's not like the, the black diamond trail, it's a more of a bunny slope, I guess, um, and then versus like, yeah, like one that's shorter, but maybe steeper elevation changes, which does take up more capacity in a battery.

Um, so I guess that would be it like across different trails, which might be better suited for a one week trial, but.

No, that that's really good.

You, you actually, you, you also honed in on something that I was seeing.

You can just watch.

Uh, people complete stress.

Right, like I see a guy go out.

I know that the trailer that he sent out on or this, you know, uh, this loop here is um uh 2.

6 miles.

How many times do they, how many, how many laps does he make? Before he comes in, right? Oh, she's out there, you know, for, for a half hour she made, you know, like 6 laps, and she paused and then she went back out for a little bit longer, right? 789, OK, came back.

Decide to go and have like hang out at the campfire instead and have some beers, right? Like done riding for the day.

Cool.

9 laps times 2.

6 miles, you know, there's how far they go.

So like I, I can ask, but I honestly just watch it happen.

Like that's another cool thing too.

Just like tracking capacity on your bikes is another thing you can, you can go observe directly, which is pretty cool.

So that's, that's another neat piece of, um, all these things are fairly measurable towards like real results.

Like you can ask somebody how far did you go out for the day.

They can look down at the odometer and tell you, or they can just get like, I don't know, I kind of maybe about 30 miles, right, and there's a difference between their perceived measures versus like actual things like real data you can get into, um, and we want to know a little bit of that too, like the quantifiable and the quantitative and qualitative metrics.

So there might even be a thing of like, did you.

We might even ask writers at some point, like, did you feel that you were going to run out of fuel or capacity for the day? Right, they had a whole other piece, right, um, that's more about, um, do they care about range, right, um, versus isn't long enough, right? Um, so here's a question, uh, track rides across different trails.

Can you do that in a day? Like if you want, I mean, it's, it's what, 1:45 to 2 now your time, like if, if you, if Shayas or She's boss were to go and say like, hey, all right, so, um, tomorrow at 120, I want the results from your experiment.

Do you think you could actually go and run that experiment? By tomorrow.

Yeah, that's, that was the tricky thing is it was, uh, I was saying I guess you could get it done in a day if you had I guess two separate, like, let's say, you know, Aaron and I each one of us went to a, a long flat trail and Aaron goes to the shorter hilly one, and we kind of run the experiment side by side in parallel.

Then we could do it a day, but I guess it's if we're looking at just like one team, one set of people, it would be more of like a multi-day thing probably.

OK, good.

So like it's maybe uh if you had a week, yes, you could do that, right? Like, like you said, the running the experiment takes a couple hours to a day, but actually setting up the experiment, getting the picking the trails, getting the right people, actually being able to get to them, etc.

to design and execute on that, it might take you more than a week to run that experiment in its entirety.

Um, what are any of the other ones, do they, do they move over a little bit more that they, you couldn't go from, from now, like if we hung up now and you wanted to start executing on experiment now, you could have results on that by noon tomorrow.

Yeah, no, or I would bump the, um, you know, the actual writer interviews, probably bump that over to a week because it would be.

Probably track, you know, get more people on the weekend.

Ah, OK, so you'd want to wait on that just because of the time of day kind of thing.

That's interesting, um.

I feel like this one's, could you do this in a day? Track rides it needs capacity.

Yeah, I mean, I think that one's, cause that's more or less just grabbing information off of the bike, um, from a from a day's use kind of thing.

I mean that would probably just tie into at what point to have you enough data points that it's a usable value, right? Yeah.

Ah, yes, so what we might then that one could expand further.

Right, um, and we could find out a little more of like, are we seeing a theme here that's worth getting more data around.

Right Go sorry, that logic extends, I would say then to watch just the generals watch people complete the trails.

Both in terms of what trails they're completing and the number of people that we're observing.

OK, but you, you could just, if you decided, like, well, here's a question, how busy is your day tomorrow? Between today and tomorrow.

You have a you have a lot of things packed.

Would you have Would you have like an hour or two to just go out to any local trail and just watch people do runs? Hm.

Yes theoretically, I possibly could if I knew where there's some trails around me.

I don't think, I'm not sure how popular dirt, but, well, I don't know about dirt biking, but maybe mountain biking up where I am in Seattle.

I don't know about dirt biking though.

We'll see.

I'd have to do research on that first.

So this is an interesting thing.

I actually think like asking res on a trail and watching them complete it are both really quick.

Like you could just Go somewhere, like, I mean, um, and actually there's maybe some shorter ones or maybe they fall a little bit more, you know, you need, you need 2 days, right, something like that.

Uh, track rides and use capacity.

So Matthew, you said you don't even know where if where the local dirt bike trails are.

So, and you want to get your bikes to these trails that you don't know that exists and be able to measure their usage on people that do, do they have them or you just gonna say like, hey, I got a bike, you want to ride this for the day? Right, like, do you think you can do that in a day? In my, in my case, probably not.

This one also feels like it's maybe a week to a little bit longer, right? Cause it's again, it's, it's a, it's a bit of that, where are we going, what's happening? Like are are we watching, are we talking to existing customers that have their bikes and saying like, hey, when are you going out? Like I just want to go watch this stuff like, you know, tomorrow, like that.

That takes time.

Like, are they even gonna go out and ride? Or are you going to, you're gonna gather up 10 bikes, you're gonna load them on a trailer, you're gonna take them out, you're gonna run this thing like, do you, do you have that available to you right now? Could you go run it? If so, great, maybe you can go do that a day.

If not, it pushes out more towards that.

Maybe it takes a week just to pull all those resources together and run that experiment.

Right, but when you go to run it, the actual running of it takes a couple hours.

Um, but the, the, the setup for it might take a little bit longer.

OK.

Um, There are What I also love about everything you came up with.

It's all very direct measures.

You could do some other stuff too, right, um, if you had a day to find out like what, what if you just Searched like um like R slash dirt bikes.

I don't know, right? Go on Reddit and and ask people like, and you know, do a look in there like how are people talking about their rides? And how far they might ride.

Right, so you could do that in a day.

You can get some anecdotal evidence.

Again, is it the best data? No, but could you execute on it in a day? Probably.

Um, do you have anybody in your office? That does this kind of writing.

Anyway that you're in your company.

Yeah, like I was gonna say it's like, yeah, I could directly ask the guy, hey, you're taking the bike out for the weekend, right? Let me know.

Yeah.

Right, talk to coworkers to give your rides.

You got one of them, that's great.

You got 3 or 4 of them, that's better.

You got 10, cool.

Um, but that's like you have immediate access to these people.

Um, Right, or maybe related, um, talk to salespeople.

Who work with Carolines.

Maybe they get have some understanding of what they do.

I don't know if that's as good, but it's something, right? It's, it's much more indirect data, but at least you're gonna find out something more about like, hey, do you do your chair owns ever talk about like being out for hours? So with those kinds of thoughts, anything else you think you can do like really quick, really cheap.

I say yeah, real quick, real cheap to get a ride myself.

Yeah, I mean that that's That's kind of cool too, like, take a ride.

That, that's a The problem is, are you a customer that you're trying to sell for? No, right, that's, yeah.

Right, so this motorcycle rider? Absolutely not.

But you know, we've got some interesting categories of things here, right? Like, um, you've kind of got, this is like your desk research, right? Search charter bikes, that kind of thing, um, this is more like, you know, like I said, leverage, leverage your co-workers, figure out what's going on there, right, start to see, and then when you start to get to like things like this, you, you're getting in on like these are direct measures, right, that that really help us know what's going on.

So.

Sometimes you're gonna get access to the right people and and you can do that, and that's really cool.

Sometimes to do those kinds of things that takes takes more time, right? You're actually gonna need like a week to get out there and do something like that.

What if you had a month? What if you had a whole month to design and execute on an experiment to learn about how far people ride in a day on the trails to see if they actually ride long enough to exhaust the battery.

It's one thing that came to my mind kind of goes on hand with.

If it kind of dips into a bunch of things I would say, but.

Like in terms of, like, you know, understanding, you know, how much they actually use.

I guess it'd be nice to know if we could make some sort of, with like our range estimates on like our street legal bikes, we typically advertise like your typical like street ride range, like if you're just kind of riding around town, if you're on the highway, if you're riding at like 55 miles an hour versus 70 miles an hour, we have all those like those three different metrics for range.

For something like trail riding and dirt bike riding like maybe.

Having something like that falls in line with a matrix to define the expected range depending on the the trail type, like we said, flat, hilly, steep, having a a matrix that better outlines to, you know, perspective owners.

Um, what they can expect depending on their preferred like trail riding experiences.

OK, um, I listed that up there as like create a trail ride estimation tool, right, cause like, or you're, you're talking a little bit on the site of like.

How fast are you going and that that puts a bit on the range.

And so now you're talking about building a building a little experimenting tool.

And maybe even measuring how much do people say that they do certain kinds of rides to give them various kinds of estimates, right? Um, yeah, it was, uh, you know, actually tracking, do people change their, you know, trail selection based on the ride or based on the range they experience like, as they become more familiar with the product, do they change their habits? Uh, change their trail, uh, I'm sorry, say that one more time.

How, how often they change their what their, their.

So their configuration, um, changed their trail selection.

OK, channel selection.

Right, if they know that they only have 40 miles of range, do they now then intentionally choose shorter trails, or do they, you know, do they not care and just continue riding the way they do? Mm, OK, and there's, well, just get through it and, you know, I'm only out for an hour today versus 5, something like that.

Yeah.

OK, um, You have a changing trail section was um you're actually talking about the physical trail, right? Yes, OK, um, what about how often they change? Like commode.

Hm.

I would say Another thing is like more of a question that to be answered is.

You know, falling in line with does the current battery design work.

Like we talked about uh, previously, like, well, do we need to just redesign a new the battery to be bigger and have better range, so I guess maybe the way to frame it is.

Like for the average rider or trail rider, is one battery enough or is one battery swap like.

I guess understanding what is the ideal number of battery swaps that would be needed for a given for the average rider.

I'm not sure how to word that per se though, but I guess um.

Yeah, go ahead.

I guess ideally that number with the with our current design, we would ideally want that to be, yeah, one battery would be sufficient or maybe like one swap, but nothing more than that, um, would be like the ideal case, I would imagine, because then you, then you'd have to buy a bunch of packs, which kind of goes into the cost thing that we talked about.

We're trying to think like what's like the shortest way to say that, I guess, yeah, I guess like what is the, is one pack enough for the average rider? I I guess what we'd want to learn.

Is one bad enough.

And that's really like the, the question, people don't ride long enough to exhaust the battery.

Right, so whatever your current battery is, I know you, you probably have multiple battery configurations, or do you not? Is it just one battery? For this, this particular dirt bike is one battery, but we do like our other motorcycle models have like different size batteries as well.

OK, um, and, and so I think like that's a, it's an interesting question that comes up for a lot of people, right? Like, When when you have a gas engine vehicle, um, you have fixed capacity as well.

How many gallons are in, you know, of capacity as the tank? Similar thing, how big is the battery? Like you, you don't change that day to day when you're riding the, you know, um, uh, a gas engine vehicle.

You can change that day to day if you had a swappable battery, which is actually kind of intriguing, right? And um you can choose like to top up your, your, you know, the battery all the way.

You can also choose to like, I just want 50% charge and it's fine just like I'm just gonna fill the tank halfway, right? Um, but when you're talking about batteries, you're also talking about like the equivalent for us is gas cans, right? Like am I bringing along.

You know, a an extra 1 gallon of gas, 5 gallons of gas, 15 gallons of gas, like how much do I need to really do the kinds of things that we want to do for the day and like, am I fueling myself or am I fueling others and, you know, that that maybe plays in for some people, not everybody, right now you're not your typical writer, but you actually can like You know, another interesting pieces, do they need a different kind of battery for different kinds of things, because you put in a bigger battery, let's say, right, like that's, let's say that it's just actually they're riding close to the limit.

Maybe if we just built a slightly bigger battery, that would be enough.

Right? Well, what are the downsides of that? Oh wait, more weight, yeah, more weight, changes the ride experience, changes the handling, changes its torque, well, not really, but it changes, you know, like just essentially how you're gonna get that, that kind of changed the experience, yeah, changes the experience.

Um, batteries aren't, they don't hold the same level of capacity over time.

Right, so you can make it a little bit bigger now, and that gives that meets actually the majority of the need for what you're finding, but Suddenly now it doesn't, and now people can't get the right experience they want anymore, right? So maybe that's another good reason for just a swap.

Um, maybe because battery weight matters if they're actually riding shorter distances.

Um, we can choose to be able to create cheaper, smaller batteries that are lighter weight that also again affect the ride experience, but perhaps in a really good way for the rider, right? So that's an intriguing piece here too, that we actually sell it with a bigger battery, but we sell smaller packs for quick swaps and shorter rides.

If you're only gonna be out for like the hour or two.

It's kind of like we, we can give them the opportunity to configure it the way they want.

Yeah.

You can't do that on a gas engine.

the same way.

Right? They're gonna run out of fuel and you have to fill up versus like I have this tuned in for this experience.

You know, a little bit differently, right? I put in a bigger pack so I don't have to change out the battery for this ride cause I'm taking.

I'm on a 100 mile, you know, dirt road trek with a bunch of friends to get up into these, you know, this canyon or something like that.

Cool, right? That's a different experience, um, versus like I'm going out to a trail that's like 2 miles long and I'm just gonna run it a couple of times just for fun.

Right, so you can really, you can hone in not just on software, but even on physical product a little bit more for them.

It's like if the, if the thing doesn't go at least 100 miles, let's say for the bike, and it probably does.

They can't even do that canyon ride, right? They can always put in less gas for a shorter ride, but they can't even get there.

They have to bring along, you know, more fuel or something like that where you might have a battery that actually meets it that.

You know, I can give you that other option.

Um, Yeah, there's just some interesting things here.

Here's a, here's another one throw out there for if you had a month, um.

And we're just, we're hypothesizing now cause like again, we're getting to a month, that's a lot of time and investment, um, but you could uh create uh sales uh for spare batteries.

Right, uh, what's the website look like for that? How do people buy them? How do they integrate them, what software needs to change in order to, if at all, to connect up the battery, like resets and such on.

I did swap and the, the OS has to know that, right? um.

So I can sell it and I can even do that cheaper too, again, maybe not a day, but I can create a sales page.

That doesn't actually sell anyone batteries.

I see what the, you know, the click through rate and the interest, how many people are following it.

Yeah, like that.

So I'm gonna, I'm gonna put that under, under one week maybe here of um.

Uh, create a sign up page.

Or um extra batteries.

Right.

I can't remember I'm trying to remember what that prototype was called.

All out of my head right now.

And say kind of more strongly ties into the second point that we're, you know, the people don't want to buy a second batter idea.

Um.

Right, it's the, yeah, and the way they're they are related, right this.

Yeah, this one's more indirect, right? So I could, it's interesting.

I could find out, I could create a sign up page for extra batteries, and by monitoring the click through on that, like how many people try to buy an extra battery, even if we don't already sell them.

That's gonna give me a lot more indication about will they even buy a second battery, but it also gives me incidental information about, are they concerned in the first place.

Yeah, right? Like it's, it's a, it's that's also why I think it was more important that you went with the people don't ride long enough to exhaust the battery.

Like if they're never riding long enough for any of their adventures, and they can always come back and plug it in.

They're probably not even gonna care about buying a second battery, you know, they won't even click that button.

Um, so this one's much more impactful to the whole thing, whereas like, it's a, if, if we get people clicking through, like I said, we, if they're not willing to buy a second battery, we don't know yet if it's because it's cost, because they don't want to haul it around, um, or because they actually have enough range as it is, right? So this one is much more impactful for understanding.

Is there a point in making that sale in the first place.

But if they do buy it, we probably have the idea that they want to go farther, and they can go now, or they want to be able to do that quicker.

Right, they don't want to wait an hour or two for a charge, and especially if you're on the trail.

OK.

So, we're starting to see some interesting experiments, right? Some much more longer term things you can dive into to learn about this, um, heck, you could monitor.

The GPS locations of riders.

There is, you know, at.

If you had a month, you could probably do that, maybe even a week, I don't know, right? of you really want to get towards software solutions, how far do people rhyme? Well, they already got our app on, or they, I can give them an app uh that just tracks their ride lengths, or I can go maybe ask.

Actually, it's kind of interesting.

You might have this some of this data somewhere, right? Like if I think about, I don't know if you, you know, you do any of that kind of tracking right now, but if you had that kind of information, some of these things are are answerable enough of like somebody somewhere has actually already done this research, we can go talk to them.

And that really short circuits it and it helps us figure out, OK, cool.

Now I have actually some validated learning on this concept because somebody already did that.

Or somebody knows how to go do that really quickly.

Let me go talk and work with them, and then come back to what's another experiment that I can, that I need to go run.

Now that I have this data, what adds the story? What gives me more of that information, um, that helps me, you know, prove out if this thing's going to fail or not.

So I can decide to go run that experiment, create this feature, and How are you feeling there? Questions? Comments? So they just kind of different ideas it's like, you know, it's like now we can also touch on like what are their charging habits.

Yeah, as you said, right, you know, like with a gas can, you know, they choose how much they want to fill up with the charging, you know, it's kind of, yeah, they can choose how long, you know, what charge percentage they wanna hit or if they just decide, yeah, you know, 2 hours with the charging is enough, I'm gonna go back and use whatever I got in those 2 hours.

Yeah, like I, I don't have a lot of time to go out and ride, but this is just, this is good enough to get me out and have some fun for a while.

And then I come back and like grill for a party for the day or like who knows what it is, right? You, um, you get to find out some more of those things.

So actually like that's an even interesting thing is like how, I don't know if it's it's a day or a week, so I'm just gonna, I'm gonna put it somewhere in the middle.

Um, how many rides? All right.

Uh, to people.

Take before they charge.

Right, that's an interesting question.

Uh, to ask too if it's like 12 rides.

You probably got enough capacity.

Right.

OK.

So, um, I, just a quick question.

Do you know, is Shas coming back or we have we lost him for the rest of the day? Uh, he, well, actually, technically, Matthew and I also was supposed to be in that meeting, I think, but OK, I would be safe to assume we probably lost him for the rest.

OK, so we're wrapping up today and we're gonna start talking about the um the homework for the next week, all right? Um, and so I just wanted to know if, you know, if I was expecting to see him come along with this or, um, or, you know, or not.

But here's the catch up on the homework.

OK, so you came up with a bunch of different ideas, right? Some of these experiments and, and maybe there's another one that you can get into, but like, I said you've crafted some different things, especially the stuff for one day is really intriguing.

One week might be possible.

We're gonna meet up in a week, right? We're gonna have our next class again in a week.

And so, uh, your homework for the class.

Um, is, is really hone in on that experiment, right, and do the last little bit of planning for it.

We, we kind of already, I said, got some ideas, so maybe what's the least amount you can do to learn? What are you gonna find out in a day or two or so, or just an hour even.

I want you to actually go run an experiment.

Pick one of those experiments, go run it.

And then use that to figure out what would be your next steps towards your hypothesis, OK? Um, let me know as well what happened with that, right, as you're doing, um, your experiments.

So, um, I'm gonna also, I'm gonna give you a little tip about the the talking to people one before we go any further, but like this is, I, I actually want you to go and and ideally.

Talk to or observe the behaviors of real people.

Mhm.

OK.

So this is, this is, uh, I loved it when I, when I did this, um, but that, that's a key piece of homework here for it and you know, we have a few videos for the next week, so catch up on this, those last little bits.

Um, so if you, if you're like actually talking to people like co-workers who ride, um, people on how long they rode for that day, that kind of thing, um, we don't get into it too much here in this class, uh, but you, you know, it's good to consider as you're interviewing people.

Um, try to suppress your emotions.

Right, you feel like, oh, how far you ride for the day, you know, uh, 20 miles.

Oh, that's cool.

You rode 20 miles.

Is that typically how far you go? Right, so like, yeah, yeah, yeah, 20 miles, that seems like a good ride, right, cause they know they saw you get excited.

And I want to go a little bit further, right? It's, it's good and said to say, um, you know, like, you know, oh, how far did you run for the day? 20 miles.

I see you rode 20 miles.

Just put that see it back out.

Good.

Um, how, how often do you ride like that? Well, not that much.

OK, so today was a different amount of time.

Yeah, yeah, right, and the, the more you leave that open, the more they'll tell you.

Right as you're interviewing them, um, another great tip, um, I don't know if it applies to this, it's um.

You, you say, tell me about a time.

Right, like, tell me about a time when.

So like, for example, tell me about a time when you wanted to do a bunch of cho writing, and you ran out.

Of gas, or like, or you you you couldn't go for the whole day.

What happened? What did you do? What you, what you do next, right? What, how, how did that make you feel? Right, and just kind of drive into questions like that, um, what would, what would make that better, right? Um, and so you, as you're trying to find out this, this bit, you can ask people about sort of like, uh, you know, tell me about a time that you wrote less.

And you thought you would.

Right, and and even dive into that of how often does that happen? Um, and that will help give you some more information.

So if you're getting into interviews with people, try to keep them open ended so that they can fill it in, rather than like, do you ride more than 30 miles in a day? Yes, cool, great, I got my data, right, like, well how often and whatever else, like we're not surveying them there so much as we are trying to learn about their experiences, their themes, and what really matters.

Uh, so that we can figure out how that helps answer our question, right, of course, how long did you ride today is like that has a definitive answer, um, but then we can maybe expand upon that, but try to stay neutral as you're talking to people, and that will get them talking a little bit more.

Yeah, like in the case of that question right now, and then trying to tease out whether or not that's, they hit their target or not if they're satisfied with how far they went or something like that.

Yeah, exactly.

Was that your tar, you know, what was your target for the day? Like how did that make you feel? and stuff like that, really good, really good points.

OK, so, so that's gonna be your homework here is go run your experiment, go talk to real people.

Who could help you learn.

About your risk, right? And uh run your experiment, figure out what's, what can you do to learn that, and how does, what does that tell you about your risk and your hypothesis, OK? Um, try to give me record like a short video, write up a, you know, a little statement about what you discovered.

And How it felt for you.

And as you're going through, that's what I would love to to learn um in in clarity flow, and then we'll discuss the results of your experiment further uh in next week's class.

OK.

Right, sounds good.

Yeah, anything else that you are curious about for me before we wrap up for the day? No, I think.

OK.

Nothing for me? All right, cool.

Well then, let me pause the recording or stop this here

Hey guys, um, I just wanted to give you a little note here about recording, I'm sorry, downloading recordings within clarity flow.

Um, so Erin, you, you asked a bit about this at the end of class.

Um, so this is gonna look a little meta because I've got a personal account and a business account that are both going on.

Um, what you see here is my personal account where I have like different Settings, you know, what I can do here.

There's very little things that I can do on my personal account, right? I message plate, I can get like notifications, stuff like that, right? Um, and Uh, and so that's, you know, my, my Gmail right there versus here's the window for the recording that I'm currently doing in this message.

So you can see it's a, it's a different account.

Um, but also it's not gonna show it to me here.

There's a setting on, um, within clarity flow for downloading messages.

It's currently enabled and it's been enabled.

Um, so the way that in which we can download a video, if you wanted to download this video of like, let's say just the initial feedback here.

Uh, the way to do that is, um, on this screen, there's a little ellipsis option here to pull up the menu.

It should just be download MP4.

Um, so hopefully, that option is there for you, um, and you should be able to go and download the videos from there.

Uh, likewise, if I go to the recording for Zoom here, um, here's the day to recording of everything that we did.

I can go and download that as well.

Um, if that option doesn't show up, let me know.

I'm going to continue to look in Zoom to see if there's any other way to get that going right now.

I'm letting it be though because it is currently uploading or processing the video from today's session.

So once that finishes, I'll dive more into Zoom to see how that goes and what I can do to download there.

I will also upload that video here to the thread.

So I hope that works.

Um, like I said, you got any questions, let me know

Matthew, Aaron, Tris, it was great going and reconnecting with you both, uh, I'm sorry, all three of you, uh, this week, um, for our session.

Uh, thanks for all the questions, the conversation, for getting through, um, some, some like some challenging activities.

And speaking of those activities, I wanted to give you a little feedback on just some extra things that I saw, um, on your board and uh for your business problem statements.

Um, personas and um your OKRs, uh, to feed into the additional homework and stuff that you get into over the course of the next week.

Um, so I'm gonna start with the, uh, business problem statement, and I think what you were discovering in the ORs, uh, around what you can actually Um, manage, right, from an objective standpoint, uh, what you can really control, and that's your sphere of influence maybe stems from the business problem statement a little bit more, right, because it is more of a, a marketing sales focused business problem statement.

Um, and I, you said you were comparing it a little bit or basing it off of what was in the class, and there was an interesting thing that I, it got me to look back at that business problem statement from the class.

Uh, and try to see maybe where we could still, uh, where we're missing and maybe how we connect more with the problems that you see your customers have, and the things that you can do in product experience, uh, to connect with that.

Uh, so the example from the class, uh, I'm just going to read it again, um, and, and then highlight some key pieces.

It says our online mortgage application tool was originally designed to make it dead simple for repeat home buyers to apply and get approved for a mortgage.

We have observed through application abandonment rates and qualitative customer feedback that the product isn't meeting our expectations, which is causing a 45% increase in support calls and a 14% decline in successful application submissions.

How might we provide a simpler application process so that our customers are more successful as determined by a 20% increase in customer in submissions and a 50% home buyer retention rate.

So Key aspects, I think, to note in the business problem statement is that it is a problem for the business and it is a problem for the customer, right, that these two come together, that when our customer has a problem, that, um, and especially if it's a problem that relates to our business, then we have a problem as well.

Um, and so the, the core piece looking at is the abandoned application rates, the quality of customer feedback is tying right back to customer pain points and problems, right? These are issues that they're going through, um, which is causing a drop off in applications for us, which means less revenue and time, right? So we can see that connection between the user behavior and our actual business.

Problem.

Um, and so what we want to see, uh, and, and as well, like they, they said, they're doing more support calls because they don't know what to do.

They're not getting successful applications through, which again means, um, more time investment for us, for each customer because of those support calls.

I think you're mentioning a little bit about that on day one too for your bikes and, and trying to go and do diagnostics.

Um, and then the other part then is like, you know, less applications, less revenue for us, but also then a customer that's not getting what they want, they're not able to go and get approved for a mortgage, so they can't buy a property.

Now, um, and if I look at your business problem statement, right, it says that our online pre-order system was designed to help potential buyers express interest in their newest model release or the newest model release.

We have observed that the conversion of pre-order interest.

To purchases is 60% lower than the original plan on record.

How can we remove barriers to purchase from the urban commuter so that we can increase our conversion rate by 35%? What I'm, I'm again, I'm noticing more of like there's a business reason to do this here and, and there's a, what's the problem that the customers have? What's the problem that the urban commuter has that we're not solving well? Do they, um, so, Uh, to perhaps help you hone this one in, I just spent some time on the Xero Motorcycles website, and when I was looking through, you list all your models and it gives some really interesting pieces here actually.

Let me just pull it back up.

Uh, some interesting bits about, uh, all the different models, the SRRS, the SRF, the S, the, you know, etc.

etc.

right? And it shows me the range, the charge time, and the speed that they can go.

Neat.

Well, beyond looking at the model, I, you know, the body of the bike, which, again, I don't ride motorcycle, so I don't, I don't know what what I would want here, what's good for an urban commuter? Right? So maybe we're also seeing a job because they don't know what they want, um, or why does the urban commuter want the newest model.

Um, so try to think more about that urban commuter and what's the problem that they have that's creating a problem for our business, right? So perhaps in tuning the business problem statement a little bit more in that way to, to really identify the problem that the customer has that is creating a problem for us.

You're gonna get more towards working that out and, and finding that customer connection which feeds the other elements further, OK? Um, and by the way, if I take a look at it, my guess is, uh, the, the SR class, the S and, uh, and maybe the, the FXE FX looked like perhaps some uh commuter bikes, so I don't know.

Tell me if I'm right later.

OK.

Uh, I'm gonna go up to your proto personas.

And, um, I gave you some feedback on this before.

You already expanded on it a little bit.

I'm just noting that this is your commuter.

Um, so it, it is not your, um, your dirt track racer.

It is not your adventure, uh, biker.

It's not your long range, um, motorcyclist, right, or anything like that.

It's the person who's gotta get to work every day, um, and he's trying to get to and from different places.

Uh, so, You could perhaps flush out some of your demographic, uh, things a little bit more, um, psychographic qualities that would impact the way this person uses your product or services, interesting, right? The differences that make a difference.

Why do they care? What's, what's special about a commuter that wants to commute on a motorcycle, um, and what behaviors and other demographic pieces tie into that? OK, so flush that out maybe a little bit more.

Needs and obstacles, I think you were doing pretty well before and you're, you're calling out both of them.

So that's pretty cool, right? Me charging at work, I thought was really good.

And I think when I looked at the, the FXE bike, it was just what, 110 volt outlet, which makes it really easy to charge at work.

Um, how do you highlight that more? How do you make that part of your experience? How do you connect that in the app and the service to go and say like, hey, you can do this anywhere, right, rather than trying to find The infrastructure that supports charging, um, it's just a plug somewhere.

Do they have to carry a cable with them? Are they finding it? You know, there's, there's lots of things that go along with that too, right? So while, while a physical cable is not a software product, if you're on the software side of Teams.

Maybe there's a bit about how that connects and how you inform people in the future as we start looking at actual solutions, but there's still a problem if they got a charge, and how do we make that better for them, right? And how do we make it so that they can, if they want to commute, and they just need to go, get up on the bike and go.

OK.

Um, so down to to to your objectives and key results, um, some things to highlight here.

I know you were struggling a bit with the objective, you just, you kind of stopped partway through our online pre-order system, right? So, going back to that problem, and if you start thinking of the customer's problem that's going in here, right, uh, like what is, what is the issue that they have that's creating a problem for us.

Then when you start thinking about the objective, you're gonna start thinking about objectives that create a great experience for the customer, that make it um qualitative in nature, that make it um maybe ambitious as well, and that really make it kind of action oriented that a customer wants to go and join us.

Um, I attended a conference years ago where we, we talked about finding your tribe, and, um, and I think as well Simon Sinek talks about it a lot in, in start with why that if people connect with your why, then they're going to get it more connected with your products and services.

Um, they need to feel like part of your tribe, and the objective can help you get there, right? If we're creating the right objective.

Um, that maybe speaks a little bit more towards why would a customer care, right? And, and, and, and what are we going to do for them and when might we go and try to see that done by, um, you're gonna create that, that bigger impact.

You're also gonna create just a better story, a better narrative from the business problem statement.

To the ORs, um, for the executives in, in your, uh, in your business or in your company, um, and, and being able to fund and explore different initiatives to go and create that customer impact, um, and to create and make um solutions that work towards those objectives.

OK.

Uh, now, on to key results, uh, I think some key things to just go and highlight here.

Uh, mostly, you were getting there towards the end, right? I, I started to call that out good to look at the percentages and the ratios, um, because they're, they're pretty key, um, at some point here.

Um, so we want to make sure that we've got those framed out right.

Um.

The other aspects for your key result that you want to start diving into uh said is make sure that they are, um, that they're specific, right? We have that they're time-bound, they're aggressive in nature, they're measurable, they're verifiable, that they give us that that aspect to really understand what's going on here, right? That, um, that we focus on the outcomes of what people are doing and by how much, right? Who does what by how much.

Um, so when I start looking at At some of the key results, especially connected with the objectives here, um, tie back to your core user and persona, right? So who, who does what urban commuters perhaps is some something that needs to start seeing coming in here and you're you're looking at that at that top one.

We're trying to change the behavior of a prospective urban commuter, right? So urban commuters buy our products, urban commuters test them out, right? Um, urban like urban commuters download the app with a demo mode.

Uh, to see what it would be like in their daily life, right? Um, you know, we have, and then likewise, uh, the number of commuters that take test rides or that register for test rides goes up by 50% through the app.

Um, so you're starting to, I think your key results are getting more on point there is if we think about who's doing what, right, and the what of they, they're taking test drives, they're clicking on these things, they're, we can measure this behavior and what that ratio is, you're still in a good spot.

Um, so keep focusing a little bit more then on that customer problem.

That's going to help you with your business problem statement.

It's gonna help you a bit more with your objective, and the key results are all going to start connecting in a little bit smoother.

OK.

Uh, so that's what I got for you for now.

Take a look at this.

I look forward to seeing what additional personas, uh, you identify around maybe that key one there, um, what other objectives, uh, maybe come up that you can, you can write, I wouldn't write too many, maybe, maybe one or two, right, but especially as your business problem changes, if it changes.

Uh, you, you're going to tune that in a little bit more than what you've got here and then, uh, finding those key results that show those measures towards.

Did we actually did we achieve that objective or not? OK, give them a test with your, uh, with, uh, with your co-workers, see what they think about it, see, um, look for their reactions and, and what else you might discover.

Uh, I'm really excited to find, uh, hear what you find.

All right, take care.

always reach out here if you have any questions, I'll talk to you soon