In this episode, the speakers delve into the complexities of implementing the Scaled Agile Framework (SAFe) in large-scale projects, particularly in the context of a large organization like Walmart. They discuss the inefficiencies and drawbacks of SAFe, highlighting how it often leads to increased bureaucracy and fails to address core engineering and delivery problems. The conversation shifts to emphasize the importance of continuous delivery, effective team structures, and engineering excellence. Real-world examples from Walmart’s transition away from SAFe to a more effective agile model are used to illustrate these points.
Inhalt
- Introduction to SAFe and its implementation in large organizations
- Critique of SAFe: Increased bureaucracy and inefficiency
- Importance of continuous delivery in agile transformation
- Shifting focus from process to outcome in software development
- Walmart’s experience moving away from SAFe
- Emphasis on team structure and engineering practices
- Introduction to Enterprise Dojos for effective learning
- Role of behavior-driven development in clarifying requirements
- Discussion on continuous improvement and agile mindset
- Conclusion and teaser for the next episode
Shownotes
Information for decision-makers considering the SAFe framework
Transkript (automatisiert erstellt, wer Fehler findet darf sie behalten)
DevOps auf die Ohren und ins Hirn.
Ein Podcast rund um DevOps
Von Dierk Söllner, Falko Werner und Luca Ingianni.
Okay, and why is that necessary?
Like, why do we
need to scale? What is scaling in the
first place? Well, I don’t think
honestly that we do need to scale, but
scaling would be, you know, we’re all
instead of having a
team trying to deliver
something, you’ve got
a much larger group trying to
deliver things and
scaling the
agile process up to
a level above those teams.
So you wind up with, instead
of a team scrum, you have a scrum of scrums
where you have, you know,
the teams have their stand-up
and then you have a stand-up of stand-ups.
That sort of thing.
You have things like, oh, we’re all going to have
standardized story points so that
the project manager knows how big
a story point is from team to team to team.
You know, we’re going to
do these things
at a scaled-up level.
Okay.
Which sounds like it’s actually
quite necessary. Like, I can’t imagine
delivering a
product that’s this, you know,
a car, let’s say. You know, something as
complex and big as a car
without having to scale up and
having to coordinate between potentially
hundreds of thousands of engineers.
But, you know, all those parts that were built for the car
were all built independently on their own
schedules and then
assembled.
You know, the first
well, and let’s just back up, you know, the
first experience I had with SAFE as I was working,
on one of these giant things.
We’re talking about, you know,
a warehouse management system that’s 25
million lines of code that’s built by
about 400 developers scattered across
multiple teams.
There was no boundaries
between teams at the
code level. We were basically
feature factories. Here’s a feature for this
team, feature for this team. We’re all touching all the
same code. You’ve got
branches that last for like a quarter
that get integrated once a quarter.
And so what obviously
what we need is SAFE to
coordinate all this effort. Strangely,
no, that didn’t help.
But we did have more meetings
and
more roles for more middle management.
So we acquired
that. But,
you know, there’s deeper levels
of problems there that you can’t
fix with process. You can’t just scale
agile
and add this process on there
to overcome the
fundamental problems of the architecture,
the team structure was terrible.
You know, we didn’t have
the right organization
actually deliver in a truly agile
manner. It was basically
waterfall with agile bolted onto it.
Okay, so
that was a fairly big word,
truly agile. What
is true agility?
True agility is the
ability to continuously deliver value to
your end user. I honestly don’t think
there’s lots of people who claim they’re agile,
My firm belief is if you can’t deliver on demand today, which you made changes to today, then you’re not Agile.
I mean, the fact of the matter is, is that the purpose of the whole Agile thing isn’t stand-ups, isn’t two-week sprints.
The purpose is to get feedback on what we’re delivering to find out if it’s the right thing to deliver.
I’ve got to, I try to pare things down to make them easy enough for even me to understand.
And one of those is Agile mindset.
And you hear about people talking about Agile mindset all the time.
It’s like this, it’s like this vague culturally thing, but it’s not vague.
Here’s the Agile mindset, really, to me, all right?
The requirements are wrong.
If they’re not wrong, we’re going to misunderstand them.
And if we don’t misunderstand them, they’ll change before we can deliver them.
And so we are trying to deliver things in such a way that we’re minimally wrong.
We have a value hypothesis that we’re trying to implement.
Okay, cool.
We think if we deliver this, we’re going to get this value.
Now, what’s the core of that?
What can we deliver, the smallest thing possible, to invalidate that hypothesis?
Let’s not build the whole thing and cross our fingers.
Let’s build small pieces, ship them frequently, get feedback, and then adjust the implementation as we go.
And if I’m doing quarterly PI planning for all of the things we’re going to deliver,
that means that I’m going to get indications of my value hypothesis.
At the end of the quarter, or even at the end of the sprint, terrifies me.
That’s much too much change.
I want to know tomorrow about what I built today.
What do you need to get this working if it’s not a framework, descriptive thing, bringing roles and processes and so on,
and prolonging the time for feedback in such a setup that you explained or described earlier?
We have 400 or several hundred people working together on a big chunk of code.
So, we went through the effort of saying, okay, we’re not meeting our goals.
We weren’t meeting our goals before we tried SAFe, and we met them even worse when we implemented SAFe.
And so we said, okay, what we need to do to meet our goals is we need to figure out how we can,
instead of delivering every quarter with, you know, giant war rooms at the end of every delivery,
because we’re going to break things, we want to deliver every two weeks.
That was the ask, was every two weeks.
And so our SVP,
told us, you know, the group of senior engineers in the area, how do we do this?
And so instead of bringing in, you know, an agile coach and saying, what we need is better agile process,
he just left it to the engineers, how do we get this done?
And so we got copies of continuous delivery, poured through them.
We had done some looking into domain-driven design, and we had these goals, you know, the team we need to be able to deliver continuously.
Every team needs to be able to deliver independently of every other team in any sequence.
Because at 3 o’clock in the morning, you can’t couple two teams together.
And let’s say with SAFe, you have release trains to couple teams together to manage that all of those dependencies as one thing.
Well, if they’re one thing, they should be in one team.
So what we do is we re-architect the application so that they’re not one team.
So we started carving out business domains, assigning those business capabilities from the warehouse management system to teams, created real product teams,
who owned that product.
eine bestimmte Fähigkeit, die Kontakte zwischen ihnen erstattet haben, Domain-Diagramme
innerhalb des Teams und über alle Teams, so dass wir wussten, wo all diese Interfacen
waren, so dass wir sehen konnten, was die Architektur sein wird, um sicherzustellen,
dass wir keine seltsamen Abhängigkeiten zwischen diesen Teams hatten, und dann
Dinge apartieren begonnen haben.
Und so konnten diese Teams sich unabhängig von einander erledigen, so dass wir de-skalierten.
Und das Team, das ich an diesem Versuch leitete, das eine Entwicklungs-Team, das wir
an einem Zeitpunkt 5-10 Mal pro Tag erledigt hatten, musste keine Verabschiedung von
einem anderen Team bekommen, musste nicht zu einem Release-Train-Ingenieur gehen.
Wir bauten Jenkins Pipelines, wir hatten ein sehr kleines Plattform-Team von ca.
vier Leuten, das Jenkins Pipelines templatiert hat, damit es so war, dass wir
wiederholbare Dinge hatten, die es schwer gemacht haben, Fehler zu machen.
Und dann haben wir einfach geschickt.
Es war liberierend.
Okay.
Ist all das, was du gemacht hast, verboten von SAFe?
Oh nein.
Nein.
Also, du kannst all das mit SAFe machen, es ist nur so, dass du SAFe tun musst.
Aber das Problem, das ich mit SAFe sehe, ist, dass es so eine schwere Liste ist, um alle
auf dem Rahmen zu trainieren.
Also was ich von SAFe-Mitgliedern immer höre, ist, dass, naja, das Ding, wo ihr
angefangen habt, das ist einfach, das ist Shoo, das ist für Shoo-Hah-Ri, das ist
deine Apprentiss-Level-Dinge.
Okay, aber du trainierst.
Ja.
du weißt, tausende Entwickler auf deinem Apprentice-Level-Ding,
der Preis für Training auf diesem Niveau bedeutet, dass nichts verbessert wird,
weil es in der Form ist, und du hast Jobs geschaffen,
du hast einen Release-Train-Ingenieur geschaffen, der nicht ein Job sein sollte.
Ich habe Pipelines dafür, aber du hast einen Job für einen Release-Train-Ingenieur geschaffen,
was bedeutet, dass es sehr schwer ist, davon wegzukommen, weil du jemanden’s Paycheck nimmst.
Ich sehe das, aber ich sehe auch die aktuelle Transformation,
wo ich von Waterfall, von Agile Teams, von vielen anderen Start-Ups sehe und versuche sie zu verstehen,
was andere Leute machen, organisieren, kommunizieren, und von diesem Punkt aus
versuchen wir es immer noch zu machen.
Und um Agile Mindset in Ordnung zu bekommen, müssen wir immer noch die Rollen abschneiden,
um zu dem zu kommen, was SAFe vorgibt.
Wir geben also keine neuen Rollen hinzu, wir müssen wirklich darüber nachdenken,
nicht nur die Rollen abschneiden, sondern auch die Leute abschneiden,
die im aktuellen Waterfall-Line-Management und Flow-Organisation-Management
um zu dem zu kommen, was SAFe vorgibt.
Ist es nicht vielleicht eine gute Sache, viel Überprüfung zu reduzieren, um zu diesem Eingang zu kommen?
Und vielleicht einen nächsten Schritt zu gehen, was du auch vom SAFe aus sagen würdest,
zum Intermediaten?
Aber anstelle von Leuten zu sagen, was der nächste Schritt ist, weil, ja, das wäre eine
Verbesserung.
Aber anstelle von Leuten zu sagen, was der nächste Schritt ist, warum sollten wir nicht
einfach nur unseren Zielern sagen und dann den Weg mapieren?
to get there.
I’ve frequently had this conversation
with people. It’s like, I don’t know, we need to meet them
where they are. Of course we need to meet them where they are.
But meeting them where they are,
the argument I get is
from people is that we just need to tell them
what SHU is.
Tell them where the next step is.
No. What if we just told them
I can deliver daily
in any architecture,
on any platform,
if I have everything in place to get that done.
And that includes the right
tooling, the right processes
and the right people’s mindsets
and the right architecture of the organization.
I can do that.
This is where we are today.
Now what’s our roadmap to get to that?
Because, you know, honestly
I’ve heard people tell me they don’t need
to deliver continuously because their
customers don’t want features that frequently.
They definitely want fixes that frequently.
If something’s broken, they want it fixed
now, not on your next monthly
release. Yeah, but they don’t also
care if that’s tested, but
and working the same process,
the same pipeline
as a release, they just
want it fixed. So hot fixes
with little or to
no testing would
be sufficient. Yeah, I mean, if it’s
a dumpster fire, the thing we want to do
is throw a bottle onto
it and hope it’s not gasoline and
hope it’s water, right?
And we’re just praying, but that’s
just no. I mean, we want
to be able to quickly
fix vulnerabilities,
you know, we’ve discovered a vulnerability
that’s leaving us at
high risk for a data breach.
You know, do we want
to hot fix that in there? Do we
want to have a safe process that we
understand works every single time to get
that done? Now,
going back to your question, though, does
the safe, you know, the scale agile framework,
does it prevent these things? No. You know
why it doesn’t? Because they
cargo culted all of the ideas from
everybody. If you go and look at
all of the information that’s
in there, that’s good, all the good
information, it’s all from
other sources
on how to do these things.
Yeah, it’s from DevOps,
even
technology,
value stream
management.
Design thinking,
yeah, that’s what I usually
call safe as a vacuum cleaner.
But the problem that I
have with it is there’s lots of really
bad ideas.
If you don’t have the experience
to know how to tell
the difference between the good ideas and the bad ideas in the
framework, you’ll just get hurt.
It’s like the
analogy I gave one time was you have a
recipe book, right?
And some
of the recipes include cyanide
in them, but you don’t know cyanide’s
bad. And you’re just like picking
recipes and then hoping you don’t
get one with cyanide in it. A feature team
is a really good example of a recipe with
cyanide in it, right? It’s just a team that
runs around delivering features for other teams.
That’s not a good idea. It’s terrible for
the architecture. If you
do know what the bad ideas are,
you don’t need the framework as a
clearinghouse of, you know, as a smorgasbord
buffet of good ideas
because you already know what the good ideas are,
right? I don’t need certification
in it. I just
need to understand what the good ideas are.
All right. So what are the
good ideas of safe? So the good ideas
in safe, right?
Improving value streams,
value stream mapping is an important
thing. You can read them all about it in
continuous delivery and a whole bunch of
other previous literature.
The ability to deliver
continuously.
Safe has included that.
Organizing teams, but
although if you ask the authors of team
topologies, they’ll tell you that they have
modified their
ideas on team topologies
to make it more in line with what safe
wants to do instead of more in line with
effective delivery.
Right. And so sometimes they’ll just bring
things in verbatim and sometimes they’ll twist
them to make them more appealing to people that
are in a waterfall mindset,
which doesn’t get us to the next step.
It gets people to fake agile.
So yeah, there are good ideas in there and
they, they, they, I know where they took them
from the original sources are.
Yeah.
Yeah. Original sources is always a good thing
to know them, to get in touch with them.
Um,
we also had one of those authors in our podcast,
didn’t we?
Which one Manuel or
no, we, we try to, uh, in the end somehow the
communication fizzled out.
I, I really need to reach out to him again.
Uh, but yeah, well, I, I would love to talk to
that, to one or both of them.
Yeah.
It was ironic.
There was a medium post that, uh, uh, safe
filed at a, uh, a DMCA.
Uh, because they used, uh, and they gave credit.
They had a picture of the, of the diagram for safe.
And it’s, it’s just ironic that they’d file a DMCA
takedown on somebody referencing their stuff.
Yeah.
Yeah.
So you talk about the good stuff in safe, which, and I
don’t think they’re hiding the fact that they, you know,
they are taking good ideas from elsewhere.
You know,
sure.
I mean, we all do.
Oh,
of course it would be, it would be terrible if we
didn’t didn’t right.
But, uh, so what’s the cyanide you already told us you
don’t like feature teams.
Um, but what, like what’s, what’s the worst idea in, in
safe that you’ve, uh,
Oh, well, I, I think, you know, worst ideas is difficult.
Uh, I think it depends on what you’re trying to do right
that moment that’s impeding me from delivering value.
Um, but I, I think feature teams is.
A big one, the having somebody responsible for release instead of
having pipelines for release is a bad idea.
Uh, uh, the, um, release trains is a bad idea.
We should architect and re-engineer to not couple teams.
Right.
And we should go down that journey and get it done instead of codifying a
release train engineer to manage release trains.
That’s that’s just, you, you don’t wanna get locked into that.
It’s ties you to the slowest process on the, and makes delivery.
It makes delivery incredibly expensive and risky at the same time.
Those, those are bad ideas for sure.
Um, I didn’t prep with a list of other bad ideas off the top of my head.
I’ve, uh, I have a list of somewhere of things I really, really hate.
Well, if you find that list, um, like if it’s in an article or something,
we can link in the show notes.
Yeah.
And I do have a, a, a blog post called safe isn’t, uh, and I’ll send that to
you.
Oh, wonderful.
That sounds great.
So.
But yeah, so it, it’s, it’s interesting.
Maybe, maybe instead of being very general about it, maybe you can tell us
the story of what happened, uh, at Walmart, where you introduce safe and
then moved away from safe again.
Um, a couple of times, but maybe, you know, give us, give us the entire
trajectory.
Yeah.
And you have to understand when you’re talking about an organization like
Walmart, there’s not a monolithic, this is what happened because, uh, you know,
when you have.
Like 18,000 plus developers scattered across multiple VP areas, and you’re in an
organization with a culture of a lot of autonomy at the, at the director level, not even at
the VP level, uh, directors have a lot of power to decide what happens in their areas.
Um, and so there was, you know, people say, well, you just had a bad implementation of
safe.
Well, there wasn’t a implementation of safe.
There was multiple areas with different implementations.
And so I, there’s stuff that can be covered in the development of safe that I could go and witness.
So the, you know, the, the thing that, that I’ve was under personally was they, they brought
this in and they sent us all through training to learn how to do, you know, all
the themes, all the ceremonies for safe.
And we had a lot of ceremonies and we had a lot of, of, uh, of not breaking things down
small and delivering things fast.
It was just, okay, we’re gonna make things bigger so we can.
and have a bunch of meetings to make sure we have all of the dependencies mapped together
and we know that everything’s, you know, this feature has to release with this feature
and coordinate all of that and just do massive coordination.
You had, you know, retrospectives at the organization level once a quarter,
which didn’t make anything better.
You had a lot of very confused and demotivated engineers, you know,
software engineers trying to do work, but, you know, we’re focusing on all these meetings
and how to do SAFe correctly and trying to get better at that
because we’re told if we get better at SAFe, we’re just going to deliver better.
And, you know, we didn’t know any better because we had just come from a process
where they sent us all through PMI training to become project managers.
All the senior engineers, they gave us project management training so we could manage projects better.
And then they gave us SAFe, right?
And it’s still just PMI.
And it wasn’t until we actually started focusing on the CD that you could actually see,
oh my God, there’s like a totally better, different way to get this done.
We didn’t expect that it was going to be that much better.
We just thought it was going to be, we were able to deliver more quickly.
We were just trying to meet a business need.
But the fact that we were able to not only meet the business need,
but do it with higher quality, with better team morale and get faster feedback on things
and make it less expensive to try new ideas.
I mean, what?
You can get a Slack message from a distribution center to try something
and then spin around in a couple of days from their idea and to deliver back a result and say,
hey, what about this?
Do you like this?
Not even bug defects.
I mean, bug fixes, but it’s actually just ideas, right?
When you see that dramatic change from focusing on the engineering and focusing on the outcomes
instead of focusing on the process, and you’ve been living under that process for so long,
you kind of developed hate.
I don’t think hate is too strong a word.
Later on, when we started up the dojo and we were starting to work with the teams
that helped them with continuous delivery, and we’d walk into some areas,
weren’t using SAFe, they weren’t using anything.
They had teams just doing Scrum, and we’d help fix that so that they could actually deliver.
But then there’s some areas that are using different implementations of SAFe still at the time.
And you’d see things like quarterly planning, where people were listing out all of the stories
that were going to be delivered.
And which?
Which month of that quarter.
I mean, sorry, which week of which month of that quarter.
And it’s just, it was straight up waterfall without the discipline of a Gantt chart.
You know, it was just bad waterfall.
You had release train engineers who were blocking things because, you know, you’re stepping
on my toes.
Well, you shouldn’t have toes to step on because we don’t need release train engineers.
We need pipelines.
We need teams who understand how to deliver quality.
And so, but I’m still told that those.
Those are all just bad implementations of SAFe.
And I’m like, well, it’s incredibly easy to make a bad implementation of SAFe based off
of anecdotal evidence that I hear about and also direct evidence that I have.
I’ve never seen a good one.
And every time someone’s claimed to see a good one or implemented a good one, it’s like,
well, I mean, we just use the stuff that worked.
Cool.
I mean, I do that too, but we don’t call it SAFe and we don’t have PI planning.
Is there another thing that you…
That you would suggest to look at if it’s not safe and pick out the good things out
of SAFe or reading, you mentioned continuous delivery from Bay Farley or other sources
that you would prefer to start off with?
You start off with the problem.
I mean, SAFe is the solution to something.
Okay.
But what’s the problem we’re trying to solve?
And let’s go back to the definition.
So DevOps, the union of people, process, and product to enable the continuous delivery
of value to the end user.
All right.
Well, number one, we need to focus on why can’t we continuously deliver value?
Okay.
We also need to focus on the people, right?
How do we make it better for the people?
What in the system is hurting the people so that they can have better lives and focus
on continuously delivering value to the end user?
Do we have the right products?
You know, do the people have the tools they need?
And how do we…
How do we lean out the process?
You know, how do we look at every single process that we’re doing and say, is this
process actually adding value to the end user?
And, you know, can we deliver sustainably?
You know, the thing that I found most effective is just why can’t we deliver continuously?
Why can’t I deliver today?
And starting with why can I integrate working code to the trunk today?
When you start peeling apart those problems and trying to solve them, you’re not going
to solve those problems.
Then you find the, you know, the tooling and process solutions that you need to solve
those problems.
You start looking at your team structure and how the people are organized and say, okay,
well, this structure is poor for what we’re trying to do.
We have too much coupling here or too many handoffs.
You know, we’ve got a testing team.
Well, that doesn’t work at three o’clock in the morning.
So we don’t want a testing team.
We want the tests to be happening concurrent with the code to make it pipelines effective
at preventing…
We need to get better at doing that.
You know, when you start finding these problems, well, why can’t we do these things?
You focus always on the outcome you want, not focus on, hey, we need this heavy process
and we might get this outcome.
No, why can’t we get the outcome?
Fix those.
It’s just like any other engineering problem.
And so when you do that and you start peeling these things apart, you find the problems
in your context that you need to fix.
We actually, last year, DevOps Enterprise Summit created minimumcd.org.
Several of us were talking about people doing CD in ways that were causing them pain.
It was dangerous.
CD was getting banned places because they just said, hey, we’ve got a Jenkins pipeline.
We’re doing CD now.
And so we put together this list of problems to solve that either we can do this or we
can’t do this.
It’s not subjective at all.
That’s your minimum to safely do continuous delivery.
But the point isn’t CD.
The point is using CD to improve your organization.
And there you see architecture.
There you see then things that come out the other end of Conway’s law and whatever.
Then you look into all the different ideas.
You might get inspired by frameworks, but you don’t say, for example, less is a better
way to start or EAD or whatever.
Yeah, I would just look at what problem do we need to solve and then go and hunt down
some solutions to that problem.
When we started with why can’t we integrate code at least daily?
And the problem we had to solve is the requirements were unclear.
And so because the requirements are unclear, the tests are difficult to come up with, which
means we have to do a lot of exploratory coding to find out if it’s the right thing
and then do a test after instead of test during.
And so we had to go find a solution.
So we went, okay, what’s the solution to that?
And then we ran across behavior-driven development as a way to clarify things and make that a
disciplined approach to getting testable acceptance criteria out of a story instead of just it
should do some things and then go figure it out.
You guys can figure it out during coding.
I hated that.
You ask a product owner, oh, you’ll figure it out during development.
No, that’s not the way quality happens.
But things like that.
You know, it’s like, okay, we have this problem.
We need to come up with a solution for it.
And the solution isn’t more process.
The solution is better engineering.
But CD is an excellent way of shining a spotlight of what those problems are.
You quickly uncover things like the team structure is wrong.
Let me ask you how that feels.
If I’m trying to implement CD, how can I see that it’s shining a light on my team structure
as opposed to, I don’t know, my code perhaps or my CD pipeline?
Maybe my CD pipeline is just terrible.
I don’t know.
Well, I mean, your CD pipeline could be terrible.
Yeah, that’s a safe bet.
Hopefully, you’ll be in a better shape than we were when we started.
We had to just go and read books and figure it out.
And if you don’t have anybody who has any experience that you can call on, which gets
easier now, but back then it wasn’t.
If you don’t have that, you read books and figure it out.
So if, you know, you can tell your team structure is bad.
If you have things like there’s, you know, Bob owns that service and Bob’s not here,
so nobody can make changes to that service, right?
Or no, we can’t actually deliver to production because it has to go to the testing team
for them to validate it and then tell us whether or not it’s good or not.
So now you have an external handoff for delivery instead of having a deterministic pipeline.
The pipeline’s not deterministic.
You have tests that are flaky and you don’t trust your tests, but you ship anyway.
Okay, well, that’s terrifying.
You have to fix that because that means you’re going to be having to do a manual step.
Any manual step between commit and production is something to fix.
That’s a bad pipeline problem, right?
And so, or I have to wait two days to get information from a product owner because the
product owner is not part of the team.
They’re separate in the PMO.
So we have a poor communication structure in the organization.
I can’t deliver that way.
You just value stream map.
I mean, that’s the core tool right there.
If you map every single step from idea to delivery in that value stream, and I would
start at the team level and work up.
Other people prefer to go top down, but I found that if you start at the team level
and you start finding the impediments that a team has, then you can start building and
linking up to bigger value stream maps to find out.
It’s more of a graph, really.
To find out where the value stream is.
To find out where those problems are.
And then you can start looking at that and visualize, hey, this is a terrible structure.
How do we improve this to flow things better?
But don’t you have to do that with every team then?
And doesn’t it take too much time?
And maybe do it repeatedly with every team in a similar way and learn the same things?
No, every team turns out to be pretty different in the experience.
In the dojo, we would value stream map teams.
Either as part of an engagement or just to, hey, would you like a value stream map done?
Because it might help you.
Every team learns something.
There’s some similarities.
Everybody does requirements, coding, building, delivery.
But the details of that fluctuate wildly.
I had several value stream maps that would show me exactly why it is that you don’t want
an external testing team.
I actually have a blog post out there somewhere where I talk about the fact that, let’s go
back to Agile.
The Agile mindset, the requirements are wrong or they’re misunderstood.
Okay, well, so if the requirements are wrong, then you have a testing team and a development
team, each building their own code, because the testing team is either writing a manual
script or testing automation.
That’s code either way, whether or not a human is running it or not.
And the development team is coding, but they’re both coding based off of poor requirements.
So there’s not a lot of value there.
But worse is the requirements are fine.
But each team has their own code.
And the development team misunderstood them differently.
And what you see at the end is that the two come together and then there’s a triage process.
I saw this repeatedly on multiple teams.
There’s a triage process.
Is the test wrong or is the code wrong?
Now, for some reason, people have this bias to thinking the testing team is correct.
And then, you know, in that case, the development team has to prove that they’re not incorrect
because the testing team are testers.
So the test is correct.
But the test is based off of the same poor misunderstanding as the development team.
A different misunderstanding.
And then you go through the triage process.
You kind of get back to the product owner.
You cycle back.
You try to figure out which is right.
Then you deliver it and find out that everything is wrong.
And the end user needed something else entirely.
And people wonder why delivery is so painful and slow.
You know, it’s like reporting.
If you have two reports, one of them is wrong.
Always.
So let’s just bake it in so we have one report and get faster feedback on how wrong we are and fix it.
You’ve been mentioning this dojo.
What?
What is that dojo you were talking about?
Oh, yeah.
Thanks for asking.
So the enterprise dojos are something that Target started up back in 2014, where what they would do is they would bring a team into the dojo and help the team learn how to improve how they deliver in the team’s context with the team’s work.
So it wasn’t like some contrived classroom thing.
It’s like, okay, what we’re going to do is we’re going to bring you in.
We’re going to use immersive learning, where we’re going to use your work and we’re going to, instead of having a two-week sprint, you’re going to have a two-and-a-half-day sprint.
We’re going to deliver twice a week so that we can get rapid feedback over the next six weeks of what needs to be improved so that we can just keep snowballing that improvement.
In the context of Walmart’s dojo, we were focused on continuous delivery.
That was an enterprise goal we had to use CD to improve engineering excellence across the organization, improve delivery.
And so we were focused on CD, so it was basically, you know, why can’t we CD, let’s value stream map you, and then let’s work with you for the next six weeks with your code, in your context, in your area.
We, like, embed in that team for six weeks.
It wasn’t drive by coaching.
It’s like, oh, your tests are terrible.
Hey, let’s get in with you and let’s see how we can help improve those tests, teach you how to test better.
You know, you’ve got pipeline configuration issues here.
Let’s help with that.
Use Git flow.
That’s terrible.
Let’s help you learn how to do training.
We’ve got things like trunk-based development, right?
And hands on.
So it wasn’t like the hand wavy agile coachy thing that a lot of people suffer with.
There’s something else that I’m really curious to hear about.
And I wonder whether you, you know, you were part of those conversations.
Which is, how did that actually, how did that conversation actually go at the higher levels of management at Walmart?
At first, you know, introducing Safe.
But then later on saying, you know, this was a terrible idea and we need to roll it all back.
So we were really fortunate in the logistics area because our SVP was a really forward thinking guy.
He had been a software engineer for a number of years.
And I think leaving Walmart and coming back helped give him some perspective as well.
He’s the one who gave us that mission of how do we do this every two weeks.
And he basically gave us a blank check to do what we needed to do.
And so when we came to him with, okay, this is how we need to organize the teams.
And here’s the first teams that we should organize that way.
We need to make sure that they’re not going through a release train because the release train breaks the model we’re after where they can deploy independently of each other.
And he was like, okay, let’s do it because we’re not meeting our business.
We’re not meeting our business goals the way we were doing it before.
So you don’t just do it harder.
You need to change something.
And he supported us.
And not only was it just like, yeah, you should do that.
But, you know, he’d have biweekly meetings talking about things that people were improving in the area.
He was, I don’t know if you guys have seen the Smarter Every Day video with the reverse bicycle.
He had one built.
We had a reverse bicycle built to talk to people about mindset.
Change, right?
What’s the reverse bicycle?
So on Smarter Every Day on YouTube, they have a thing where the guy built a bicycle where if you turn the handlebars left, it turns the wheel right.
So you put a gear train on the on the on the handlebars and then demonstrated that you could learn how to ride that.
And in fact, it would adjust your brain so much after you got good at riding it, that you’d have a hard time riding a normal bicycle again until you readjusted your brain again.
It’s similar to I’ve seen experiments.
It’s where people put prisms on on glasses where you see upside down and eventually your brain adjusts.
And so he was using that to demonstrate, you know, the level of mindset change required.
That’s actually what you do is you see upside down and the brain adjusts so that it relates to everything is the other way around.
So it’s quite nice to see that the brain is that flexible.
But it really takes a lot of time.
Yeah.
Yeah.
Yeah.
It really takes a lot of time to unlearn if you already know how to ride a bicycle, a normal bicycle, to have this capability to ride the reverse bicycle.
And that really took took a lot of time.
And the same thing was what your ways of working that you have in an organization, the culture, the structures.
Those are the things if you want to get improvements.
Yeah.
Improvements.
Then you have to do change and you always have a lot of energy to put into to change something.
And the problem we have, though, and it’s coming back to safe is, hey, we’re going to we’re going to now we’re going to use safe.
Okay.
Why?
What problem are we trying to solve?
The issue is that you can’t change.
Culture is an outcome of change.
It’s not something you change.
And you don’t want to just change process because how do you know you’re doing the process?
Better or worse?
What we want to do is we want to focus on those outcomes.
What is the outcome we want to achieve?
Are we achieving it?
Okay, now let’s keep changing and improving things.
And then the process we use is an outcome of us improving our outcomes.
We don’t just dictate process.
We we focus on outcome and it involves, you know, on the the first team I was on, you know, I made a rule on the team is like we have all this all this scrum process we’re supposed to do.
I said, Well, no, we don’t have to do that anymore.
What do you mean?
No, no, no.
We’re we’re focusing on.
Continue.
So we’ve got this goal of delivering at least daily from this team.
So we’re going to do is we’re look at every process we do and make it prove that needs to exist.
We don’t do it, you know, stand ups taking too much time while we we trim them down.
Stand ups not delivering value.
Okay, we just won’t do that, right?
If we’re always if we’re sitting in the same area talking all the time, you know, do we need a stand up in half?
We’re visualizing if we’ve got a Kanban board up in the area, do we need to stand up?
You know, if we’re all focusing on that teamwork thing, then we’re a team.
What’s the point?
What’s the point of doing the stand up?
You know, we said we’re going to have retrospectives.
We’re going to do it much more frequently.
Because why can’t we every single day is like, why can’t we integrate code today?
What’s stopping us from doing continuous integration?
And then what’s stopping us from delivering every single day, multiple times a day?
And yeah, just focus on that.
Just make things better.
Hello, dear listeners.
It’s me again.
So this is the end of the first part of our interview.
And please look forward to the second half of it.
In roughly two weeks time.
See you then.
Goodbye.