The Bionic Coach

Introduction

Being extraordinary, it’s a good feeling.

My first electronic enhancement was a calculator. Forget long arithmetic, this was awesome.

My second was my first computer, a Tandy Colour Computer in 1981. I still own it, and it’s the heart of my retro-computing collection. I remember my father labouring over a calculator working out the core and winding wire gauges for the Bandura, the Ukrainian national musical instrument, which he’d begun building. I wrote a program that took hours of that calculator work and did it in seconds.

By then I was hooked. Using technology to do things better. To do the math. To know things.

The advent over the past few years of large language models (LLMs), that are trained on a wide array of publicly available content and knowledge, has spurred a revolution in how we think about knowledge.

All those times I wasn’t allowed to bring a calculator to a math exam? Well, today it’s “don’t use AI to do your homework.”

In an exam in school, we don’t allow collaboration, copying the answer from someone else, using a calculator or computer or your phone or the Internet.

But this is business.

Copy each others’ answers. Work together. Bring that calculator, that computer, use the Internet. Do what we need to do to keep the advantage in our court.

And now, use that AI. Carefully, of course, because we don’t know how to trust them yet, but let’s get that advantage in play.

We can all be a generalist

All we have to do is describe our context, and in seconds an LLM can not only recall the general knowledge we could apply, but also communicate a simple application right away within that context.

Can’t remember how to apply Little’s Law in a situation? Ask the LLM.

Need fast talking points on why placing someone on multiple teams is a bad idea? Ask the LLM.

Here are some examples from my chat history:

• Give me some fast talking points about the changes that need to happen to move from project to product thinking.

• If best-practice is a tool used by bureaucratic leadership, what is the corresponding tool used by servant leadership?

• What would a leader who uses bureaucratic leadership tools think of a leader who uses servant leadership tools, and vice versa?

• Design me a quick workshop agenda on the topic of servant leadership.

These are all fast questions you can use running between meetings to help centre you, and avoid that dreaded “why didn’t I bring up XYZ, it’s far more important than what I talked about back there.”

Manage your energy and stress

You’re still responsible for the critical thinking, for deciding whether the LLM actually gave you something useful or not.

But in today’s fast-paced, always-on, need-answers-now kinds of environments, the fast transactional responses from an LLM can buy you time.

With the extra slack, you have a better chance of intentionally choosing where to spend your precious thinking moments throughout your day.

Learning to wield the tool

Zero-Shot

This is the simplest way to use it. Ask it a question, get an answer. Go and choose how to apply it.

Chain of Thoughts

Traditionally the realm of automated “agents” that can apply a pattern of thought automatically. For example, ask a question, ask it to critique its response, ask it to produce a better response given the critique it just gave.

Now that a chat conversation is the popular method of interaction, you can engage in this kind of conversation on your own. Guide it how you want.

Ask it to provide step by step instructions as to how it thinks it should solve the problem you give it. Ask it to critique the instructions. Ask it to follow the first instruction. Ask it to critique and refine its response. Ask it to take the second step. Ask it to reconsider the first step based on what it learned in the second step.

You get the idea. You can be as methodical as you want. Within limitations.

Although the limitations are regularly being expanded. ChatGPT’s latest 128K context window provides seemingly unlimited (300 pages or so) conversation length.

Tree of Thoughts

You’re probably best to leave this to the “agents” or bots for now, but if you’re methodical you can do this by hand. Basically you could explore multiple answers at the same time in different chats, cross-pollinating the best ideas as you advance towards the most reasoned answer.

I raise this idea now, because it’s likely only a matter of months, maybe a year, before we evolve from leveraging a single chat window, to leveraging agents on our computers. This is the thought progression pattern for which they’ll be tuned.

This is not the end of expertise

LLMs are generalists. They lack real understanding of the topics at hand. As soon as things start to get specific, or complex, they will not offer much in the way of useful solutions.

This is part of why I’m concerned about its application in complex endeavours like software development, or law, where creativity and deeply intertwined contexts require specialist understanding to get past superficial responses.

This tacit knowledge we have goes beyond words, which are the sole domain of the LLMs. It lets us hold unique viewpoints without washing them over with the popular viewpoints. Because that’s what’s special about the work that experts do, we hold and prove out those viewpoints to give them a chance to one day become popular.

Rather, imagine you had an assistant. Someone that could ease the burden of the banal work that we do every day, to give us the space to do the more important work, the real thinking.

Ethics

At what point does it become “not me” doing my job?

Are we just still “standing on the shoulders of giants” as we advance or work, or is it something else?

Can we trust ourselves to always recognize when we are being gas-lit, when the LLMs hallucinations, stated with such authoritative language, might slip by us in a moment of inattentiveness?

Is it a different kind of learning when we make our own mistakes, versus when we fall prey to the mistakes of the LLM?

In Lawrence Lessig’s notion of Remix Culture, he asks what-if we have already explored all of the ideas in art and music. What about when all that’s left is the remix. The re-combining of existing ideas into a new form. Like how hip-hop was created. Can you really say it’s not its own legitimate art form?

Lawrence Lessig, Cory Doctorow, and Canada’s own Michael Geist, these are the pioneers that for decades now have explored the legal ramifications of all this. Explore their work if you want some insights into the legal future of generative AI - it’s not cut and dry. It’s not exactly “stealing other people’s work.”

But truly, all an LLM can do is remix. It’s a statistical algorithm. Sophisticated, yes, but still just math.

Did the developer actually write that code? Who’s liable for it? These questions sound nebulous, but decades of “sampling” in music, digital reproduction in other realms, have provided a great deal of foundation. Companies like OpenAI are even confident to provide legal protections for their users in the field of copyright violation.

The impact on coaching

Meet people where they are

Managing the level of change and disruption starts with meeting folx where they are, understanding their context, and working with them to apply the principles of agility in ways that help them reach their goals.

Modern product and engineering techniques can be so foreign to folx, that we can’t just expect them to take some generic training and begin to apply it.

So what coaching has become is taking these techniques and contextualizing them - customizing them for where folx are.

Drop that story mapping powerpoint, or even that 3-day workshop, and turn it into an experience guided by the coach that you weave into your coaching work. Help people see what they’re doing through the lens of the modern practice, and paint a path towards its adoption that they want to take.

These customized micro-learnings, mini-workshops, still take a lot of work to create and manage. However, they’re basically applications of general industry knowledge.

You can use the LLM to help you retain coherence of context across the teams you support. It has more working memory than any human, and it knows he surface level generalities.

Prompting

Imagine a context by context prompt that frames the situation a team is in. Using your own senses and sensibilities to understand, craft a “system prompt” that details their condition and constraints. Supporting 4 teams? Create 4 baseline prompts, one per team.

Feed your coaching notes into the context. Just as your interactions with the team are personal, your notes are personal, keep your interaction with the LLM personal.

The LLM has far more working memory to recall aspects of their context than you do, and will give you remixed ideas as frequently and as quickly as you can try things

Consider common prompt patterns:

• Give the LLM an identity
  • eg “You are an assistant to a busy agile coach. Answer the coach’s questions in a way that keeps them centred on agile values. Explain your reasons, and take a step by step approach.”
• Give the LLM context
  • Name the people in the context, tell it their job titles and roles, let it connect the dots between general functional patterns and the individuals
  • Describe the structure of the context, functional dependencies outside the team, technical aspects of the work they do
• Be sparing, but use modelling constructs
  • Describe simple models to the LLM that help you be concise in your contextual descriptions. eg “My interaction with this team is primarily transactional, and I am looking for more ways to build trust to deepen that relationship.”

eg - and this is intentionally vague for the purpose of this post

You are an assistant to a busy agile coach. Answer the coach's questions in a way that keeps them centred on agile values. Explain your reasons, and take a step by step approach.

We are working with the ABC team. Jean is the scrum master, and Nora is the product owner. The team generally follows scrum, but consistently has too much work in progress and doesn't tend to complete the work in their sprints. They tend to have open conversations with each other, but I sense some tension between the developers and the testers.

Give me some ideas to safely explore the tension between the developers and testers during our next retrospective.

Don’t be surprised if what you get out of that prompt is a whole lot of generic advice. Think about it this way, sometimes that’s how coaches can sound to our coachees too!

Apply Chain of Thoughts techniques to the chat. When you get that wall of generic advice back, push the LLM to critique what it just said.

This is very generic advice. I'm specifically looking for something practical I can do at the next retrospective to explore the tension between the developers and testers.

Look at each point you just made for me and assess if it is just generic advice. Assume I know the generic advice. Give me specific activities or tools we can use.

Maybe something specific juts out of the wall of generalities from the LLM you can explore, like:

I like the idea of Appreciation Cards. I don't want the developers and testers to feel like I'm targeting them. I have a hard time sometimes getting them to go deeper than platitudes (Joe is great! Thanks for the help!) into specifics (Joe pointed out that it would be useful to know why, and when we did we learned...)

Give me some examples of Appreciation Cards that I can use to prompt the team during the retrospective.

Now the LLM will apply itself to some specifics you can cherry pick and bring to the retro as you frame the conversation to anchor your expectations.

Important - Start fresh each session

Start a new chat each time to reset the context with just your prompt. If you need to carry change from chat to chat, incorporate what you want to preserve into your prompt.

LLMs couldn’t care less about how little hard drive space you have!

Don’t bury yourself in the generated responses of the LLM! They will fill up your computer with mountains of pointless drivel! Instead, extract what you found useful and incorporate it into more focused notes, or into your prompt templates.

Confidentiality

Just like your notes are confidential, and you’ll be protective of them because of the damage they can do when interpreted by others, your whole conversation with the LLM is confidential.

Be wary about using any of its raw output, because it may regurgitate your every private thought in a hundred unexpected ways. Sometimes, you will read what it says and go back and change your prompt, because it has misinterpreted the nuance of what you were trying to describe.

Cleaning up our thinking

Use the agile prime directive, codify it into your thinking, and codify it into your context prompts.

“Regardless of what we discover, we understand and truly believe that everyone did the best job they could, given what they knew at the time, their skills and abilities, the resources available, and the situation at hand.” Norm Kerth

The more work you’ve done in the field, the more you realize its truth. It’s not just a saying. People are trying to do their best with what they have. People are limited by the system around them, and our ability to raise awareness of the system around them has always been our superpower as coaches.

Take the opportunity provided in your new private conversation with your LLM assistant, to expose your thinking more broadly. To be critical of it. To continue to expand your self-awareness about the work you do.

Start the community conversation

If you’re not already considering using these tools, consider where you want to be on the forefront. We have a lot to learn about using this new technology, and we’re only going to learn it by starting the conversation and sharing our experiences.

This post only scratched the surface of how we can use LLMs in the coaching profession. Leverage your community to continue the exploration!

Use the tools, assess the outcomes, and find better ways to use them.

Connect with your professional community, and explore.

Connect with leaders in other communities that have already been exploring.

Because the cat’s out of the bag. The technologies are here. And our competitors are already moving.