One of the comments I get most frequently when I talk about new AI technologies and tools is “That sounds awesome! How are people actually using that?”
I have my own examples (most of which I share right here on this site), but I wanted some 3rd-party examples as well. That’s when I found Episode #61 of the “Smart Friends” podcast.
There’s a lot of talk about AI features, but there should be at least as much talk about AI helping make non-AI features.
Sean Devine on the Smart Friends podcast
That quote from the episode caught my attention because it perfectly aligns with the philosophy behind this very site.
Titled “How CEOs Build with AI”, it’s a fantastic discussion about how Sean Devine is actually using ChatGPT to make himself more efficient, empower his employees and delight his customers. He mentions that using AI tools like this has increased his personal productivity at least 50%, which is wild but not out of line with some of the usage I’ve seen.
The entire episode is just under 2 hours and I recommend listening to the entire thing. I ended up taking about a page of notes from beginning to end. However, if you want some of the highlights before jumping all the way in, these are some of my favorite bits.
Writing a newsletter with ChatGPT as a collaborator (@ ~21:00)
Early on, Sean goes through the process of how he took an idea for a newsletter, had ChatGPT flesh it out, cite research to support his point and get a fully-formed newsletter. One of the most interesting points here that he talks about is the importance of asking ChatGPT for a lot of ideas.
What you want to do is [say] “generate as much as you can”. And then, to your point, I’m reading it basically as fast as it’s writing it, I find the two things I like, I say “chuck the rest, let’s go with this”.
ChatGPT as a collaborative programmer (@ ~35:00)
After discussing his newsletter workflow for a bit, Sean dives into how he had a relatively complex routing and optimization problem that he was trying to solve. After blocking out a day dedicated to going back and forth with ChatGPT in an attempt to write a solver, he had something reasonably workable.
This is crazy to me, because this is a very real-world, immediately valuable use of ChatGPT to create something that would previously have taken a team, or at the very least more than one day.
So I started at 6 in the morning…And, it took me until 11 or so, but in one day…I built a library that could solve, not a toy problem, the exact problem that we’re solving with sample data generated from real scenarios.
He goes on to talk a bit more about his process, and how he actually went about solving all the bugs and edge cases. But one of the pieces I found interesting was that he mentioned asking ChatGPT a ton of questions (180+ over the course of this exercise). Down to figuring out how to host and deploy this tool that he built, ChatGPT was able to help him complete the whole thing.
The biggest thing that stood out to me, especially as someone who’s gotten stuck on programming problems before, was this:
There wasn’t a single moment the whole time where I had any fear of not knowing, because it doesn’t matter if I don’t know…the expertise was completely irrelevant.
This is taken a bit out of context, given that he mentions his prior programming experience and I think that experience helped him ask better questions and give better feedback on the solutions ChatGPT suggested. Still, having an endlessly-patient programming assistant that can take feedback and provide solutions is going to change how work gets done.
Some ChatGPT best practices (@ ~40:00
A key theme here is to not get self-conscious with how many questions you’re asking. Interfacing with something like ChatGPT is different than interfacing with a human, in that the social norms around being “annoying” with questions don’t really apply.
In addition to being comfortable with the back and forth, Sean gave a few more “best practices” that he’s found for using ChatGPT in his work.
There’s a lot of leverage in the questions themselves. I’ve learned that…you need to explain your goal and everything you know to it.
If ChatGPT can work with the same knowledge that you have, and as much context as possible, it will be able to give you better answers and be a more effective assistant.
With the context window constantly expanding and ChatGPT being able to handle having access to more information, there’s no reason not to give it as much as possible. However, even with access to this additional information, having it walk through problems step by step and helping it break down a larger problem into smaller problems is still key.
Don’t say “hey, write me a solver that does the following”…first it’s “let’s think through the objective function and constraints”…now once we’ve got those right, I’m going to scrap that conversation and say “here’s our objective and constraints, let’s formulate that into a linear program”…
He goes through how he creates a ChatGPT conversation with each of these individual steps and how to prompt ChatGPT to work through a project plan, whether that’s one you provided or one that ChatGPT developed itself.
The “bug fix” story (@ ~51:00)
He then goes on to tell a story about how the team had a bug where many of the buttons in their app were non-functional, the support team was in a time of transition and it needed to get fixed. By looking at the recent code that was merged and narrowing down which pull request was most likely to be the culprit and then bringing in ChatGPT, he was able to find a solution.
By taking the diff (the changed code) and the user reports of errors that were in Slack and passing that entire context to ChatGPT, it was able to suggest a list of 5 potential issues, two of which (combined) were the underlying issue that was able to be fixed.
This sort of automated code audit or assistance as a code investigator is another use case where I can see ChatGPT already being helpful.
How to motivate everyone to fully embrace the tools (@ ~54:00)
For me…what, six months ago was good, now looks kind of so-so to me…what, six months ago, was fast, now feels very slow to me. And…speed and quality are…all relative to what’s possible. And so, when you’ve seen that what’s possible has fundamentally changed…I can’t unsee it.
From minute 54 until about minute 59 is what I think is the most important segment of this podcast. Sean talks about how his interest in AI isn’t just fan-boying, but being compelled to notice this fundamental shift that’s happening in how work gets done. He mentions how you’d definitely notice if your business productivity dropped by 50% overnight, so not taking advantage of a similar potential productivity increase is just as bad.
That’s the same mindset that I have when it comes to this AI tooling. It’s a lot of fun to experiment with and it’s definitely a hobby. But the impacts on work and the potential for it to impact how work actually gets done in the future is impossible to unsee.
The future of work (@ ~1:12:00)
Towards the end of the podcast, Sean starts to touch on what the future of work might look like and how he and his team are hiring and making future-looking decisions with these tools in mind.
The value of expertise has cratered…I think it’s critical to go through the organization…and ask the question for everyone, ‘To what degree is that person’s role about what they know?’
He points out, correctly, that most of these new AI innovations are impacting people with specific knowledge, because this sort of knowledge is becoming commoditized. When the answers are always available, asking the correct questions becomes even more important. He talks further about recruiting for people who are good problem solvers and people who are good at asking questions.
We are focused primarily right now when we recruit in two areas: one is problem solving, by itself. We’ve reduced the amount we care about their technical expertise…how they approach problems and break them down into their components…that skill is the one that’s valuable.
This is why I think we’re really coming into the “age of the generalist” where having knowledge from multiple areas and being able to combine them or to find out the best way to apply all that knowledge is what’s going to be most valuable in the coming years.
AI as a global enabler
He talks, as they wrap up, about using AI not only in features in a product, but also as AI helping professionals make features.
There’s a lot of talk about AI features, but there should be at least as much talk about AI helping make non-AI features.
This is exactly the idea behind what I’m trying to do here at Floorboard. If you’re looking for help implementing these sorts of workflows in your business or advice about the AI landscape in general, shoot me an email at keanan@floorboardai.com. Let’s talk about how you can start to see these kind of productivity and efficiency gains in your business.