How Good Is Buddy?

In this chapter, we dive into the world of Buddy, an AI assistant that can do some pretty cool stuff like answering questions about your code repository, providing helpful code snippets, and even autocompleting your work. But as you spend more time with Buddy, you'll start to notice its quirks. Sometimes it struggles in longer conversations or gets stuck repeating incorrect information. You might also find that asking the same question can yield different results depending on how you phrase it or which conversation you're having.

We’ll look at specific scenarios where Buddy falls short, such as when it falsely claims to have used a tool like read_file even though it didn’t actually do so. Despite these challenges, experimenting with different models and tweaking the system prompt can sometimes help improve its performance. So grab your curiosity and let's see how far we can push Buddy!