This article is a working draft; I will update it as I go.

If you have yet to read the first part of this series, you can find it here.

We built a tool to help us analyze GitHub issues. Below is the result in action or a page.

Regarding the output:

• Titles are not passed through GPT, and they are raw user entries. Normalizing the title is a good next step.
• The body is more evident than the title most of the time.
• Red boxes are issues that need to be classified correctly.

Sampling over 31 issues “user-support” (10%) and doing this classification manually, assisted by the AI tool, we make this classification:

TL;DR

This post showcases a creative approach to analyzing issues in the Ordinal GitHub repository by leveraging the GitHub API, GPT-3.5, and GPT-4. The author demonstrates how to obtain the necessary data using the GitHub CLI, then uses GPT-4 to generate a prompt for GPT-3.5 to analyze the repository’s issues further.

Some key takeaways from this analysis include the following:

Automating the collection and analysis of issues and pull requests can save time and effort. AI models like GPT-4 and GPT-3.5 can help generate valuable insights and overviews of the issues. This approach has certain limitations, such as model availability and token limits. However, overall, the method proves to be efficient and cost-effective. The author provides links to the source code and the raw output of the analysis, along with examples of how to filter the result using jq play. This solution offers a practical way to quickly gain an understanding of a large corpus of issues in a repository, making it easier to identify and prioritize tasks.