Have you ever wondered what your conversations with AI reveal about you? In seeking a solution to a work task, accelerating your learning or just having fun, perhaps our ChatGPT conversations might reveal insights about our behavior, thoughts, aspirations, and even who we are?
ChatGPT and our interactions with it represent a dramatic step in our evolving relationship with technology. AI is not just a tool but a partner in our digital lives, a co-creator in our creations, and potentially a constant companion. I believe understanding our interactions with these advanced systems is becoming increasingly important.
Like many, ChatGPT has had a dramatic effect on my work, journaling and even writing. When I’m struggling on a creative project, trying to navigate a new topic or skill I’m learning or simply trying to ideate and deliver work, I often start a conversation with ChatGPT. I even use ChatGPT to empower self-reflection and generate questions.
How often do we pause and try to understand our interactions with AI (or any of the other dozens of internet, computer and technology products firmly embedded in our modern lives)?
For nearly a decade, I have take seriously the idea that data from the digital products we use whether for health tracking or productivity can help us see ourselves different, change our behavior and even cultivate greater self-awareness. I track a few dozen areas of my life and create a few yearly data visualizations. I’ve written extensively on the topic of self-tracking and built several so-called “quantified self” products centered on leveraging data to lead a data-driven life.
As an extension of my passion for personal data analysis and visualization, found in one of my most popular open source projects Quantified Self Ledger, I recently embarked on a new venture to try and understanding my dialogues with AI, specifically my conversations with ChatGPT. Quantified ChatGPT is an open source project I designed to enable me to process, parse and visualize my interactions with ChatGPT.
In this post, I’ll share the background on Quantified ChatGPT, a tool I’ve developed to not only track but also reflect on my interactions with ChatGPT. First, I’ll provide some of the technical how-to for setting up and using the tool. I’ll then dig into my own AI interactions over roughly the last year and look at a few specific data analysis and data visualiation questions, namely how often do I use ChatGPT and what are my most common conversations. From exploring usage patterns centered on quantitative analysis, I’ll conclude on how potentially this project and our usage of AI will continue to evolve and create new insights for personal and professional growth. Part art project centered on data visualization, part cultivator of self-awareneness, Quantified ChatGPT is more than just a data tool; it’s a window into our evolving relationship with AI.
Quantified ChatGPT
About Quantified ChatGPT: Dialogue Repository, Data Analysis and Data Visualization for ChatGPT Export
Quantified ChatGPT is a tool I designed to help me gain insights into my interactions with ChatGPT. Quantified ChatGPT is intended to be an enabler into seeing, exploring and better understand various data about ourselves.
It’s in the same spirit as my larger open source project around personal data analysis and data visualization called Quantified Self Ledger, which I wrote about here. The goal of both projects is to provide a starting point for data analysis and data visualization using data from tools we use. This could be merely for novelty or it could help you cultivate new forms of self-awareness and paths to self-discovery.
Currently Quantified ChatGPT uses an export of your conversational history and then parses it into various outputs, including a plaintext markdown document for each conversation, a csv for statistical analysis and various charts and graphs.
Using an analysis of your ChatGPT history, it can then generate a summary output of your top chats and your main questions or interactions. This summary output can then be input into ChatGPT to then further articulate trends on how you are using the AI service.
In the future, I hope to convert these charts and statistic logs of usage into more human readable summaries and reflection questions and feedback on my usage and tasks. Obviously it makes sense to use ChatGPT to help you understand how you use ChatGPT.
About the Tech:
Quantified ChatGPT is an open-source Jupyter Notebook written in python. You’ll need some basic python skills to use this project. You can read my note here from separate project about setting up python and Jupyter on your local machine. If you are interested in a web service or want help, contact me.
What does it do?
- Create and saves a repository locally for quantifying your interactions with ChatGPT over time.
- Converts your ChatGPT conversation history into human-readable exports (in markdown) and structured data (i.e. csv).
- Analyzes and visualizes your chat data into some charts and graphs for a better understanding of your dialogues.
Basic Usage / Quickstart
- Export your ChatGPT history and data following OpenAI’s documentation here.
- Extract ZIP.
- Place the file
conversations.jsonintodatadirectory such that the file is available atdata/conversations.json - First run
chatpt_data_parser.ipynbusing either jupyter notebook or jupyter lab which will convert your raw JSON file into a simple CSV/spreadsheet. - Second run
chatpt_data_analysis.ipynbfor data analysis, data visualization and some charts on your ChatGPT utilization.
NOTE: There is some optional code for a Markdown converter that takes your conversations log and converts it to plaintext markdown files.
Sample Outputs:
Example of Month-on-Month Chat Frequency
Local repository of my ChatGPT Conversations in Markdown
Exploring My ChatGPT Usage: How am I using ChatGPT?
At present, Quantified ChatGPT is a pretty simple tool. It basically brings your ChatGPT history into some local files and generates some nifty charts.
Let’s first look at two charts that offer some insight into my ChatGPT usage.
How often am I using ChatGPT? My Monthly Usage Trends
So far, I’ve had a total of 273 ChatGPT conversations. I have an an average of 14.1 messages per conversation.
Based on message count, here is what my usage of ChatGPT has looked like in recent months:
There is considerable variance in my usage and messsage count. Per month on average, I have 385.2 conversational messages with ChatGPT. My peak was October 2023 at 812 monthly messages and my lowest was just 37 message in March.
One thing you might notice is that my ChatGPT history only started in March 2023. Due to using ChatGPT on a separate account and some issues following an OpenAI upgrade, I lost a bunch of early conversations with ChatGPT.
April and May 2023 saw a significant increase before decreasing in June, July and August. September and October have seen a marked increase in my usage. As will be clearer from most frequent conversations analysis, this increase in recent months corresponds with ChatGPT’s evolve role in aiding me in learning, writing and and journaling as well as its increasing role for coding tasks.
What am I conversing about with ChatGPT? My Most Frequent Conversations
The obvious I was most interested in was: What were my most active conversations with ChatGPT?
Here is a chart:
With some manual labeling, here’s the breakdown and categorization of my 20 top AI conversations:
Coding (3 conversations, 408 messages):
- 268 in Flutter and Dart Overview
- 91 in FireCMS Assistance for Database Needs
- 49 in Drupal 9 Module
Reflection Note: ChatGPT can be a real enabler for developers and technical product managers like me. Coding with ChatGPT was a bit hit or miss for me early on, but it continues to improve. It would be interesting to compare my conversation count using CoPilot directly in my IDE (VS Code). There is little doubt that ChatGPT as a coding assistant can help you learn, debug and write code. It’s by no means perfect yet, though I suspect it will improve rapidly.
Startup/Product Ideation (4 conversations, 258 messages)
- 104 in Goal Planning Startup Assessment
- 54 in Journaling for Course Reflection
- 52 in Reflection App
- 48 in ScienceOfGoals Business Ideas
Reflection Note: As an entrepreneur, startup mentor and product designer, I get a lot of value out of exploratory conversations using ChatGPT. It can quickly take basic ideas and flesh them out. It does a good job of imagining customer profiles, user needs, and user journeys before generating strategy docs, UX/UI flows and even end-to-end mock user stories with quotes. It significantly shortens the time to visualize and imagine the full scope of a startup or product idea.
Creative writing, like Blog Posts (8 conversations, 504 messages)
- 94 in Empowering Self-Understanding with AI
- 82 in Authenticity Reflections
- 76 in Tangible Artistic Creations - V1
- 58 in Narnia Word Counts
- 58 in Review & Revise Blog Post
- 46 in Data-Driven Growth Hacking Guide
- 45 in Music Album Spreadsheet Assistance
- 45 in Productive Readiness
Reflection Note: As I explain in AI as a Journaling Companion, ChatGPT has become a powerful enabler for my own journaling and reflection processes, including generating early prompts for my blog and book writing. Rather than using it to give me answers, I probe it to give me questions, ideally questions that can me think about myself, clarify my intentions and reflect critically on myself and my projects. While I think it’s crucial to write up your own initial thoughts, rather than outsource it to AI, ChatGPT can help you express your thinking and explore your idea by generating reflection questions. Whether it’s about sparking creativity or finding an appropriate question to nudge you to think about yourself and a situation, ChatGPT is a very capable and powerful reflection question generator.
I still don’t feel comfortable with ChatGPT for generating actual text copy for my writing. In particular, the voice, tone and language is often quite generic and cliche, and if I attempt to use it to write for me, I tend to struggle to still feel in control of the overall writing and flow.
Even though I now regularly use ChatGPT to give me journaling prompts or pre-writing reflection questions for my blog and book writing, for me, it’s important to still feel like the owner or driving in my journaling and writing. This might result in writing that isn’t quite as crisp or clear as what you get from ChatGPT alone. But doing so I feel like I lose aspects of my authentic voice and core intentions in what I want to communicate and write on my own.
Marketing (3 conversations, 220 messages)
- 86 in BioAge Tests
- 72 in Create Video Script.
- 62 in Kickstarter Campaign Preparation Tips
Reflection Note: In small and big ways, ChatGPT has help me in my product marketing work and growth hacking. It can help generate value propositions, define mock customer profiles, and even strategize marketing copy. I particularly like it for creating alternative marketing headlines for ads and social media.
Studies/Research
- 78 in Define Reflection
Reflection Note: Personally I would have guessed that learning was my top conversation category. It’s possible it is more long-tail, meaning lots of short conversations. That said, I still think ChatGPT will remain a key component in my approach to learning and synthesis of my research. I particular like to use LLMs to help me define key terms and provide study guides and outlines based on specific questions and intersections that I’m curious about. All in all, using ChatGPT seems to shorten my learning time on new topics and ideas and gets me excited about particular tangents, rather than just generic knowledge we might get from Wikipedia or online articles. It pushes and I push it to delve into novel aspects of knowledge and comprehension.
Conclusion: Future of Meta-Reflection on AI Usage
Quantified ChatGPT is intended to be an enabler into seeing, exploring and better understand various data about ourselves.
In this post we looked at how to setup and use my open source project, Quantified ChatGPT, a tool I designed to help me gain insights into my interactions with ChatGPT. We looked at what it revealed it terms of my trends and top conversations.
Like Quantified Self Ledger), Quantified ChatGPT has opened new avenues for me to understand my interaction with AI and with technology in general. It’s not just about the quantity of conversations or the frequency of usage. It’s about recognizing patterns, drawing insights, and, most importantly, using these findings to foster a more mindful and intentional approach to technology.
Reflecting on how one is using an. LLM like ChatGPT requires both qualitative and quantitative analysis of those interactions. At present Quantified ChatGPT provides a technical solution geared towards the quantitative, i.e. how frequently, top conversations, and types of conversations.
There is still a huge opportunity in my own project and for OpenAI to offer more insight into the type of conversations you are having. Personally, I’d like to start categorizing and tagging my conversations in order to know what type of conversations I’m having with ChatGPT. For example, a few questions come to mind like:
- What percentage of my conversations are geared towards pure learning, startup ideation, coding, personal writings for my blog and book, or working on a specific work task, (like a meeting or deliverable)?
- What are the main topics a single conversation has and do any of these topics overlap with other conversations I’ve had?
- How are my interests, core topics and conversations evolving over time?
- What are some potential intersections, gaps or follow-ups I might be missing and that ChatGPT might recommend I consider?
In short, I believe future versions of this tool and ChatGPT itself will grow towards some form of meta-cognition and meta-reflection about our AI usage and our selves. For example, how might we taking our trends and summary output and re-feeding into ChatGPT for deeper analysis and follow-up reflection questions?
As I continue to evolve with this tool (and evolve with AI), my hope is that others too can find value in such analysis, using Quantified ChatGPT to enhance their understanding of their AI and technology usage. Whether it’s for improving productivity, aiding in learning, or simply exploring the creative frontiers of AI interaction, the potential is vast.
How are you using AI tools like ChatGPT? What are your trends? What are your top conversation types? How do you see yourself in your AI interactions?
As we stand at the intersection of AI and self-exploration, what new frontiers of understanding might we uncover about our digital selves?