Last week, Anthropic (with a $4 billion investment from Amazon) announced that Claude 3.5 Sonnet now supports data analytics.
TechCruch’s headline for this capability was: “Anthropic’s AI can now run and write code.” That’s technically correct, but don’t get too excited. Claude isn’t going to sit there and simulate all your code for you. The reality is a lot more limited.
Also: Can AI make data science obsolete?
The feature Anthropic announced is similar to ChatGPT’s Advanced Data Analysis. One difference is that Claude’s analytics tool is available to everyone, including free users. ChatGPT’s advanced data analysis is only available to Plus and Enterprise account users who pay $20 or more per month.
Generate code
Both ChatGPT Plus and Claude perform their data analysis by writing and running code snippets that parse and process the data. An important difference is that Claude writes his code in JavaScript, while ChatGPT writes his code in Python.
Also: The Most Popular Programming Languages in 2024 (and What That Actually Means)
These are interesting choices. Python has a rich ecosystem of numerical analysis libraries such as Pandas, NumPy, and SciPy. JavaScript also has a rich ecosystem, but its data and AI offerings are not as extensive as those for Python. Python is very strong in machine learning and AI, with frameworks such as TensorFlow, PyTorch and Keras. Python also has excellent support for big data, although as you’ll see, none of Claude’s current analytics tool can even be considered medium data.
JavaScript, on the other hand, is ideally suited for data visualization on web pages. The Anthropic solution uses React, but there are also great visualization libraries such as D3.js and chart.js available for information presentation. I found it strange that with such great visualization tools, the pie charts I generated with Claude tended to cut off the data labels for some categories.
Also: The Best AI for Coding in 2024 (and What Not to Use)
When you ask Claude to process data, you get its output, but you can also look at the underlying code that is generated to do that data analysis. Here’s an example.
Usage limits
I decided to use Claude to test its analysis capabilities. I have limited my use to the free version. According to Claude’s FAQ, the $20/month Pro version increases usage limits by five times.
That’s probably necessary for serious use, because after about 20 minutes of testing I got shut down.
I tried opening a new chat, but it wouldn’t let me in anymore. After waiting an hour I was able to ask some questions.
Write code to clean data
To test Claude’s data analysis capabilities, I went to the data.gov website and downloaded a Social Security Administration dataset on the use of baby names derived from applications for Social Security cards.
This data came in the form of a ZIP file. I extracted 145 comma-separated text (CSV) files of baby name data from 1880 to 2023, one file per year.
Also: I compared Claude 3.5 Sonnet to AI coding tests that ChatGPT passed – and it failed creatively
I first tried selecting all the files and importing them as a group into Claude. I was told that Claude would only import five files at a time.
So I decided that I would write a script that would create a single file with all the data. The problem was that each individual file did not contain the year as one of the fields. So my script would have to add the year from the file name to each record in the file, and then do this for all files.
Instead of coding this myself, I asked Claude to do it for me.
I need to quickly combine 145 text files on a Macintosh. Each file name consists of the letters yob followed by four numbers, indicating the year, followed by .txt. The files themselves are comma separated values. For each file, I need to put the year in the file name before each line in the corresponding file, followed by a comma. Next I need to combine all 145 files into one main file. How can I do this quickly?
It created a shell script that looked like it would get the job done.
I saved and ran the script.
It worked and did exactly what I asked. The result of running that shell script was a 37 MB file. Unfortunately, I quickly discovered that 37 MB exceeded Claude’s upload limit of 30 MB. I needed a data set that was significantly smaller.
Instead of using name data from every year, I thought that if I used name data from just one file per decade, I would reduce the size of my dataset to 10% of its original size. So I changed my prompt and gave it back to Claude.
I need to quickly combine 145 text files on a Macintosh. Each file name consists of the letters yob followed by four numbers, indicating the year, followed by .txt. The files themselves are comma separated values. Because each file has a file name that ends in 0.txt, you must put the year in the file name before each line in the corresponding file, followed by a comma. Then all files ending in 0.txt should be combined into one main file. Write a shell script for that.
That worked just as well as the first prompt and I got a 3.9 MB file.
Overall, I was quite pleased with today’s Claude 3.5 Sonnet coding work. I previously passed that LLM through my battery of coding tests without much success. So it was nice to see that things went smoothly this time. Unfortunately, that was the last part of today’s testing process that went smoothly.
More boundaries with Claude
So let’s take a look at Claude’s data analysis. Unfortunately, Claude seems to be very limited in the amount of data he can process. Claude says that the Pro version allows “at least 5x the usage compared to our free service” and that “if your conversations are relatively short, you can expect to send at least 45 messages every 5 hours.”
Also: AI scams have infiltrated the knitting and crochet world – why it matters to everyone
That’s not much. And while Claude says you can upload five files and 30MB, I found that my combined 3.9MB file was a whopping 9124% over the length limit. That file contains 219,181 records.
Okay, fine. So then I tried a file for just a year. The yob2020.txt file is only 561 KB in size and contains only 31,550 records. That file is apparently 1239% longer than Claude’s length limit.
If you do some math, and assuming you haven’t reached the message usage limits, it looks like Claude limits his data analysis to about 2,000 lines of about 25 characters each.
Let’s compare that to ChatGPT Plus, shall we?
Now, yes, I’m using the free Claude version, but if Claude Pro offers 5x the capacity, we can generalize (because the company doesn’t offer hard limits) that Claude Pro could hold a maximum of about 10,000 lines of 25 characters.
Also: Anthropic’s latest AI model can use a computer like you – bugs and all
In contrast, I entered 69,215 records with an average of 50 characters per line into ChatGPT Plus and it worked fine. I entered a 22,797-record dataset consisting of sentiment data from users who had uninstalled my apps (with most records containing both sentiment phrases and fixed data) into ChatGPT Plus and it worked fine. I imported two files consisting of over 170,000 lines of G-code from the 3D printer into ChatGPT Plus and it worked fine.
I found ChatGPT Plus’s data analysis really useful and productivity-enhancing. But if a pro account was limited to just 10,000 records or less, as Claude Pro seems to do, I probably would have found it an interesting technology demonstration, but not something I could reliably add to my workflow package.
Actually testing Claude’s data analysis
I downloaded about thirty datasets from data.gov before I found one small enough for Claude to examine. That’s a November 2020 dataset of adoptable pets from the Montgomery County Animal Services and Adoption Center in Derwood, Maryland.
This dataset contains 85 records of approximately 190 characters each. Let’s see what it can tell us.
With the question “What can you tell me about this data?” Claude identified the most common pet type (dogs), the most common types of pets (owner surrender and wandering – that just seems so sad), notable patterns and unique features (Molly is a common name).
I asked for a pie chart showing the distribution of the animals. It gave me this, which showed the major animal species but left ‘Other’ at almost 50% of the bar chart.
I wanted to know what that “Other” category represented. There is something poignant about the idea that thirty percent of the ‘Other’ category consists of tropical fish. I have this depressing vision in my head of row after row of goldfish bowls, each containing one lonely goldfish.
Look at that graph and the one just above it. Note that although there is enough space for the diagram to display the labels, they are cut off in both diagrams. I know there are about 30 percent tropical fish, but I don’t know the exact percentage because all it shows is a “3”.
JavaScript has excellent graph libraries. I would think Anthropic could have adjusted the output to fully display the map data, especially in landscape view.
Well, that’s a shame
I was really hoping that Claude’s data analysis features would be similar to those of ChatGPT Plus. Even if Claude’s free version could only do a fifth of what ChatGPT Plus could, I might have signed up for a subscription.
I really like the idea of running my data through multiple analysis tools and comparing the results. That alone would have justified my signing up for another $240/year AI bill.
Plus: Google’s AI podcast tool transforms your text into stunningly lifelike audio for free
But since it’s clear from my extrapolations above that the Claude Pro version can’t handle even the smallest data sets I’ve previously successfully entered into ChatGPT Plus, it certainly doesn’t seem worth the investment.
I’ve reached out to Anthropic for comment but haven’t heard back yet. If the company responds, I will update this article with the feedback.
In the meantime, what do you think? Have you used advanced data analysis in ChatGPT Plus? Are you a Claude or ChatGPT user? When would you or would you ever consider using Claude instead of ChatGPT? Let us know in the comments below.
You can follow my daily project updates on social media. Make sure to subscribe to my weekly update newsletter and follow me on Twitter/X at @DavidGewirtzon Facebook at Facebook.com/DavidGewirtz, on Instagram at Instagram.com/DavidGewirtz and on YouTube at YouTube.com/DavidGewirtzTV.
