Like many people, perhaps including you, I’ve been exploring the possibilities around OpenAI’s ChatGPT tool. I’ve already used it for many of the SEO (Search Engine Optimization) use cases discussed on social media and elsewhere in detail over the last couple of months.
In this insights article, I’ll share some interesting data about whether the content created with ChatGPT will or won’t rank when detected as AI-generated.
Why?
Creating websites for the purpose of signal monitoring allows us to check if something continues to work through Google’s algorithm updates. These can sometimes initially be a website you use as a playground for SEO testing. This is particularly useful right now with AI content, including ChatGPT content.
There is huge potential in using AI content as an assistant for creating better content than what humans can do alone.
The research phase, planning phase, and actual creation phase should all get some attention. However, using the content without editing has been scorned as a bad idea because it’s easy to detect most AI-generated content.
I agree with the arguments, but many sane arguments in the SEO world have often been proven wrong. I prefer to test and create a monitoring site to draw conclusions from the data.
So I created a website with around 50 pages of content with at most 500 words per page to see what happened.
What happened?
The website was created on a year-old partial-match domain with no links. The niche is a low-medium competition niche in the health/nutrition sector.
All the same, it burst out of the blocks quickly. The visibility dropped off over the holiday period, which could be seasonal trends or due to something happening with Google. It’s hard to know for sure, but the visibility did start to return after the holidays.
Around 10-15 articles are still not indexed, which could be due to the following:
- Google does not see them as valuable enough.
- Someone else has already used ChatGPT to create and publish this content, which flags mine as duplicate or plagiarised.
- Simply because they don’t like AI content but will happily use it on less competitive queries.
Does Google know it’s fake?
This GPT-2 Output Detector works fantastically at detecting unedited ChatGPT content. This is despite ChatGPT using GPT-3.5 (at the time of writing). You can use other detectors, but this one worked fine for me on this particular website.
The first article I checked came back with a 99.98% probability that it was “fake content,” e.g., AI-generated.
While this is a great tool, it’s hardly sophisticated compared to some paid options on the market. Especially the ones that work with the latest GPT models. It’s safe to say that Google has something far better than anything available anywhere. So if these tools think it’s fake, Google will also know that.
AI-Detection Vs. Google Rankings
So the successful performance and abundance of detection tools presented a useful opportunity.
- Compare the ranking articles to the non-performers.
- Check to see whether there is any correlation between detection and search performance.
Here are some of my results.
| Article ID | Main KW Position (US-Mobile) | AI Probability % |
|---|---|---|
| 1 | 3 | 99.98% |
| 2 | 4 | 99.98% |
| 3 | 5 | 99.98% |
| 4 | 5 | 99.98% |
| 5 | 5 | 99.98% |
| 6 | 68 | 99.98% |
| 7 | Not indexed | 99.98% |
| 8 | Not indexed | 99.97% |
| 9 | Not indexed | 99.98% |
| 10 | Not indexed | 0.40% |
While this is only around 20% of the article on the website, I’ve tried to include a mixture of performing, non-performing, and non-indexed.
What we can see right now is how Google doesn’t seem to care that the content is obviously fake.
Why?
- The information is accurate.
- It summarizes the answer close to the beginning.
- The content has no fluff.
Ultimately, the content provides value to the user and to Google’s index. The queries are low-medium competition in a niche with no obvious competitors benefiting from topical relevance at the domain level.
We can even see that one of the non-indexed articles comes across as 99.60% real and doesn’t index, while almost every other indexed article on the site is in the top ten.
Conclusion
While it’s hard to conclude anything definitive from this, it’s obvious that Google does not care about AI content at this point, which makes sense. If something is actually valuable, what’s the issue?
Do we need to edit the content to make it valuable? Probably not.
It’s either valuable or not – it’s how you create the content that matters and how much work you need to put into that varies depending on the query.
So, it’s probably safe to use AI content, whether it’s ChatGPT or anything else – for now. That’s my overall conclusion here, and the site will continue to stay online without editing so that I’ll know if Google changes its mind.
In the meantime, using ChatGPT content on existing websites with editing is probably a very safe bet.