A high traffic law news website that we have worked on many times had a long-time problem with Google News – a problem that we were made aware of a few weeks ago. It wasn’t a serious problem, but it was quite annoying: whenever they posted a new article on the website, Google News would index the article, but would not show the article’s image in its listings. Now, of course, this isn’t catastrophic, but this problem reduced the traffic directed to this website (people are more willing to click on articles with images) and, of course, made the website look slightly unprofessional.
Now, any developer would agree with us that the worst website problems to solve are those that cannot be directly controlled from within the website. And that problem was exactly that – no one has control over what Google decides to show and how it indexes websites. Of course, there was a problem on the website, but we just didn’t know what it was because it wasn’t actually happening on the website.
So, what did we do?
We tried everything. And when we say everything, we mean everything. We read Google News‘ technical requirements (available here) many times, many many times, but with no avail: the image simply wasn’t displaying – instead – the alt attribute of the image was displaying.
What made things even more complex is that we didn’t know whether our fixes worked or not immediately as we had to wait until Google indexes the website. We also couldn’t just add test articles to the website (and then delete them) because of its importance in the legal world (the team managing the website had zero tolerance for on-site debugging).
We were about to declare defeat, total defeat, when a lightning bolt struck us (well, not literally as there aren’t lightnings this time of year in Montreal) after re-reading the technical requirements for the gazillionth time (we are slightly exaggerating here, we only read it about 2 billion times), where we noticed this little sentence about the robots.txt file: “Please read our Help Center article about robots if you believe your site’s robots.txt file, meta tags, or HTML header specifications might be blocking our crawler from accessing your content.” Well, what if it was the robots.txt file?
So, we checked the path of the image of a random article on the website, and we noticed that the path was something like: http://www.ourclientjoomlawebsite.com/cache/multithumb_thumbs/image_file_name.jpg (the client uses the BK-MultiThumb plugin for generating thumbnail images).
But, Joomla’s default robots.txt clearly instructs search engines not to index anything in the cache folder! That’s it! That is really it! Google News wasn’t displaying the images because we told it not to in the robots.txt file! So to fix the problem, all we had to do was to tell Google to index the cache/multithumb_thumbs folder by adding the following line to the robots.txt file immediately after Disallow: /cache/:
Allow: /cache/multithumb_thumbs/
Phew! That simple line fixed a year-old problem!
It really took us a long, long time to get to the bottom of this, and we are glad that we have shared our knowledge in this post so that other people with the same problem will be able to easily fix it. But, if even after following the instructions above, you are still having problems with Google News not displaying your article images, then please contact us. We’re always happy to help, we love solving problems, and our rates are super affordable!