Almost 11 years ago, we published an article on how to disable hit tracking in Joomla. The process explained in that old article worked for Joomla 1.5, 1.6, 1.7, 2.5 and 3 (it didn’t work for Joomla 1.0 because the code in Joomla 1.0 was more of a spaghetti code than real code).

The solution described in our old post consisted of a core modification – which is something that we love to do (for some reason), but it is also something that should be avoided when possible. Joomla 4 and 5 have a new option that allows you to disable article/category hit tracking from within the CMS (without any core modification), here’s how:

• Login to your Joomla website.

• Click on Content -> Articles on the left menu.

• Click on Options on the top.

• Click on the Articles tab (this should already be clicked by default).

• Scroll down below, and change the value of “Record Hits” to “No”.

• Save & Close the configuration settings for the articles.

• That’s it! You’ve now disabled article hit tracking on your Joomla website. Not only that, you’ve disabled hit tracking on the categories as well – as both the articles and the categories share the same setting.

As you can see above, the solution is simple and straightforward, but, if you like to do it in the core itself (one reason to do it from the core is that the mod_articles_popular module will not work if the “Record Hits” configuration setting is set to “No”), then here’s how to do it:

• Open the file ArticleModel.php file under the components/com_content/src/Model folder of your Joomla website.

• Add the following line at the very beginning of the function hit:

  return true;

• Save the file and upload it back. Note that if you want to also disable hit tracking for categories, you will need to make the exact modifications to the CategoryModel that is located in the same folder.

So, what is hit tracking anyway?

Hit tracking consists of incrementing the views of an article – so, if Joomla says that an article has 31 hits, this means that the article has been viewed 31 times.

Why would anyone want to disable hit tracking?

Hit tracking is a very useful feature, and it is used in several Joomla modules/features, such as the mod_articles_popular module that is used to display the most viewed articles. That’s the bright side of things, the other side of things though is that this feature is heavy – very heavy especially on large Joomla websites (e.g. news websites) as it modifies/locks the #__content table for every page view. If left unchecked, this feature can crash a large Joomla website during peak traffic hours. So, an alternative method to record the views must be used – for example, on very active/large Joomla websites, we send a hit request about 5% of the time.

Is hit tracking only available on Joomla content items (e.g. articles/categories)?

No, it is also available on contacts items (contacts and categories), newsfeed items (newsfeed and categories), and tags. Hit tracking on each of these items can be disabled from the Options of the respective component.

We hope someone, somewhere found this post useful. If you need help with the implementation or if you need any advice, then please contact us (our fees will apply).

Last week, we were tasked by a new client to make the usernames on their Joomla 5 website case sensitive. For example, they wanted to have something like “admin” and “Admin” as different usernames – this of course, means, that the logins are also case sensitive (in case you didn’t notice, by default, you can use “john” or “John” or “JOHn” as the username when logging in to a Joomla website, and they’ll all work, even if the username was initially added as “john”).

It was a fun task… We first thought about changing the Joomla core to accomodate the request, but the more we dug into it, the more we discovered that we needed to work on the database to make this happen – and this is because varchar strings, in MySQL, are typically case insensitive – this means that even if we solve the problem on Joomla’s end, we will also need to solve it on MySQL’s end. Then it hit us: what if we change the string type of the username field to something binary (instead of a regular string) – all collations have a binary option, which should make this test a breeze!

Here’s how to do change the username‘s case insensitive collation to a binary collation:

• Open phpMyAdmin and select the database powering your Joomla website.

• Go to the #__users table (where #__ is your database prefix).

• Click on Change on the username row.

• Under Collation, select utf8mb4_bin instead of utf8mb4_unicode_ci.

• That’s it!

We tried the above and, to our surprise, it worked. We were able to add the same username with uppercase characters, and Joomla accepted it. In addition, Joomla automatically imposed case sensitivity on the username. This whole experiment meant that Joomla relied exclusively on MySQL when it comes to case sensitivity for usernames.

But the “password” field’s collation is “utf8mb4_unicode_ci”, still it is case sensitive – how?

We’re glad you asked – Joomla does not compare the password field as it is, it converts your input to a hash, and converts the password in the database to a hash (this automatically imposes case sensitivity), and then compares the two. It doesn’t do a simple string comparison between the password inputted by the user and the password in the database.

Note: This has been tested on Joomla 4 and Joomla 5 – it should work on other versions of Joomla but we haven’t tested it.

We hope you found this post helpful and informative – if you need any help with the implementation, then please contact us. We are very experienced in Joomla, we are friendly, and our fees are affordable!

So you’re working on a new extension, or in the Joomla core itself, and you’re using a DatabaseQuery object, and you want to print the query object as a SQL string. How do you do that?

Well, assuming the name of the object is $query, there are 4 ways to do this:

1. echo $query->__toString();: This statement prints the SQL string out of the $query object, but there’s a catch: the SQL string has the database prefix as #__.

2. echo($query);: This simple code is equivalent to the previous one, since (in PHP) printing an object as a string is really calling the __toString() method of that object.

3. echo($query->dump());: This code is equivalent to the first and second code, with one minor difference: the #__ is replaced with the actual database (or table) prefix (or alias). This is the best method to print the SQL string out of the DatabaseQuery object.

4. str_replace('#__', $query->db->getPrefix(), $query): This code yields the exact output as the previous one – this is normal, because the dump method simply consists of this str_replace code.

Note that all 4 methods work in Joomla 2.5, 3, 4, and 5 – it’s probably worth to mention though that the implementation of the __toString method (which is the base method used by all the other methods) differs between Joomla 2.5/3 and Joomla 4/5.

So, which method is the most used to print a SQL query? One would think the 3^rd method, but, from our observation, it’s actually the first one – which is odd, because the second method is the simplest, and the third method is the best (as it prints the SQL command with the database prefix). What’s even more odd is that the third method is deprecated since Joomla 4 with no replacement (according to the documentation). We’re not sure why is that exactly.

We hope you found this short post useful – and if you need help with anything Joomla, including development, then please contact us. Our fees are right, our Joomla expertise is unquestioned, and our work is very clean!

We monitor the Joomla sites of our enterprise clients – and we do not do so with the help of a paying 3^rd party tool – we do so with a little tool that we developed ourselves. The tool is nothing fancy, but it works really well!

Today, we are sharing full details about this small tool, including the source code!

So, what is this tool?

This tool is a simple Linux shell script that checks if the website works by grabbing the content (using wget), and seeing if the length of the content is above a certain (fixed) amount of bytes. If it’s not able to grab the content, or if the size is not above the threshold, then it emails us telling us that the website is down.

Will it know that a website is hacked?

In short, the answer is no, it won’t know if the website is hacked. This is because a hacked website, in general, does not result in smaller pages (in fact, in many cases, a hacked website will result in larger pages). Having a tool that discovers if a website is hacked is more complex (but still feasible).

Can I have the source code of this tool?

But certainly! You only needed to ask (well, not even)! Here’s how to do it:

• Create, in the Linux shell (and as root), a file called sitebeat.sh (place it under the /home/scripts/ folder – if that folder doesn’t exist, then create it), make it executable by root by issuing the following commands:

  chown root sitebeat.sh
chmod 700 sitebeat.sh

  It is important that the above file is created on a server other than the server hosting your site(s) – because if the server hosting your site(s) goes down, then your script will still work (and will send you notification in this case).

• Add the following code to the sitebeat.sh file:

  strMessage=""
  intMinimumSize=50000
  siteSize="`wget -qO - https://www.yourjoomlawebsite.com/ | wc -c`"
  if [[ "$siteSize" -lt "$intMinimumSize" ]]; then
          isError="1"
          strMessage="https://www.yourjoomlawebsite.com/ Not Responding\n"
  fi
  siteSize="`wget -qO - https://www.yoursecondjoomlawebsite.com/ | wc -c`"
  if [[ "$siteSize" -lt "$intMinimumSize" ]]; then
          isError="1"
          strMessage="$strMessage""https://www.yoursecondjoomlawebsite.com/ Not Responding\n"
  fi
  if [[ "$isError" == "1" ]]; then
          echo -e $strMessage
          echo -e $strMessage | mail -s "Warning - At Least One Website Is Faulty" youremail@yourjoomlawebsite.com
  else
          echo -e "Heartbeat is normal"
  fi

  Some explanation here:

  • The intMinimumSize value is the minimum size, in bytes, of your homepage. If you expect the minimum size to be bigger or smaller then you should increase/decrease the intMinimumSize.

  • You should change the https://www.yourjoomlawebsite.com/ and https://www.yoursecondjoomlawebsite.com/ links to your actual Joomla websites – you should also change youremail@yourjoomlawebsite.com to your work email.

  • You can remove the blue lines if you only have one website. If you have more than 2 websites, then you can clone the blue lines and just change the URL in each cloned block of code. You may need to adjust the intMinimumSize (or hardcode it for each website) when you are checking multiple websites.

• In the crontab, add the following line:

  */10 * * * * /home/scripts/sitebeat.sh #check the heartbeat of the website every 10 minutes

  The above code runs the sitebeat.sh script every 10 minutes, you can increase or decrease the frequency by just changing the */10 code – for example, if you want the script to run every 5 minutes, then you can just replace */10 to */5.

• Ensure that on the destination server (e.g. the server hosting your websites), you have the IP of the sitebeat server whitelisted. This may not be necessary, but it is a good idea.

• That’s it!

We hope you find this post informative and useful. If you need help with the implementation, then all you need is to contact us. Our fees are super cheap, our work is super professional, and we deliver very quickly!

Note: It is important to know that your Joomla setup may require any of the plugins disabled below, so it’s a good idea (make that a mandatory idea) to test the whole website during off hours or on a test server each time you disable a plugin.

As a followup to yesterday’s post, which concluded that Joomla 4 has a slight performance edge over Joomla 5, we thought: “What if we disable unnecessary plugins in Joomla 5? Will things get better? And if yes, by how much?”

We thought of a methodology to test, and we quickly came up with one. What if we disable unnecessary plugins one by one and see how much performance gain we get after each “disable”?

We used a test Joomla 5 website (the same test Joomla 5 website from yesterday, where we loaded the sample data), joomla502.com (this is an internal domain only accessibly using a hosts file). We first measured the performance of the website before disabling any plugin using the ab tool in the linux shell…

ab -c 10 -t 10 "http://www.joomla502.com/"

…and we got the following Time per request: 336.035 milliseconds.

We then started disabling plugins one by one. Here are our findings (note that once we disable a plugin we leave it disabled, e.g. we don’t re-enable a plugin and then disable the next one):

• Disabling all the Fields plugins: This group of plugins consist of the following:

  • Content – Fields
  • Fields – Calendar
  • Fields – Checkboxes
  • Fields – Colour
  • Fields – Editor
  • Fields – Imagelist
  • Fields – Integer
  • Fields – List
  • Fields – Media
  • Fields – Radio
  • Fields – SQL
  • Fields – Subform
  • Fields – Text
  • Fields – Textarea
  • Fields – URL
  • Fields – User
  • Fields – Usergrouplist
  • System – Fields

  You may be using the Fields plugins so this step might be moot, but if you’re not, then disabling the plugins is a good step, as the Time per request will drop to 313.774 milliseconds – a considerable improvement over the default load time (6.62%) with just a few clicks!

• Disabling all the Schema.org plugins: These plugins generate the schema.org HTML tags in the code. They are useful, but if you’re not using them, then you can just disable them (we believe these should be disabled by default, as this is a new Joomla feature). Here are the plugins that should be disabled:

  • Schema.org – BlogPosting
  • Schema.org – Book
  • Schema.org – Event
  • Schema.org – Organization
  • Schema.org – Person
  • Schema.org – Recipe
  • Schema.org – JobPosting
  • System – Schema.org

  Disabling them reduced the Time per request to 304.954 milliseconds (2.81% over the previous step). Not bad!

• Disabling the System – Passkey (Passwordless) Login plugin: The passwordless login is a practical but seldom used feature that allow users to login without entering a password (using fingerprint, face recognition, dongle, etc…). Disabling this plugin reduced the Time per request to 302.847 milliseconds and we saved the system from loading 16 files!

• Disabling the System – Guided Tours plugin: If you want to know, this plugin is responsible for the guided tours feature on Joomla, that is mainly available in the backend (the “Take a Tour” button at the top), and that almost nobody uses, and that loads a few unnecessary files on every page on the frontend. Having said that, we think that this is a good feature to have, especially for a custom component where you want to provide an explanation for the end user on how to use the component. The Time per request was reduced to 297.713 milliseconds – a minor but welcome improvement. In addition, we saved 5 files from being loaded.

• Disabling the Action Log plugins: This set of plugins consist of:

  • Action Log – Joomla
  • Privacy – Action Logs
  • System – User Actions Log
  • Task – Delete Action Logs

  These plugins are mainly responsible for recording user actions in the backend – so, disabling them not only lessens the number of files loaded and enhances performance, it will also prevent the Joomla website from performing unnecessary writes to the database (mostly during peak hours) which can negatively impact performance. The Time per request shot up slightly to 299.002 millieseconds – still, we believe that on real websites, these should be disabled if the action logs are not monitored by anyone, as those unnecessary database writes will throw additional stress on the server especially during peak hours.

• Disabling the Behaviour – Backward Compatibility plugin: This plugin is responsible for loading libraries that are used for backward compatibility. Disabling this plugin reduced the Time per request to 286.032 milliseconds – a welcome (4.33%) improvement over the previous step. We also reduced the number of loaded files by 4.

• Disabling miscellaneous plugins: We then disabled miscellaneous plugins, which are:

  • System – Debug
  • Content – Email Cloaking (really useless plugin in this day and age)
  • System – Task Notification
  • System – Joomla Accessibility Checker

  Doing the above didn’t result in any improvement, but we are certain that, with enough traffic and content, disabling unnecessary plugins will yield non-negligible positive performance results.

So, how much improvement did we get in total just by “clicking”? Well, let’s see, we started with 336.035 milliseconds and we ended with 286.032 milliseconds, a whopping 14.88% improvement, and this is only with a few minutes of work! But there’s more, much more serious optimization – and if you’re curious, just contact us. You will be amazed at what we can do for a relatively small price!

A very common question that we get from our clients who are hesitant to migrate to Joomla 5 is: “Does it run any faster than Joomla 4”? Our answer is: “We don’t know, but we’ll soon run some benchmarks and get back to you!”

Well, we decided that soon is now. Here’s what we did to find the definitive answer to this question…

The Environment

For our benchmarking tests, we selected a dedicated server with the following specs:

• Processor: Intel Dual Xeon Gold 6226R (this processor has 32 cores)
• Memory: 128GB DDR4 SDRAM
• Main storage: SSD
• Database storage: NVMe SSD
• Operating system: AlamLinux 8
• Hosting environment: cPanel/ WHM
• PHP version: 8.2.16
• Database: MariaDB 10.6.17

The above server should handle a large Joomla website with many articles (provided the right core optimization is done).

The Setup

We created, on the above server, 2 cPanel accounts, one called joomla442 (to host the joomla442.com domain) and the other is called joomla502 (to host the joomla502.com domain). As you may have guessed, the first one is meant to host a test Joomla 4.4.2 website, and the second one is meant to host a test Joomla 5.0.2 website.

We then downloaded Joomla 4.4.2 and installed it on the first account, and then downloaded Joomla 5.0.2 and installed it on the second account. After that, we modified the hosts file of the server and our client machine to point the joomla442.com and the joomla502.com domains to the IP of the dedicated server that is hosting the above accounts (not being cheap or anything, but we really didn’t want to buy real domains just for this test).

We then created two unique articles on the first website and copied them to the second website. Both articles were about 1,000 words (if you’re curious, we used AI to auto-generate those articles, and if you’re really curious, the first article was about Medieval England, and the second article was about Medieval Switzerland – both interesting topics for history fans). We ensured that both of these articles were featured, and then we tested each domain and it correctly listed both articles on the homepage.

We did not make any modifications to the websites – we left all the default settings as they were and we did not publish/unpublish any of the plugins or the modules. We also did not install any extension whatsoever and we used the default template, which is Cassiopeia (in case you’re curious, Cassiopeia is the name of a constellation that has a few stars).

The next step was to install ab, which is the apache benchmarking tool (part of the httpd-tools package), by issuing the following command in the linux shell (as root):

yum install httpd-tools

Now we’re ready for the comparison between Joomla 4 and Joomla 5! Are you?

The Comparison

The methodology that we followed for comparing the Joomla 4 and Joomla 5 consisted of 4 areas: the total number of files, the total number of included files (on the homepage and the article page), the number of database queries (on the homepage and the article page), and the benchmarking.

• The total number of files

  While not an important factor when it comes to performance, the total number of files in the Joomla folder indicates whether Joomla 5 is going in the right direction when it comes to removing unnecessary files that are no longer used, which is a great indicator of coding methodology. We are happy to say that this is the case, as Joomla 4 has 9,571 files, 3,293 folders, and a zipped size of 76.4 MB, while Joomla 5 has 8,989 files, 3,261 folders, and a zipped size of 69.6 MB. This means that the Joomla team has taken the initiative to reduce the technical overhead in the Joomla core.

• The number of included files

  The number of included files is (usually) inversely proportional to the performance of the website, that’s why we wanted to determine how many files each website is including/requiring when loading the homepage and an article page. We did this by adding the following code to the end of the index.php file in the root directory of each website:

  $arrResult = get_included_files();
echo(count($arrResult));

  We then loaded each website on the homepage and on the same article page.

  • Homepage

    For the Joomla 4 website, the number of included files was 581, for the Joomla 5 website, the number of included files was 660, 13.60% more files than Joomla 4 (79 more files are loaded in Joomla 5).

  • Article page

    For the Joomla 4 website, the number of included files was 590, for the Joomla 5 website, the number of included files was 668, 13.22% more files than Joomla 4 (78 more files are loaded in Joomla 5).

  Clearly, the results are disappointing as Joomla 5 is including almost 80 more files on both the homepage load and the article page load. What is interesting though was that the number of additional files is almost the same in both tests, which means that something is running on all pages that is requiring those additional files – naturally, we investigated further!

  We compared the included files on Joomla 4 with those on Joomla 5, and the biggest difference that we noticed was the additional inclusion of the Fields plugins’ files, the Schema.org plugins’ files, and some events files. Here are the main files that we think are problematic and should not be included on every page load:

  libraries/src/Event/AfterExtensionBootEvent.php
  libraries/src/Event/Application/AfterDispatchEvent.php
  libraries/src/Event/Application/AfterInitialiseDocumentEvent.php
  libraries/src/Event/Application/AfterInitialiseEvent.php
  libraries/src/Event/Application/AfterRenderEvent.php
  libraries/src/Event/Application/AfterRespondEvent.php
  libraries/src/Event/Application/AfterRouteEvent.php
  libraries/src/Event/Application/ApplicationDocumentEvent.php
  libraries/src/Event/Application/ApplicationEvent.php
  libraries/src/Event/Application/BeforeCompileHeadEvent.php
  libraries/src/Event/Application/BeforeRenderEvent.php
  libraries/src/Event/Application/BeforeRespondEvent.php
  libraries/src/Event/BeforeExtensionBootEvent.php
  libraries/src/Event/Content/AfterDisplayEvent.php
  libraries/src/Event/Content/AfterTitleEvent.php
  libraries/src/Event/Content/BeforeDisplayEvent.php
  libraries/src/Event/Content/ContentEvent.php
  libraries/src/Event/Content/ContentPrepareEvent.php
  libraries/src/Event/Model/BeforeValidateDataEvent.php
  libraries/src/Event/Model/FormEvent.php
  libraries/src/Event/Model/NormaliseRequestDataEvent.php
  libraries/src/Event/Model/PrepareFormEvent.php
  libraries/src/Event/Module/AfterCleanModuleListEvent.php
  libraries/src/Event/Module/AfterModuleListEvent.php
  libraries/src/Event/Module/AfterRenderModuleEvent.php
  libraries/src/Event/Module/AfterRenderModulesEvent.php
  libraries/src/Event/Module/BeforeRenderModuleEvent.php
  libraries/src/Event/Module/ModuleEvent.php
  libraries/src/Event/Module/ModuleListEvent.php
  libraries/src/Event/Module/PrepareModuleListEvent.php
  libraries/src/Event/Module/RenderModuleEvent.php
  libraries/src/Event/Plugin/System/Schemaorg/BeforeCompileHeadEvent.php
  libraries/src/Event/ReshapeArgumentsAware.php
  libraries/src/Event/Result/ResultAware.php
  libraries/src/Event/Result/ResultAwareInterface.php
  libraries/src/Event/Result/ResultTypeArrayAware.php
  libraries/src/Event/Result/ResultTypeStringAware.php
  libraries/src/Event/User/LoginButtonsEvent.php
  libraries/src/Schemaorg/SchemaorgPluginTrait.php
  libraries/src/Schemaorg/SchemaorgPrepareDateTrait.php
  libraries/src/Schemaorg/SchemaorgPrepareDurationTrait.php
  libraries/src/Schemaorg/SchemaorgPrepareImageTrait.php
  libraries/src/Schemaorg/SchemaorgServiceInterface.php
  libraries/src/Schemaorg/SchemaorgServiceTrait.php
  plugins/fields/calendar/services/provider.php
  plugins/fields/calendar/src/Extension/Calendar.php
  plugins/fields/checkboxes/services/provider.php
  plugins/fields/checkboxes/src/Extension/Checkboxes.php
  plugins/fields/color/services/provider.php
  plugins/fields/color/src/Extension/Color.php
  plugins/fields/editor/services/provider.php
  plugins/fields/editor/src/Extension/Editor.php
  plugins/fields/imagelist/services/provider.php
  plugins/fields/imagelist/src/Extension/Imagelist.php
  plugins/fields/integer/services/provider.php
  plugins/fields/integer/src/Extension/Integer.php
  plugins/fields/list/services/provider.php
  plugins/fields/list/src/Extension/ListPlugin.php
  plugins/fields/media/services/provider.php
  plugins/fields/media/src/Extension/Media.php
  plugins/fields/radio/services/provider.php
  plugins/fields/radio/src/Extension/Radio.php
  plugins/fields/sql/services/provider.php
  plugins/fields/sql/src/Extension/SQL.php
  plugins/fields/subform/services/provider.php
  plugins/fields/subform/src/Extension/Subform.php
  plugins/fields/text/services/provider.php
  plugins/fields/text/src/Extension/Text.php
  plugins/fields/textarea/services/provider.php
  plugins/fields/textarea/src/Extension/Textarea.php
  plugins/fields/url/services/provider.php
  plugins/fields/url/src/Extension/Url.php
  plugins/fields/user/services/provider.php
  plugins/fields/user/src/Extension/User.php
  plugins/fields/usergrouplist/services/provider.php
  plugins/fields/usergrouplist/src/Extension/UsergroupList.php
  plugins/schemaorg/blogposting/services/provider.php
  plugins/schemaorg/blogposting/src/Extension/BlogPosting.php
  plugins/schemaorg/book/services/provider.php
  plugins/schemaorg/book/src/Extension/Book.php
  plugins/schemaorg/event/services/provider.php
  plugins/schemaorg/event/src/Extension/Event.php
  plugins/schemaorg/jobposting/services/provider.php
  plugins/schemaorg/jobposting/src/Extension/JobPosting.php
  plugins/schemaorg/organization/services/provider.php
  plugins/schemaorg/organization/src/Extension/Organization.php
  plugins/schemaorg/person/services/provider.php
  plugins/schemaorg/person/src/Extension/Person.php
  plugins/schemaorg/recipe/services/provider.php
  plugins/schemaorg/recipe/src/Extension/Recipe.php

  The inclusion of the Fields files, in particular, was concerning. We thought that the latest version of Joomla 4 had those plugins disabled by default, and that’s why they were not included, but a quick check revealed that all the Fields plugins were enabled on Joomla 4, but Joomla 4 had enough intelligence to know that they were not being used, and hence they were not included. The same can’t be said about Joomla 5, that is also including the unused Schema.org plugins. As for the event files, such as the AfterExtensionBootEvent.php, we didn’t want to investigate, but we highly suspect that the website can run without them loading on every page.

  To its credit, Joomla 5 is no longer including about 24 files that were included in Joomla 4 on every page load.

  Why do we think that this has anything to do with performance?

  Every file include (whether used or not) is an overhead that, to a certain degree, impacts the performance of the website negatively. Joomla 4 is much better in that area.

• The database queries

  Judging from the additional number of included files on every page load in Joomla 5 – we were not very optimistic about the database queries, as we had a hunch that Joomla 5 will also disappoint us in the “database” area.

  In order to do the test, we added, for each website, the following code to the libraries/vendor/joomla/database/src/DatabaseDriver.php at the very beginning of the execute function, just before the $this->connect() function call:

  $sqlQuery = $this->replacePrefix((string) $this->sql);
file_put_contents("queries.txt", $sqlQuery."\n", FILE_APPEND | LOCK_EX);

  The above code writes all the queries to the queries.txt file under the root directory of the Joomla website. We first tested the homepage, and, to our surprise and delight, the queries were exactly the same. Not one single difference (other than the table alias, of course)! We didn’t believe it, considering the number of additional files Joomla 5 was loading, so we repeated the test multiple times, and every time we had the same results!

  We then tested a single article page, and also had the exact same list of queries for both websites, with just one difference: Joomla 5 ran the following additional query:

  SELECT *
FROM `#__schemaorg`
WHERE `itemId` = :itemId AND `context` = :context

  As you may have guessed, the above query is caused by the new Schema.org plugin(s) (which, in our opinion, should be disabled by default).

  So, from our little experiment, we confirm that Joomla 5 has negligible additional overhead over Joomla 4 in the database department. This is to prove that sometimes our “hunch” can be wrong.

• The benchmarking

  For most non-technical people, this is what really matters: the benchmarking. Which Joomla performs better? If you read through the whole article, then you may have seen signs that suggest that Joomla 5 is slightly slower, mainly because of the additional included files, and the minute database overhead, but let’s do some tests to make sure.

  ab is the ideal Apache benchmarking tool, and it works by issuing the following command in the Linux shell:

  ab -c 10 -t 10 "http://www.joomla442.com/"

  The above command means that we are sending 10 concurrent requests (-c 10) over 10 seconds (-t 10) to the joomla442.com website.

  So, we ran ab on both sites, and here are the results:

  ab -c 10 -t 10 "http://www.joomla442.com/"
  
  Benchmarking www.joomla442.com (be patient)
  Finished 368 requests
  
  
  Server Software:        Apache
  Server Hostname:        www.joomla442.com
  Server Port:            80
  
  Document Path:          /
  Document Length:        35062 bytes
  
  Concurrency Level:      10
  Time taken for tests:   10.020 seconds
  Complete requests:      368
  Failed requests:        0
  Total transferred:      13107792 bytes
  HTML transferred:       12902816 bytes
  Requests per second:    36.72 [#/sec] (mean)
  Time per request:       272.295 [ms] (mean)
  Time per request:       27.230 [ms] (mean, across all concurrent requests)
  Transfer rate:          1277.44 [Kbytes/sec] received
  
  Connection Times (ms)
                min  mean[+/-sd] median   max
  Connect:        0    0   0.0      0       0
  Processing:   247  264   9.9    262     317
  Waiting:      239  254   9.7    252     306
  Total:        247  264   9.9    262     317
  
  Percentage of the requests served within a certain time (ms)
    50%    262
    66%    263
    75%    265
    80%    265
    90%    270
    95%    287
    98%    301
    99%    307
   100%    317 (longest request)
  
  ab -c 10 -t 10 "http://www.joomla502.com/"
  
  Benchmarking www.joomla502.com (be patient)
  Finished 344 requests
  
  
  Server Software:        Apache
  Server Hostname:        www.joomla502.com
  Server Port:            80
  
  Document Path:          /
  Document Length:        35825 bytes
  
  Concurrency Level:      10
  Time taken for tests:   10.010 seconds
  Complete requests:      344
  Failed requests:        0
  Total transferred:      12515408 bytes
  HTML transferred:       12323800 bytes
  Requests per second:    34.37 [#/sec] (mean)
  Time per request:       290.992 [ms] (mean)
  Time per request:       29.099 [ms] (mean, across all concurrent requests)
  Transfer rate:          1220.97 [Kbytes/sec] received
  
  Connection Times (ms)
                min  mean[+/-sd] median   max
  Connect:        0    0   0.1      0       0
  Processing:   267  280  10.4    277     327
  Waiting:      258  270   9.9    268     317
  Total:        267  280  10.4    277     327
  
  Percentage of the requests served within a certain time (ms)
    50%    277
    66%    279
    75%    280
    80%    282
    90%    290
    95%    308
    98%    314
    99%    320
   100%    327 (longest request)

  We highlighted the important numbers in bold red, and they are the Complete Requests (which indicates the number of performed requests in 10 seconds) and the Time per request (which indicates the time it takes to process each request). As you can see, Joomla 4 performed 368 requests in 10 seconds, where the mean request time is 272.3 milliseconds. Joomla 5 performed 344 requests in 10 seconds, where the mean request time is 290.9 milliseconds. This small benchmarking test indicates that Joomla 5 is 6.83% slower than Joomla 4.

  But, what if we increase the number of concurrent request from 10 to 20 to 30, etc… What will the results be?

  Here comes the chart!

  

  Figure 1: Joomla 4.4.2 vs Joomla 5.0.2

  You can clearly see, from the above chart, that Joomla 4 outperforms Joomla 5 consistently.

  Note: We didn’t install the blog sample data in our tests because we were not sure that the data was the exact same on both Joomla 4 and Joomla 5. But, for those of you who insist that we try the benchmarking test with the sample data, we installed the sample data after finishing the main tests, and the results were pretty much the same, where Joomla 4 has a slight edge over Joomla 5.

  Another note: We repeated the same benchmarking on an article page, and, while Joomla 4 still fared consistently better, the difference was really negligible (less than 0.2%).

So, there you have it: with the exception of the number of files/folders in the installation file (which is really the least important factor), Joomla 4 is slightly faster/more optimized than Joomla 5 in every single aspect, just like Joomla 3.8.1 is slower than Joomla 3.7.5. There is still hope though for those of you who want to have a super fast Joomla 5 website, you can contact us, and not only we will help you match Joomla 4’s performance, we will also help you beat it (our fees apply)!

While trying to upgrade the Joomla websites for one of our clients to the latest Joomla 4 version, we ran into the infamous “red screen” Joomla error page with the following ambiguous error: “An error occurred”. This meant that the update didn’t work (duh), but, on the bright side, the website wasn’t broken either (e.g. the update didn’t fail mid-update).

So, we enabled debugging in Joomla’s global configuration the following way:

• We logged in to Joomla’ backend.
• We clicked on “System” on the left panel.
• We clicked on “Global Configuration” in the top middle.
• We clicked on the “System” tab, and then we changed the values of “Debug System” and “Debug Language” to “Yes”

Once we did that, we tried to do the update again, and this time we saw an error saying that there was a problem allocating enough memory in one of the files, meaning that we needed to increase the memory for the update to work.

We checked the “PHP Settings” tab on the “System” -> “System Information” page in the Joomla backend, and it turned out that the “Memory Limit” was set to 64M (e.g.64 megabytes), the update possibly needed double that number for it to work.

So, we opened the file .user.ini (the file needs to be created if it doesn’t exist) which is in the root directory of the Joomla website, and we added the following line:

memory_limit = 128M

We then saved the file and uploaded it back, and then tried to do the update once more, and this time it worked!

But why did the problem happen in the first place?

It seems that the latest Joomla 4 updated just needs more memory. We confirm this information as we tried to update 6 other sites, and none of the updates worked with 64M memory.

What can one do if the above doesn’t work, or if the problem is not a memory limitation issue?

Simple! Just contact us! We’ll solve the problem for you quickly, efficiently, with minimum downtime, and for a very reasonable fee!

Yesterday morning was unsettling to us – to say the very least, as were faced with a blank page when we were checking the homepage of a Joomla 4 website of one of our regular clients.

Our first thought, of course, was to enable error reporting, so, we went to the login page of the Joomla website, and then entered the login credentials, and, to our horror, we saw the following:

1 Can’t create/write to file ‘/tmp/#sql-temptable-5aaf-117a9e3-b8e8.MAI’ (Errcode: 30 “Read-only file system”)

As you may have imagined if you’re a fellow developer, the only thing that we ever read on that error message was “sql” – and so we thought that there was corruption at the database level. But then we checked every other website on this server, and they all had the same error, could it be that the whole database server is corrupted – we know that the database server is not down because Joomla was able to serve the login page, it’s just wasn’t allowing us to login and wasn’t displaying any content on the frontend.

So, we re-read the error message, and now we saw the “Can’t create/write to file” bit, which means that the problem is that the disk is full? Right?

Maybe, but we’re sure it wasn’t the case as we maintained the whole server and we knew for a fact that all disks/partitions on that server were < 20% full, but in this day and age, everything’s possible.

Everything was normal when we logged in to the server, and no disk/partition was full. In fact, we had plenty of space on every disk/partition on that server. Could it be that the system is suddenly unable to write to the “/tmp” folder?

So, the next step was checking the “/tmp” folder, and the moment we issued the following command in the Linux shell:

cd /tmp

We were greeted with the following error:

-bash: cannot create temp file for here-document: Read-only file system

Oh! “Read-only file system”, what does that mean? The permissions were OK (in fact, the /tmp folder’s permission were set to 777), so it’s not that. Could it be what every system administrator dread?

We checked the logs, particularly the /var/log/messages file, and we saw the following:

Aug 28 05:24:09 prod02 kernel: JBD2: Detected IO errors while flushing file data on sdb5-8

It was, as we feared, a hard disk error (sdb5 was a disk partition). In fact, we felt it was a hard disk error from the get-go, but we chose to ignore our feelings, which is obvious when you notice that we ignored seeing the flashing “Read-only file system” in every error message until we had no other option but to see it! (By the way, when we first saw the blank page, we thought that the server was hacked, which was even a much more dreadful scenario).

So, how did we fix the problem?

Restarting the server (through the client’s hosting provider) fixed the problem – but we all know it’s temporary. A permanent solution is to replace the problematic hard disk, and, better yet, replace the whole server, which is what we’re doing at the moment.

We hope that you found this post useful. If you’re running into the same problem on your website(s)/server and you want help, then don’t hesitate to contact us. We will solve the problem for you and we’ll get your website(s)/server up and running in no time (well, in as little time as possible), and for very little money.

We were recently tasked with an interesting and what we thought was a simple task: a client asked us to write a script that checks if a user is logged in to a Joomla 4 website from an external website using Ajax. “Simple!” – we thought: we can just copy the code of the root index.php file until the line $app->createExtensionNamespaceMap(); and then add the following code:

$objUser= JFactory::getUser();
if (empty($objUser->id))
	die('0');
die('1');

0 in the above code indicates that the user is not logged in, and 1 (you guessed it), indicates that the user is logged in. So, we created a file called check-user-login.php containing the index.php code until the line $app->createExtensionNamespaceMap(); followed by the above code, and we placed it at the same level of the index.php file, and then we checked if it worked. Well, to our rejoice, it did! Well – when using the direct link at least!

The issue was when we tried to check whether the user is logged in from an external website (e.g. a different website) using the below ajax function:

function checkUserLogin(){
	$.ajax({
		url: "https://[URL]/check-user-login.php",
		cache: false,
		type: "GET",
		success: function(data) {
			if (data == '1'){
				console.log('User is logged in');
			}
			else{
				console.log('User is not logged in');
			}
		}
	});
}

The above code did not work – instead the JavaScript console complained about CORS (which stands for Cross-Origin Resource Sharing). “Easy”, we thought. Let’s fix that CORS! Let’s just add the following code to the .htaccess of the website (where we have the check-user-login.php script):

<IfModule mod_headers.c>
	SetEnvIf Origin "http(s)?://(www\.)?(clientexternaldomain.com)$" AccessControlAllowOrigin=$0
	Header add Access-Control-Allow-Origin %{AccessControlAllowOrigin}e env=AccessControlAllowOrigin
</IfModule>

The above fixed the main CORS problem, but we had another problem – the script was always returning 0. It turned out that we had 1) to tell the JavaScript code on the external server that we need to send the cookies, and 2) we need to tell the source website (where we have the check-user-login.php script) to accept cross site cookies over a secure connection.

The first part was resolved by modifying the checkUserLogin function to the following:

function checkUserLogin(){
	$.ajax({
		url: "https://[URL]/check-user-login.php",
		cache: false,
		type: "GET",
		xhrFields: { withCredentials: true },
		success: function(data) {
			if (data == '1'){
				console.log('User is logged in');
			}
			else{
				console.log('User is not logged in');
			}
		}
	});
}

Notice the addition of the xhrFields: { withCredentials: true }, line.

The second part was resolved by adding the following code to the very beginning of the check-user-login.php file:

ini_set('session.cookie_samesite', 'None');
ini_set('session.cookie_secure', 1);
header('Access-Control-Allow-Credentials: true');

We then tried to see if the external website was able to check if the Joomla user is logged in to the main website, and, it worked! Well, sometimes it worked…

We spent a lot of time and tried many things in our quest to resolve (or at least understand) the problem – but none worked – we ended up with the same stability issues – sometimes the login worked, other times it didn’t (to be fair [or unfair?] most of the times it didn’t). Eventually (after so much time), we had this great idea! Here’s what we did:

• We created an article on the source (main) Joomla website, and we ensured that only registered users can access it. The name of the article was Check User Login.
• The article only had the following content “Success”.
• We created a menu item, pointing to that article, and we named it Check User Login, so the direct URL to that article was https://[URL]/check-user-login.html.
• We assigned the menu item to a near blank template called user-login where the main index.php of that template had the following code:

  defined('_JEXEC') or die;
<jdoc:include type="component" />

  and the article layout override file (e.g. the default.php file under user-login/html/com_content/article folder) had the following code:

  defined('_JEXEC') or die;
  $objUser= JFactory::getUser();
  if (empty($objUser->id))
  	die('Fail');
  die($this->item->introtext);

• We changed the JS function to use check-user-login.html instead of check-user-login.php.
• We added (for CORS reasons) the following code to the defines.php file at the site root (e.g. at the same level of the root index.php file):
  

  $strCurrentURL = 'https://'.$_SERVER['HTTP_HOST'].$_SERVER['REQUEST_URI'];
  if (stripos($strCurrentURL , 'https://[URL]/check-user-login.html') === 0){
  	ini_set('session.cookie_samesite', 'None');
  	ini_set('session.cookie_secure', 1);
  	header('Access-Control-Allow-Credentials: true');
  }

  Note that we initially forgot about doing the above step and immediately jumped to the next step, but we were “greeted” with the following error in the JS console:

  Access to XMLHttpRequest at [URL] from origin [clientexternaldomain] has been blocked by CORS policy: The value of the ‘Access-Control-Allow-Credentials’ header in the response is ” which must be ‘true’ when the request’s credentials mode is ‘include’. The credentials mode of requests initiated by the XMLHttpRequest is controlled by the withCredentials attribute.

• We checked whether the feature is now stable, and it was! Hooray! We solved the problem!

As mentioned at the very beginning of the article – we thought this was an easy task (and, in our opinion, it should have been an easy task), but quirks in JavaScript, Joomla, browsers, and the way cookies work in general made this a really challenging one, but it’s no longer challenging for you, our dear reader (now that you have the secret recipe). But, if you need help with the implementation, or if you think this whole coding scheme is way outside your realm, then you can always contact us. Our fees are reasonable and we are possibly the friendliest^* programmers on planet earth.

^*Internal data.

Almost a decade ago, we explained how Joomla’s Redirect Manager extension works. Fast forward to now (April 2023), nothing really has changed; the system still works in the same weird and inefficient way in the latest version of Joomla (4.2.9 at the time of writing this article). But not only it is weird and inefficient, it can also be harmful. How?

Well, let us explain the scenario that we just experienced with one of our new clients…

A few days ago, and on a very rainy afternoon, the phone rang… It was a new client who told us that their news website (they had a bit over 70K articles) was intermittently slow. A quick investigation revealed that many legitimate bots constantly tried to index non-existent pages. This led to the following avalanche of events:

• The “System -> Redirect” plugin was triggered and tried to find a matching page through a bizarre heuristic.
• If it did, then an entry linking the old (non-existent) URL to the new one was added to the database and the system redirected back to that new page.
• If it didn’t, then the system just wasted some processing power.

As you can see, whether a match (assuming it was really a match) was found or not, there was an unnecessary overhead that increased the server load. The irony is that the client is not even interested in those redirects – the client is happy with a 404 page for those non-existent links.

So, what did we do to fix the problem?

Simple! We just disabled the “System -> Redirect” plugin and all was fine – and this is our recommendation to you, our dear reader, if you don’t care about “all of the features” of the Redirect extension, then just disable it, or, at the very least, disable the “System -> Redirect” plugin.

We hope you found this light post useful! If you’re having any issues with the performance of your Joomla website, then just contact us. We’ll be happy to help and we won’t cost you much!

Note: Needless to say, it is important that you backup your website before migrating to Joomla 4.

Warning: Before migrating to Joomla 4 from Joomla 3, it is critical that K2 (as an extension) is completely disabled, otherwise, you will end up with a broken website and you will need to restart the migration from a backup.

Free Advice: K2 is just not compatible with Joomla 4 (at the time of writing, which is November 2022), so do not waste your time.

We love (or loved?) K2 – it is (was?) one of the most wonderful extensions out there, but, for some weird reason, the K2 developers haven’t created a Joomla 4 version yet (they are still promising they will eventually do it), and so we decided to do this ourselves for one of our clients. So, how did it go for us?

Well, we started working on a website which had about 50K articles – the content system was powered by K2, although none of the exclusive K2 features was used (so there was really no need for using K2). We did some research and discovered that no K2 compatible version was released for Joomla 4 yet. “Easy”, we thought – we’ll make it compatible ourselves – “Shouldn’t take long, right?”. Well, we couldn’t be more wrong.

We started working on a copy of the website, we disabled K2, performed the website migration to Joomla 4, and then re-enabled K2. We visited the frontend and we saw the first error:

Call to undefined method Joomla\CMS\Application\AdministratorApplication::isSite()

“Easy”, we said, we opened the k2.php file located under plugins/system/k2 folder and replaced:

if ($app->isSite()) {

with:

if ($app->isClient('site')) {

We refreshed the homepage and we saw another error, which was this one:

Class “JRequest” not found

“That’s a no-brainer”, we thought, and we replaced in the k2.php file which we had already opened before all occurrences of:

JRequest::getCmd, JRequest::getInt, and JRequest::get

with:

Joomla\CMS\Factory::getApplication()->getInput()->get

“That is fun”, we kept saying, but, as time went by and we saw our hairline slowly receding, it slowly became less and less fun and more and more like a torturous whack-a-mole game. Eventually, after a couple of weeks of fixing these errors (some of them were really insane to fix, as some Joomla 3 functions didn’t have a Joomla 4 equivalent, which we’re guessing is one major reason why the K2 developers were delaying the migration to Joomla 4) we had a working frontend, and we noticed, all of a sudden (it wasn’t really “all of a sudden”, but we’re trying to spice this post a bit), that our client uses the K2 backend to add content, which was much harder to work on because of the major template engine difference between Joomla 3 and Joomla 4. In other words, the backend work also had to consist of layout conversion work, which is outside of our realm for sanity reasons. So, while we’re known to have the tenacity of a hungry wild bear holding to the last salmon on a cold and slow fishing day, we gave up.

So, what did we do next?

We changed strategy altogether; instead of trying to adapt K2 to Joomla 4, we migrated the K2 content to Joomla 4’s core, which was not-a-straightforward process (we will talk about this in the near future), but infinitely less complex than getting K2 to work flawlessly on Joomla 4.

Is it worth waiting for the K2 team to develop a Joomla 4 version?

In all fairness, we wouldn’t hold our breath for it as there are several signs that tell us that it just won’t happen. We hope we’re wrong, and we hope it will happen, but for the moment, our only recommendation is to migrate to Joomla’s content system. This is sad, because, as we mentioned in the beginning of this article, and in all of our K2 articles, we love K2, and we really hate to see it being deserted en masse. Note that we may eventually do the migration ourselves, you never know! But for now, if you want to migrate your K2 content to Joomla 4, then you can always contact us. Our fees are reasonable, our work is quick and professional, and we’re in love with Joomla!

Note: This post partly assumes that your server is running WHM/cPanel. Solutions may differ if your server is running a different platform.

About a week ago, a client approached us and told us that they were having a problem on their corporate website, where they are seeing the following error:

0 Missing field in database: Joomla\CMS\Table\Extension extension_id

Fortunately for that client, we did see this exact problem before, and we know what causes it: it is a combination of PHP update (to PHP 8.1.x), as well as an old Joomla website (<= Joomla 3.10.3). We quickly confirmed our theory after checking the website and the hosting platform.

But how did the problem happen?

Some hosts are over anxious to update their clients to the latest version of anything – unfortunately, many of these hosts do not fully test the applications that their clients have before proceeding with the update (some do not test at all). In the case of our client, the host performed the update on the dedicated server without even telling the client.

So, how did we solve the problem?

Well, there are there three ways to solve the problem:

1. Manually updating the Joomla website (through FTP/sFTP/shell) to the latest Joomla 3 version: This is the most dangerous method and must be used as a very last resort, simply because a manual update typically causes more problems that it usually solves; one can end up with a much bigger mess if they are not very careful. It is also very important to make a backup of the filesystem and the database before proceeding with such (again, very dangerous) move.

2. Reverting to a previous version of PHP: This can be done by adding the following lines to the .htaccess file:

   <IfModule mime_module>
     AddHandler application/x-httpd-ea-php74 .php .php7 .phtml
   </IfModule>

   The above code forces the website to use PHP 7.4 (we are assuming that the previous version that the website was using was PHP 7.4 and that this PHP version is still installed).

   You can also revert back to a previous version of PHP at the server level, this can be done by logging in to WHM, and then going to the MultiPHP Manager page, and then selecting the immediately previous PHP version from the drop down next to PHP version. Note that after doing this, you must restart Apache, by going to the Restart Services page, and then clicking on HTTP Server (Apache), and finally clicking on “Yes” to restart the Apache server (it is important that you do this only during off-peak hours).

3. Copying the “Table.php” file from the most recent version of Joomla 3: This is by far the easiest and most straightforward solution. All you need to do is to download the latest version of Joomla 3 (which is Joomla 3.10.11 at the time of writing this article), extract the folder, and then copy the file libraries/src/Table/Table.php to the corresponding location on your Joomla website. We recommend that, after doing this, you update your Joomla website to the latest Joomla 3 version (this is not necessary, and it is super important that you backup your website [table + database] before proceeding with the update).

As mentioned above, the last solution is the best solution, and, unless you have a problem with it, then we recommend that you try the second solution, and, as a very last resort, try the first solution. If you’re a bit scared of implementing any of the solutions, or if you got stuck while implementing a solution, then you can always contact us. We will be able to fix your website quickly, affordably, and reliably.

We are currently migrating a Joomla 3 website to Joomla 4. Among the hundreds of errors that we saw during the migration process, there was this one:

Modulo by Zero

This error was interesting, because it was just weird and very short. It meant that we were dividing by zero somewhere in the code, but where? Before giving you the details of our investigation, let us explain the modulo concept for the non-developers/non-mathematicians who are reading this post.

Modulo means the integer remainder of a division of one number by the other. For example, 9 Modulo 2 (this is written 9 % 2 in PHP code) is 1, 73 Modulo 7 is 3, and 170 Modulo 19 is 18 (19 is a prime number). Of course, some people may think this function is useless, but it is very handy for all developers, and for many of those who work with numbers.

So, what caused the modulo problem?

When we see any problem, we first switch the template to a basic one, to see if the problem is caused by the template (we later disable other extensions until the problem is no more). So, in this scenario, we switched to the Cassiopeia default Joomla 4 template (by the way, in case you are wondering just like we were wondering, Cassiopeia is the name of a constellation) and the problem was suddenly no more. Investigating the problem deeper, we found that it was caused by the following line in the default file of the featured layout:

$rowcount= (((int)$key-1) % (int) $this->columns) +1;

In the previous version of Joomla (Joomla 3), $this->columns used to return a default value of 1 (when left empty in the backend), in Joomla 4, $this->columns returns just empty, which translates to zero when cast to integer. This very change caused the problem.

But how was the problem fixed?

Fixing the problem simply consisted of adding the following 2 lines of code just before the problematic line:

if (empty($this->columns))
	$this->columns = 1;

Of course, there was a big bunch of other problems after fixing this one, but, it was a step in the right direction.

Now if you have the same problem (or any other problem) while trying to migrate your website to Joomla 4, then you can just contact us. Our fees are always competitive, we are very knowledgeable in Joomla, and we are extremely humble (while the last 2 sentences may seem like an oxymoron, they really are not)!

It’s Saturday, and we received an interesting email from one of our clients. The client said that someone was manually spamming their HubSpot forms with odd data. HubSpot is good at blocking spammers, but manual spam is really hard to catch. The client asked us to block this person from accessing their HubSpot pages.

There were several obstacles for accomplishing what the client wanted: 1) we didn’t know the IP of the person because HubSpot doesn’t tell us, 2) there is no method to block an IP in HubSpot (you can only block an IP from being tracked), 3) there is no method to block an individual email in HubSpot (you can only block domains), and 4) even if there was a method to block individual emails in HubSpot, it wouldn’t work, because the spammer was using a different email each time they submitted a form.

So what did we do?

There was a service on one of the client’s servers that was called when someone visited any of their HubSpot pages, so, going through the logs helped us determine the IP of the person. That was a first step!

The next step was to block the IP. What we did was that we added the following code to the JavaScript file that is loaded on all the client’s HubSpot pages:

$( document ).ready(function() {
	if ('{{request.remote_ip}}' == 'Offending IP Address')
		document.body.parentElement.innerHTML = 'Blocked';
});

And that’s it! That IP was seeing a blocked message when they were trying to fill out the form.

There are a couple of notes about this solution: 1) This is a JavaScript solution, so it might not be ideal, and 2) using the {{request.remote_ip}} HubL variable results in HubSpot no longer using page caching, which might impact the speed of the page load.

We hope that you found this post useful! If you need help with the implementation, or if you need development work for HubSpot, then please contact us. Our rates are still affordable, our work is still top notch, and we always care for our clients!

Here’s a common scenario: you create a HubSpot landing page with a form. Once people fill in that form, they are redirected to a gated document (possibly an e-book or a whitepaper). The system is simple and works flawlessly, you are happy, and your client is happy! A few weeks later, your clients emails you and tells you that their PDF has been indexed in Google, and now it’s accessible for anyone, without even going through the HubSpot gate.

You do a research, and then you discover that you had to set the “File URL Visibility” to “Public – No-index” by doing the following:

• Going to the “Marketing” -> “Files and Templates” -> “Files” in HubSpot
• Clicking on the name of the file affected, e.g. client-file.pdf
• Setting the “File URL Visibility” to “Public – No-index” in the “File details” side window

So you do that, and you think (and rightly so) that it’s probably too late for this particular file since it’s already indexed by Google, and you apologize to the client, and promise that “this won’t happen in the future”.

Another couple of weeks later, you are asked (by the same client) to host another gated asset (e.g. yet another whitepaper), and so you gladly do it, and you ensure that its visibility is set to “Public – No-index”. You assure the client that it won’t get indexed by Google because you “ensured it won’t”. Yet another couple of weeks later, you get an email from the client complaining that the document is indexed by Google and that people are able to access it directly. How could that happen when you ensured that its visibility is set “Public – No-index”?

You start thinking that you are missing something, maybe the problem is that Google is allowed to index those landing pages? You talk yourself into believing this twisted theory, and so you disallow Google from indexing the landing pages altogether (after all, 99% of the traffic to those landing pages comes from direct email marketing) by going to the “Settings” page in HubSpot, and then clicking on “Website” -> “Pages” on the left navigation bar, and then clicking on the “SEO & Crawlers” tab, and finally adding the following to the robots.txt file:

User-agent: *
Disallow: /

You feel considerably less confident this time, but you still inform the client that “you missed something” last time, and that the problem will not happen again. The client is quickly convinced (because of all of these years of building that precious trust with you), and asks you to host another gated content, which you do, and you ensure that the file’s visibility is set to “Public – No-index” and that Google isn’t even allowed to index any landing page.

5 days later, your left (or right, or both) eye immediately start twitching when you hear the phone ringing and you see the client’s number; it’s not time for another gated content so there must be something wrong. You answer the phone and your client suddenly has a completely different tone. You apologize for nearly 15 minutes and you say “Do not worry! I’m on it”, just before hanging up.

You start biting your nails/lips and you start apologizing to every single person you’ve hurt intentionally or unintentionally since 1^st grade. You then go for a walk and are suddenly gentle and kind to everyone you see – maybe that would lift the curse?

But it’s not a curse, in fact, the problem is from both HubSpot and Google. The latter chooses to disrespect, more often than not, the robots.txt directives, and the former (HubSpot) is gullible enough to believe that Google will respect those directives.

So what’s the real solution here?

Well, the solution is to disallow Google (and other search engines) from indexing the assets, even if they wanted to, and that can only be done if the asset is hosted on a server you fully control, because it must be done at the .htaccess level.

In other words, you will need to go into one of your servers, create a directory, upload your assets (ultimately link to those assets from your HubSpot landing pages), and then create an .htaccess file with the following rules:

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ^(.*)googlebot(.*)$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)bingbot(.*)$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)alexa(.*)$ [NC]
RewriteRule .* - [F,L]

This should prevent Google from even attempting to index those PDFs. Adding a robots.txt file to that directory with the following code…

User-agent: *
Disallow: /

…should also help (it will prevent other bots from indexing the assets).

So there you go, the right way of hosting gated assets! If you need help with deploying the above, or for any other HubSpot help, then please to contact us. Our fees are super affordable, we are ready to help, and we have solid and proven HubSpot experience.

Say you have a HubSpot form that you want people to fill in. The HubSpot form that you have is very long with many input fields that you need for your business. You embed it on one of your pages and then you send out an email with a link to that page. People fill in the form, and then they see blank. They become confused and possibly disinterested because of the perceived unreliability. A small percentage will also give your business a call saying “I tried to submit your form but I got a blank page”.

In reality, the page wasn’t a blank page, but the “Thank You” message was at the very top of the form, which caused the confusion, since the user who filled in the form was at its very bottom. This is a very annoying problem which seems to be ignored by HubSpot (possibly because they can’t find a generic method to solve it).

But there’s always a workaround. The workaround consists of leveraging HubSpot’s onFormSubmitted event to position the “Thank You” message in the center of the page, this can be done by getting the size of the div containing the “Thank You” message (in jQuery), and then getting the size of the screen, subtracting the former from the latter, dividing the number by 2, scrolling to the position of the “Thank You” message, adding the divided number to the top of the “Thank You” message, and finally setting the top of the “Thank You” message to the number that was just returned.

We agree, this is a lot of jumping through hoops for a feature that should have existed by default, but it is what it is.

We hope that you found this post informative and useful. If you need help with HubSpot development, then please contact us. Our fees are affordable, our work is clean, and we finish things!

Here’s the problem: A client of ours has many gated whitepapers on HubSpot, and they wanted to group them into pages where each page contains the related whitepapers, by topic. So, in essence, the HubSpot landing page consists of a simple lead generation form which, at the end, contains a checkbox list consisting of whitepaper names that belong to similar topics. What the client wanted was to 1) allow the users to automatically download all the selected whitepapers when they click on the “Submit” button on the HubSpot form, and 2) send them an email containing the list of the whitepapers that they have just downloaded and where they can download them again (e.g. a list of whitepaper title/link).

At first glance, what our client wanted seemed impossible, but, after analyzing their request, we broke it down to 2 possible tasks (still complex, but not impossible):

1- For the automatic download on submit, we leveraged the onFormSubmit HubSpot event (which we have used before for creating a multi-step form in HubSpot), where we sent, through AJAX, the request to a PHP script on one of the client’s servers, which processed the request by generating a zip file containing the PDFs that the client selected, and then pushed the zip file back to the browser as a download.

2- For the email, we created a workflow which triggered a webhook on one of the client’s servers, and the latter was responsible for sending the email to the lead.

Of course, the big question is how we made it generic? Well, at first, we didn’t as the client had a preset list of whitepapers to be included on each page. But then, when the client wanted to modify the lists, we knew that the best option would be to make it generic rather than fiddling with the code each we wanted to add/delete a whitepaper.

So, again, how did we make it generic?

Well, first we ensured that each option in the checkbox list had a value that is the name of the campaign of the associated HubSpot form. Then, in both of our processing scripts (the one that handles the automatic download and the one that handles the email sending) we grabbed the title of the page and the associated PDF file for each campaign by searching in the HubSpot page dump.

We hope that you found this post useful and fun (implementing this feature and writing about it were both fun for us). If you need help with the implementation, then please contact us. Our rates are right, our work is top notch, and we just adore HubSpot!

As you may already know, HubSpot only allows for single-step forms. They don’t have a built-in function to create multi-step forms. They do advertise a workaround, which is creating two landing pages, each containing a form with a set of fields, where the first page, when submitted, is redirected to the second page using the Redirect to Another Page option.

This solution is not very elegant, for the following reasons: 1) if you want to embed the form on one of your pages, you will need to use an iframe, which is always a horrible option and 2) no real connection is done between the forms (for example, it might be that some of the fields on the second form are connected to the first form).

At itoctopus, we developed a much more cleaner solution, mainly using jquery. Here’s what we did:

1. We embedded 2 HubSpot forms on the same page.

2. We hid the second form using the display:none CSS styling.

3. Using the onFormSubmit event of the first form, we got the value of the fields that were filled in the first form, and passed the relevant values to the second form.

4. Using the onFormSubmitted event of the first form, we hid the first form and we showed the second form using the display:block styling.

Of course, this took us some time to figure out and ensure that the solution is solid, but the end result was just loved by the client, as it was really smooth!

As you can see from this post, we really are experts in HubSpot development and can help you achieve the results that you want. If you have a complex task in HubSpot that you think it can’t be done, then we probably can do it, so go ahead, contact us. Note that our affordable fees apply!

A client of ours asked us to list some of their HubSpot pages as virtual events in Google. These pages were mainly webinar registration forms. To the untrained eye, the task seems to be very simple, because it consists of simply adding some structured data in the HTML as described by Google here. In reality, the task was a small project. You see, the issue is that we mainly needed to know the start date and the end date of the webinar, both were included in the text of the page, but cannot be easily parsed. The complexity of the system meant that we couldn’t rely on HubL (HubSpot’s scripting language) to do the job. So, how did we do it?

Well, we first created a system that will grab all the pages from HubSpot and parse them (this was done in a cron on a daily basis). So, for each page, we had the URL, the title, the description, the start date (of the webinar), and the end date. We then created a service that will get the URL of the page as input, and will generate the structured data based on the URL. Finally, on HubSpot’s end, we added to the global JavaScript file a simple code that will grab, through jquery, the structured data for the current URL, and that will inject it in the head in a json script JS tag. That was it and it worked!

Of course, it seems much easier now that we know how to do it, but this task was very complex to implement, it was also somehow scary as we weren’t sure if Google will be able to parse dynamically generated structured data after full page load. Luckily, it did!

We hope that you found this post informative. If you need help with a complex task related to HubSpot, then please contact us. We are here to help, we adore HubSpot, and our fees are still super affordable (get them while they last!).

While working on a Joomla website powered by K2, we noticed that one of the pages, containing a big list of company names, is only limited to display the first 100 companies. We checked the backend of the Joomla website, and the limit was set to display 1,000 items for that particular category. We increased the limit on the parent category, but that didn’t work: only the first 100 items were displayed. What could be the reason?

We traced the issue and this trace led us to the file view.html which is located under the components/com_k2/views/itemlist folder. The file had the interesting few lines:

// Protect from large limit requests
if ($limit > 100) {
	$limit = 100;
}

Clearly, the K2 developers thought that whoever sets the limit about 100 must be mistaken, and so they capped the limit to 100 (this subtle cap was later introduced – it wasn’t there before). Removing the above code solved the problem.

Note that we did contact the K2 team over twitter – and they said they will make this (hard-set limit) configurable in the next K2 update.

If you are experiencing the weird 100 item limit in K2, then try the above, this should solve the problem. If you would like help with the fix, or if you need help fixing any other Joomla issue, then please contact us. We are always here to help, our prices are super affordable, and our work is really clean!

We manage very large Joomla installations, and a constant question we get from editors working on these websites is: “I am trying to order articles on my website, but it is not working, how do we do that?”

And then we start to explain that first they need to choose the category when listing articles in the backend, and then they need to click on the arrow next to the first checkbox to sort descending, and finally they need to drag and drop articles to sort them. It feels awkward every time we explain the process, because we know that it is confusing and that there must be a better way of doing this order (keep in mind that there is an extra step for many others, and it is clearing the Joomla cache – our clients don’t need to go through this step because we have a plugin that will automatically clear the cache for them). In our opinion, the ordering should be done at the article level, and not using a drag and drop.

Each article must have 2 ordering fields: “Order on the homepage” and “Order on the category”. “Order on the homepage” will be a dropdown that will dictate the ordering of the article on the homepage, “Order on the category” will be a dropdown that will dictate the ordering of the article on the category page (the category that the article belongs to). This will get rid of the “drag and drop” confusion, which, in our opinion, is a major drawback in Joomla and one of the main reasons why people go elsewhere.

We hope you found this post useful and informative. If you are interested in becoming one of our clients, then please contact us. Our fees are right, our work is clean, and we are fully dedicated to our clients!

For the past decade, our (almost) exclusive focus was the Joomla product. However, in the past few years, we suffered from an increasing appetite to a specific marketing niche: integrations. We tried to stem this appetite by avoiding such jobs as much as we can, we tried to quench it by taking as many integrations as we possibly could, but both attempts were futile. We always felt the same way that Claudia felt when she (in)famously said in “Interview with the Vampire”: I want some more! and our Joomla conscience gave us the same look that was on Louis’ face when she said that (you can ignore this whole sentence if you haven’t watched the movie).

We know exactly why we love integrations and why we can’t get enough of them: they’re challenging and they are so much fun! Have you tried to integrate HubSpot with Tealium with Google Ads? We did, and it was so so exciting! Integrations are also about optimization, security, data integrity, etc… They really encapsulate every aspect of programming out there!

But why do we feel guilty about this?

It’s not that we feel guilty, it’s that we feel that this increasing passion is at the expense of another passion of ours, which is Joomla. But, we shouldn’t feel guilty, since most of the integrations that we are doing revolve around Joomla (some our integrations are directly with Joomla, like this one). Still, there is our Joomla conscience that it is telling us: “you don’t love me as before”, but we still do, and, as strange as it might seem, we are doing it for you Joomla. In fact, we believe that any growth in our business in any direction will solidify our Joomla interests, pushing us to come up with ways to make the Joomla product even more versatile and robust for enterprises.

So there you have it, we are now at peace with ourselves, we are still super passionate with Joomla, but we have a new passion, which is integrations! Satisfied, Joomla conscience?

While trying to update a Joomla website to the latest version we encountered the following weird error:

Could not open /home/[user]/public_html/administrator/modules/mod_latestactions/mod_latestactions.xml for writing

At first glance, we didn’t know what that error was because we didn’t know that the module mod_latestactions existed in Joomla. A quick research revealed that this was a new module and was used for displaying the latest admin actions in the backend (e.g. admin abc did action xyz), but why did the update have a problem creating that module?

A research revealed that this issue was caused by a permission setting: Apache just didn’t have permissions to write to that folder, and it was because of the chattr +i command (which makes files/folders immutable) that we explained in details here. Fixing the problem consisted of simply removing the immutable setting from all the Joomla folders before updating the website (this was done by issuing the chattr -i command).

As usual, if you are experiencing the above problem and if you were not able to fix it yourself by using our guide, then please contact us. Our fees are right, our work is clean, and we always aim for long term and strong relationships with our clients.

One of HubSpot’s powerful features is the ability to have a workflow send an email. Typically, the workflow contains the logic (if-then-else), and the email contains some dynamic data, such as the first name of the person, the title of the person, the company that the person works in, etc… However, the email cannot contain complex logic, such as “show this section if someone has X field set to ‘ABC'”, leading to having another version of the email and another workflow just to have this functionality.

There is a workaround, however, to do the above. The workaround consists of leveraging the “Smart Content” feature that HubSpot supports in all of its templates, including its email templates. So, in order to show a specific section in the email for people with a field (property) set to a specific value the following must be done:

• A dynamic list that contains the aforementioned property (set to the specific value) is created.
• The module (section) in the email that should be displayed only when the property above is set to a specific value is converted to “Smart Content”.
• The “Smart Content” module is set to display only when the client (the person who receives the email) is a member of the list created in the first step.
• That’s it!

As you can see, having some complex logic (such as displaying a module when a property is set to a specific value) in your HubSpot email is possible, and is relatively not that hard. Having said that, we do reckon that it is not straightforward, so if you need help with implementing the above on your website, then all you need to do is to contact us. Our rates are affordable, our work is professional, and we love working with new (and old) clients!

An emerging and an extremely annoying issue that (almost all) large websites are currently experiencing is bot traffic. Of course, bot traffic has always been an issue, but it was, to a certain extent, manageable and way less vexing. However, recently, bot traffic has become very aggressive, especially on high traffic websites. So why this is becoming a major issue all of a sudden?

A couple of months ago, one of our clients asked us to check the validity of their traffic for the previous day as reported by Google Analytics. In case you don’t know, Google Analytics is known for “trying” to discount bot traffic for their stats, but the numbers that the client was seeing were way out of their daily range. So we checked their data and we noticed that there was a huge traffic from cities with very low population, such as Boardman, OR. A quick examination revealed that traffic from most of these cities originated from an Amazon network, and so we filtered traffic from Amazon networks in Google Analytics. That made things much better, but later on, rogue IPs from other networks started hitting the website, and so the whack-a-mole game started. It was fun at first, but later it became tedious and annoying. So we decided to automate the process:

• We got the top IPs visiting the website using a Linux shell script.
• Any IP hitting the website with more than 10K visits was checked for its network and its location (the checking was done through API).
• If it was outside the US, then it was automatically banned (through the .htaccess file).
• If it was inside the US, then it was sent for human examination, where a human must decide on whether traffic from this IP was automated or coming from a legitimate organization (highly unlikely, but still possible).

Adopting the above process (which was mostly automated) lessened the load on our clients’ servers and reduced those fake impressions substantially.

Yes – we noticed that we didn’t use the word Joomla anywhere in the article, but really this an across the board problem and affecting all kinds of CMS’s.

We hope that you found our little post useful. If you are running into the same issue, then please contact us. Our rates are fair, our work is scientific, and our Sherlock Holmes investigative techniques are improving by the day!

A couple of weeks ago, we received an interesting email from the marketing team of a high traffic Joomla website: they told us that their traffic in Google Analytics was increasing substantially, and while they were excited about the issue, they were thinking that it was too good to be true, and so they wanted us to investigate the issue.

As usual, we gladly obliged. At first glance, we noticed nothing – everything was normal. On second glance, however, we noticed that they were receiving a huge amount of traffic from two not-so-huge cities: Boardman and Ashburn. Boardman, for example, is a city in Orgeon, and has a population of 3,220 – but they were receiving about 80K visitors each day from that city. Either the people in Boardman, OR are obsessed with this website and are accessing it from about 25 different devices each and every day, or not. Our first instinct was to go with the latter theory (“not”), and so we investigated the issue a bit more, and it turned out that the Amazon network was sending out automatic traffic from this city. This was also the same for Ashburn. But why would Amazon do that?

Additional investigation revealed that some anti-virus companies were using the Amazon network to scan each and every element that the user was seeing in order to ensure that everything was clean. So, for each person using some types of anti-virus and visiting our client’s website, there is another visit from a different IP using a different user agent from the Amazon network (sometimes, there are multiple bot visits). Clearly, that bot traffic was fake and needed to be excluded from Google Analytics. Here’s how we did it:

– We logged in to Google Analytics.
– We clicked on the settings icon at the bottom left.
– We clicked on All Filters
– We clicked on Add Filter
– We typed in “Amazon” in the Filter Name
– We chose “Predefined” for Filter Type
– We chose “Exclude”, “traffic from the ISP domain”, “that contain” from the 3 dropdowns next to each other.
– We typed in “amazon.com” in the ISP Domain field.
– We added the view under Apply Filter to Views and then we clicked on Save.

That’s it! After doing that, the traffic from Boardman and Ashburn was no longer included in Google Analytics (the effect was not retroactive though, so data from before the filter still included traffic from those cities).

We hope that you found our post useful. If you need help with the implementation, or if you need help investigating a similar issue, then all you need to is to contact us. Our fees are super affordable, our work is super clean, and our experience is proven!

At itoctopus, we have optimized many, many Joomla sites and we have written many articles on optimizing Joomla sites. But, before optimizing any Joomla website, we do check/ask about the server powering the website. If we think the server is too weak for our beloved CMS, then we recommend that the client upgrade to a better server before even hiring us. In fact, recommend is such a weak word – we insist!

Let us give you a quick example: yesterday, we got a call from a very nice client telling us that his server comes to a crawl when they send out newsletters. He told us that his peak traffic is 5K to 7K visitors, and that peak traffic happens once every one or two month (when they send out the newsletters). On normal days they get only a few hundred visitors. He told us that he thinks that there is something from within Joomla causing this issue, but he also told us that the CMS was powered by a server with 2GB of RAM. Yes, you read that right, that’s two gigs of RAM (by the way, is it gig as in garage or jig as in giraffe?), which was really odd. We immediately stated that we can’t work on such server, and that they must upgrade to a server with at least 16GB of RAM in order for us to work on the server.

So, why did we do that?

First, we know what Joomla is and we know that 2GB of RAM is not nearly enough, even for a small website. Keep in mind that the 2GB of RAM are not solely dedicated to the web server and the database server, but there are many other resources sharing it. In our opinion, a small Joomla website must run on a 16GB server, and a large Joomla site must run on a 256GB server.

Second, in most of these cases, upgrading to a better server typically solves the problem. This will of course save the client money on the short run and on the long run, despite the additional monthly overhead for the hosting company.

Third, debugging and working on slow servers and/or servers with low memory (typically a slow server is also a server with low memory) is not very efficient, as we will also be fighting the restricted environment (and not just some rogue plugin on the Joomla website).

So, before asking a company to optimize your Joomla website, make sure that your environment provides ample power for your Joomla website to run properly. If it doesn’t, then you should upgrade to a more powerful server before asking for a costly optimization. If you need help with the process, then please contact us. We are efficient, we are affordable, and we are excellent developers!

After updating PHP through EasyApache 4 to PHP 7.3 and then changing the PHP version in the MultiPHP Manager to 7.3, we encountered the following error on a Joomla website:

0 – Using $this when not in object context

A fast research revealed that this problem was caused by some legacy code in the template, and switching to a different template confirmed this. Debugging the page revealed the following:

Using $this when not in object context /home/[cpanel-user]/public_html/libraries/src/Application/CMSApplication.php:370
Call stack
# 	Function 	Location
1 	() 	JROOT/libraries/src/Application/CMSApplication.php:370
2 	Joomla\CMS\Application\CMSApplication::getMenu() 	JROOT/libraries/src/Application/SiteApplication.php:275
3 	Joomla\CMS\Application\SiteApplication::getMenu() 	JROOT/templates/[cpanel-user]/html/com_content/article/default.php:22
4 	include() 	JROOT/libraries/src/MVC/View/HtmlView.php:697
5 	Joomla\CMS\MVC\View\HtmlView->loadTemplate() 	JROOT/libraries/src/MVC/View/HtmlView.php:230
6 	Joomla\CMS\MVC\View\HtmlView->display() 	JROOT/components/com_content/views/article/view.html.php:210
7 	ContentViewArticle->display() 	JROOT/libraries/src/MVC/Controller/BaseController.php:672
8 	Joomla\CMS\MVC\Controller\BaseController->display() 	JROOT/components/com_content/controller.php:118
9 	ContentController->display() 	JROOT/libraries/src/MVC/Controller/BaseController.php:710
10 	Joomla\CMS\MVC\Controller\BaseController->execute() 	JROOT/components/com_content/content.php:43
11 	require_once() 	JROOT/libraries/src/Component/ComponentHelper.php:402
12 	Joomla\CMS\Component\ComponentHelper::executeComponent() 	JROOT/libraries/src/Component/ComponentHelper.php:377
13 	Joomla\CMS\Component\ComponentHelper::renderComponent() 	JROOT/libraries/src/Application/SiteApplication.php:194
14 	Joomla\CMS\Application\SiteApplication->dispatch() 	JROOT/libraries/src/Application/SiteApplication.php:233
15 	Joomla\CMS\Application\SiteApplication->doExecute() 	JROOT/libraries/src/Application/CMSApplication.php:196
16 	Joomla\CMS\Application\CMSApplication->execute() 	JROOT/index.php:49

A quick glimpse at the code revealed that the error wasn’t in the main index.php file of the template, so we renamed the html folder (which is located under templates/[joomla-template] folder) to html_old and the problem was resolved, which meant that the problem was in one of the folders under the html folder (e.g. the error was in one of the layout overrides). We renamed them one by one to folder_name_old, until we determined the culprit folder, which was com_content.

There was only one folder called article under the com_content folder and it only had one file, which was default.php. We opened that file and we changed this line:

$menu = &JSite::getMenu();

to:

$menu = JFactory::getApplication()->getMenu();

and that resolved the problem!

We hope that you found this very short post helpful. If you are having the same problem on on your Joomla website, then it is likely some old code in your Joomla template. If you can’t find/fix the old code in your Joomla template, then just contact us. We’ll address the problem for you swiftly, professionally, and affordably!

We are currently moving a large client of ours to a very powerful server, and we thought it would be a good idea to share the hardware specifications of said server with our lovely readers (all 1 billion of them!). Without further ado, here they are:

Intel Dual Xeon Gold 6130 (32 cores): This dual processor will provide ample power for about 300K visits/day.

RAID 10 SSD Storage – 4 x 480 GB: This will provide the Joomla website with abundant storage for both reading and writing.

NVMe SSD Storage – 1.6 TB Micron 9200 MAX: This NVMe drive is assigned to the /var/lib/mysql partition in order to make reads/writes to the MySQL/MariaDB database much faster.

SSD Drive – 960 GB: This SSD drive is assigned to the backup partition and is used for local backups.

RAM – 256 GB: For this particular client, we are switching from 64 GB to 256 GB RAM. This will allow us to serve more simultaneous clients, and assign more memory for each PHP instance if the needs arise. This abundant RAM will also be very useful for enhancing the speed of MySQL operations.

Dual PSU: Such powerhouse server will need a lot of power to keep ticking.

We hope that you found this short post useful and we hope that you consider the above specs for the next server powering your Joomla website. If you need help with configuring the server for optimal performance, then do not forget that we are the Joomla optimization experts. Just contact us and we’ll make sure that your Joomla website benefits from your new server to the max, and we’ll do that for a very affordable fee!

One of our clients was having serious load issues on their Joomla website during peak hours. At first, we thought it was MySQL – but further investigation revealed another, more subtle culprit: it was clamd, yes, the seemingly innocent virus scanner, was the root cause of the problem. To make a long story short (is it too late?), what was happening was that every time the cache was cleared (which was a frequent operation when the team adding content to the site was active), clamd was checking all the recreated cached files over and over again. This was causing clamd to hijack the processing power for itself, thus causing other, even more critical processes such as Apache and MySQL to starve.

So, how did we solve the problem?

Luckily, clamd allowed, in its configuration file, to skip directories from being scanned. So, we opened the file clamd.conf which is located under the /usr/local/cpanel/3rdparty/etc/ folder (this is a WHM environment) and we added the following line to it:

We then saved the file and restarted the clamd service from WHM. Once we did that, the load dropped substantially and the website was healthy again! Hooray!

If you are experiencing load issues on the server powering your Joomla website, then check your top processes (using the top -d 1 shell command). If you notice that clamd is always at a 100%, then try the above solution and it should resolve the problem. If it doesn’t work for you, or if you still need help, then please contact us. We are always read to help, our work is always clean, and our prices are always right!

A substantial portion of our work for our managed clients consists of monitoring their servers, and a large part of that monitoring consists of checking the logs, and seeing if there is anything unusual about them.

We check all the pertinent logs, but there is one log file that is really close to our hearts, and it is the lfd.log file which is located under the /var/log folder. The lfd.log file contains many actions such as:

• ModSecurity/CSF blocks/unblocks.
• Memory issues with scripts (e.g. scripts that are trying to consume more memory than they are allowed).
• Timeout issues with scripts (e.g. scripts that are not getting executed within the server timeout limit).

All of the above are generally associated with malicious behavior on the server, but, in many cases on Joomla sites, the behavior that might be causing any of the above is benign, and either indicates some poor programming practices or the need for a more powerful machine.

Some benign issues reported in the lfd.log file can be fixed simply by modifying the server settings (e.g. increasing the memory limit, disabling rules in ModSecurity, etc…), but some other (also benign) issues really require a serious optimization of one or several Joomla extensions, or the absolute need for a server upgrade.

Of course, the lfd.log file is also a great place to check for attacks on the website, and that’s the beauty of it: you have one file that you can check for monitoring both benign/malicious issues on your website, and that’s why we love it so much and why we encourage you to check it!

We hope that you found this short post helpful and enlightening. If you are having issues (any issues) on your Joomla website then please contact us. Our rates are always right, our work is always clean, and we are always excited for challenging Joomla tasks!

We had a weird error yesterday morning. A client emailed us and told us that his Joomla website stopped working all of a sudden, and it was displaying a blank page. We checked his Joomla website and we checked the logs, and it didn’t take us long to discover that the problem was with the MySQL database server powering the client’s Joomla website. So, we ssh’d to the server and then we tried to restart MySQL using the following command:

/etc/init.d/mysql restart

But it didn’t work. What was odd is that it did stop the MySQL server (which wasn’t really technically running), but it threw the following error when trying to start it again:

The server quit without updating PID file

We wanted to fix this problem quickly (at the request of the client, as the website was down), and restarting MySQL from within WHM seemed to be the quickest solution. So, we tried to login to WHM, but, to our surprise, WHM was down. We felt we were on to something, we just didn’t know what it was!

We thought, OK, this might be because MySQL is down. So, we researched the MySQL problem a bit, and we discovered that we may solve the problem if we delete the ib_logfiles from the /var/lib/mysql folder and then restart MySQL, and so we did that (we moved the files to another location; we didn’t delete them). That didn’t solve the problem, but, this time MySQL threw a different error: it complained about disk space not being enough.

We quickly checked the disk space using the df -m shell command, and, you’ve guessed it, the disk space was full in the main partition. We deleted some temporary files, backup files, etc…, which quickly reduced the main disk usage in the main partition to about 68%. We attempted to restart MySQL and this time it worked! Hooray!

We hope you found this post useful. If you are having the same problem, where MySQL unexpectedly stops working, then it might be that your disk is full. If that’s not the case, or if you need help with implementing the fix, then please contact us. Our fees are affordable, our work is professional, and we are always dedicated to serving our clients!

During the past 13 years, we had our fair cases where updates to Joomla articles were not reflecting immediately. We have finally decided to make a quick list containing the 4 reasons of why your changes to a Joomla article are not reflecting or are reflecting but not immediately (it only took us 13 years to take this decision). So, without further ado, here they are:

1. You are working on the wrong website.

   Believe it or not, this still happens to us! This problem typically happens when either your hosts file was changed to reflect a staging environment (but your forgot to change it back), or you are working on the staging environment instead of production.

2. The System – Page Cache plugin is enabled.

   Unfortunately, Joomla’s main caching plugin, the System – Page Cache plugin, still doesn’t take into consideration the fact that people change their articles after they are published. As such, when this plugin is enabled, changes to your Joomla articles are not reflected unless you clear the cache (or the cache is expired and it has to refresh). We have developed a caching plugin that addresses this issue and we may later release it for free on our website.

3. A server caching module is caching your pages.

   One of the most annoying features that a system administrator can deploy and enable on a production server is a caching module, such as OPCache. The annoyance is exponentiated when the system administrator doesn’t inform the people using the server of this feature (which what usually happens in almost 100% of the cases), because the changes they make to their Joomla articles are not reflected immediately, and they have no idea why. Typically, a quick solution to this problem would be to disable the server caching module in the .user.ini file, like we did when we disabled the OPCache caching module for a client who was experiencing the same problem.

4. You are working on the right website, but some DNS changes are taking effect.

   This happens, for example, when you are switching the website from one server to another, and you are working on the new server, but you are still seeing the website on the old server. We did experience this very weird issue once before, and we were only able to solve the mystery once the witness the client gave us more background.

We hope that you found this post useful and informative. If you are having a problem with your article changes not being reflected on your Joomla website, then most likely the cause of the problem is listed above. If you still need help, then you can always contact us. Our fees are always right, our work is professional, and we really care about our clients!

A non-regular client of ours, after updating their Joomla website to 3.9.1 a couple of weeks ago, started seeing the following classical error on their homepage:

Internal Server Error

What was interesting was 2 things: 1) the error didn’t happen immediately after the update, but rather a few hours later, and 2) the error disappeared after clearing the Joomla cache (the error was only on the frontend, and not the backend).

Naturally, the first thing that we did was investigating the server logs, which we did, and this is what we saw in the error_log file which is located in the the /usr/local/apache/logs folder (the client was using a WHM/CentOS environment):

[Fri Dec 14 13:18:13.607733 2018] [core:error] [pid 11040:tid 47479697520384] [client ip_number:59084] End of script output before headers: index.php

As you can see from the above, the problem was in the index.php file, but was it really? We looked at the index.php file, and it was exactly as the one in the Joomla distribution, so the problem wasn’t there, which was logical, but not very helpful. What was even more unhelpful was the fact that the problem was erratic, and it was oddly taking place every morning around 11:30 AM.

We didn’t have much to go on, but the fact that the problem was resolved after clearing the cache gave us a lead. So, we compared the generated cache file of the homepage (located under the system/page folder) immediately after deleting the cache, and the generated cache file of the homepage immediately after seeing the problem. Everything was more or less the same, except for the last line: the below entry was repeated hundreds of times in the second cache file (the one that is not working):

CP="NOI ADM DEV PSAi COM NAV OUR OTRo STP IND DEM"

The above line existed in the first cache file, but just once, and, in case you’re wondering, it is generated by the P3P (Platform for Privacy Preferences) system plugin, which injects the (now mostly obsolete) P3P policy header tags into the HTML response.

So, we disabled the P3P system plugin, and tried our tests again, and this time, we saw 2 other HTML response tags that were repeated: Pragma: no-cache and Cache-Control: no-cache. Clearly, there was something else going on, and the P3P plugin was not the cause of the problem.

We spent hours debugging the problem while wondering why this was suddenly a problem after updating to Joomla 3.9.1. We kept looking into the wrong places, until it hit us: why not trace the System – P3P plugin and see at what point it injects its headers, and we quickly discovered that the headers were sent on the onAfterRespond event, we even discovered something even more important, is that, when the System – Page Cache plugin is enabled, the onAfterRespond event can only be triggered by the following lines in the cache.php file which is located in the plugins/system/cache folder:

if (JDEBUG)
{
	JProfiler::getInstance('Application')->mark('afterCache');
	JEventDispatcher::getInstance()->trigger('onAfterRespond');
}

But, why was the above code running when the Joomla debug was turned off? Or was it really turned off? So, we checked the Joomla configuration under System -> Global Configuration -> System, and, indeed the “Debug System” under “Debug Settings” was set to “On”. Huh?

We quickly turned it off and the problem was immediately resolved, but why wasn’t the problem happening in previous versions of Joomla, and why didn’t we notice it? The answer to the second question was obvious, the client was using a template that didn’t have any position for debugging. As for the first question, the answer was more alarming: it seems that Joomla 3.9.1 (and perhaps Joomla 3.9.0 as well) has a very quirky bug when the system is in debug mode, where the headers keep piling up in the cache file as the onAfterRespond event is triggered. Reproducing the problem couldn’t be easier:

• Create a Joomla website from scratch with the default data.
• Enable the System – Page Cache plugin.
• Enable debugging in the Joomla configuration.
• Go to the homepage on the frontend (load it a couple of times), and then check the generated cached file under the cache folder, you should see 2 things: 1) the cached file is changing on every refresh (which shouldn’t be the case), and 2) the Cache-Control and the Pragma headers keep getting added to the cache file with every refresh. The end of the cache file would look something like the below after 2 homepage refreshes:

  s:13:"mime_encoding";s:9:"text/html";s:7:"headers";a:8:{i:0;a:2:{s:4:"name";s:12:"Content-Type";s:5:"value";s:24:"text/html; charset=utf-8";}i:1;a:2:{s:4:"name";s:7:"Expires";s:5:"value";s:29:"Wed, 17 Aug 2005 00:00:00 GMT";}i:2;a:2:{s:4:"name";s:13:"Last-Modified";s:5:"value";s:29:"Wed, 19 Dec 2018 04:42:56 GMT";}i:3;a:2:{s:4:"name";s:13:"Cache-Control";s:5:"value";s:62:"no-store, no-cache, must-revalidate, post-check=0, pre-check=0";}i:4;a:2:{s:4:"name";s:6:"Pragma";s:5:"value";s:8:"no-cache";}i:5;a:2:{s:4:"name";s:4:"ETag";s:5:"value";s:34:""f0c25fdc4c9774cdd8632f0a1e83d791"";}i:6;a:2:{s:4:"name";s:13:"Cache-Control";s:5:"value";s:8:"no-cache";}i:7;a:2:{s:4:"name";s:6:"Pragma";s:5:"value";s:8:"no-cache";}}}

So, this was definitely a Joomla bug and not caused by 3^rd party extension. Having said that, there shouldn’t be many Joomla websites out there with debugging turned on!

We hope that you found this post exciting, informative, and helpful! If you need help with solving an “Internal Server Error” problem on your Joomla website, or if you need help with any other Joomla issue, then please contact us. We are always excited for help, our fees are affordable, and our quality is top notch!

We were commissioned by the marketing department of a global education website to enhance the Search plugin of their Joomla website. In essence, they wanted us to include the synonyms of a word in the search results. For example, if someone types in the word “house”, we also include the words “home”, “residence”, etc…

At first glance, this might seem like an ultra complex project, however, for (ahem!) seasoned PHP developers like us, we knew that the solution was not that complex since we worked on a similar project before. In short, we made the whole thing possible the following way:

• We downloaded a thesaurus database. We downloaded the Princeton WordNet database in MySQL format. The beauty of the Princeton thesaurus is that it is free for commercial use, and the client doesn’t have to pay monthly/yearly royalties to use it.

• We imported the WordNet MySQL database into the client’s Joomla database. This step simply consisted of creating all the WordNet MySQL tables (we prefixed all the tables with the abc_wordnet_, where abc_ is the database prefix of the Joomla website, which is defined in the configuration.php file as $dbprefix) and then importing the data to these tables.

• We modified the core Joomla search plugin to include synonyms as well. We modified the onContentSearch function in the content.php file, which is located under the plugins/search/content folder to query the WordNet database for the synonyms of the search keyword and then include them as an OR condition in the search.

The above worked, albeit very slowly, so we couldn’t deploy it on the live server since the website had a huge number of articles and the traffic it was getting was insane. So, we recommended the client to switch the search on their Joomla website to Sphinx for blazing fast search speeds, and they approved our recommendation immediately. So, we implemented Sphinx and then we deployed everything onto the live website, and guess what, search was much much faster than before, even after including the synonyms.

Were there any limitations?

The main limitation was that the synonyms engine only worked when the search consisted of just one word. Making it work for multiple words was extremely difficult because of the many permutations and the hardware limitations. For example, searching for a “large house” becomes a search for a “large home”, “large residence”, “big house”, “big home”, “big residence”, “vast house”, etc… This just wasn’t practical and required a lot more work to implement.

We hope you found our post fun, exciting, and informative. If you need help implementing the above, then please let us know. We are always excited to work on challenging and fun project, our fees are reasonable, and our friendliness is legendary.

A regular client of ours contacted us this morning telling us that after updating PHP from 5.6 to 7.2 on his server, he saw the following fatal error on the homepage of his company website:

[] operator not supported for strings.

We did see this problem before on another client’s Joomla site, and it was caused when a PHP variable is assigned to a string, and then used as an array. In previous PHP versions (versions less than PHP 7.2), this mixed type shenanigan was acceptable, but as of PHP 7.2, this resulted in a fatal error. For example, the following code…

$arrResult = '';
for ($i = 0; $i < 5; $i++)
	$arrResult[] = $i;

…was OK for PHP 5.6, but yields a fatal error in PHP 7.2+. To comply with PHP 7.2, the above code should be:

$arrResult = array();
for ($i = 0; $i < 5; $i++)
	$arrResult[] = $i;

As developers, we’re always happy for strict coding practices. Having said that, we acknowledge that the above can cause many scripts to break under PHP 7.2.

So, how did we fix the problem?

We traced the issue to a file in a 3^rd party system plugin that had the problem described above (a variable defined as an empty string and later used as an array). We fixed the problem by just assigning the variable to an empty array (instead of an empty string), and that fixed the problem!

Is the Joomla core affected by this change in PHP 7.2?

No – the PHP core developers ensured very long ago that the Joomla core was compliant with this change in PHP 7.2. If you see this problem on your website, then it is definitely caused by a 3^rd party extension and not by the Joomla core.

We hope that you found this post useful and that it helped you fix the problem on your Joomla website. If you need help with the fix, then don’t be shy to contact us. We always love to work with new (and, of course, existing) clients, our rates are always right, our solutions are always clean, and we are the friendliest developers on this planet!

If you are an avid reader of our blog, you will notice that our main focus is currently on 3 things: security, performance, and marketing. Marketing is an extremely vast domain and our clients always have exciting challenges for us. For example, we just finished working on a huge HubSpot-Tealium-Constant Contact integration project. Essentially, the project consisted of identifying visitors to the website from their HubSpot profile (using the hubspotutk), getting their reading habits (audiences) from Tealium, and then dynamically subscribing them to newsletter lists in Constant Contact that match their reading habits. It was a tremendously exciting project and we will have a separate post about it in the near future.

Going back to the title of this article, a major client of ours with a very high traffic Joomla site asked us if they can track who downloaded a certain (zip) file from a HubSpot page. They essentially wanted to know the names of those who downloaded the file from that page. A thorough research about the subject revealed that HubSpot do not have this functionality built-in, which is weird, considering that HubSpot is a major player in the online marketing world and is used by many Fortune 500 companies out there. So, we had to implement this functionality, but how?

We quickly thought it out and figured out (a very smart) workaround:

• We create a PHP file called hubpsot-who-downloaded.php on the Joomla website (say under a scripts folder).

• The file hubpsot-who-downloaded.php will take the hubspotutk of the person who tried to download the zip file as a _POST parameter. The file will then send an API request to HubSpot asking it about the email of the person with that hubspotutk. Once that information is received from HubSpot, it will be saved in a MySQL table called hubspot_who_downloaded. The aforementioned table will have 3 fields: the id of the row, the hubspot token (hubspotutk or hutk), and the email of the person.

• We then, on click of the file in HubSpot, add a JS script that will send the hubspotutk to the hubpsot-who-downloaded.php file using an AJAX POST request (note that the hubspotutk is always stored in the cookies).

• That’s it!

We implemented the above method immediately, and it worked like a charm on first try! We were impressed and so was the client. Note that an alternative method, if you don’t want to use a MySQL database, is to create a new property (such as downloaded_file), have it default to “No”, and then update it to “Yes” from the API for those who actually download the file.

We hope that you found this post informative. Now if you need help with the implementation then just contact us. Our experience in HubSpot and its API is unquestionable, our work is super clean, and our fees are ultra affordable!

A client of ours emailed us this morning and told us that the search functionality on their website is acting funny, and asked us to take a look. So, we tested the search by entering a term relevant to the client’s business, and we noticed that many titles in the search articles had something like <span class=”highlight”> in them.

A quick research about this issue revealed that this was a Joomla core problem, and fixing it consisted of changing the following line in the function highlight (which is located in the view.html.php file, which, in turn, is located in the components/com_search/views/search) folder from:

$highlighterLen = strlen($hl1 . $hl2);

to:

$highlighterLen = '';

Doing the above nearly fixed the problem, because we were still seeing “&#16” at the end of most titles. Also, we were not convinced that this so obvious problem exists in the Joomla core, and nobody stepped up and resolved it, so, we investigated the issue further.

Our Sherlock Holmes investigation finally led us to the client’s template, specifically the file default_results.php, which is located in the templates/[joomla-template]/warp/systems/joomla/layouts/com_search/search folder. The aforementioned file, which is the search results listing override in a YooTheme powered template, contained the following line:

<h1 class="title"><?php echo $this->escape($result->title); ?></h1>

The above line contained one method, which was the essence of the problem, and it was $this->escape. In short, this method was trying to escape what was already escaped by Joomla. Modifying the above line to this one fixed the problem:

<h1 class="title"><?php echo $result->title; ?></h1>

So, the next time you are seeing any weird HTML code in your search results, don’t immediately blame Joomla for it (and, most importantly, do not change any core files to fix the problem) as it might be your template trying to re-escape content. Don’t forget, we’re always a call or an email away if you need help in fixing the problem, oh, and our fees are super affordable!

In this day and age, most web developers out there want to load everything asynchronously – you know – just to please Google, despite the fact that loading JS files asynchronously may result in some erratically unpredictable behavior. In fact, from an SEO perspective, ensuring that the website looks good on mobile devices is an easier and better thing to do than to try to load files asynchronously. But who cares, the most important thing in the world is to have a PSI score of a 100, right?

Having said that, we understand that, at many times, pressure comes from above (just like air pressure), and many Joomla developers out there are left with no choice, and that’s why, we will willingly divulge, in this post, a well kept secret: how to load JS files asynchronously in Joomla 3.8, the right way!

If you are a Joomla veteran, then you probably already know that you can load a file asynchronously the following way:

$objDocument = JFactory::getDocument();
$objDocument->addScript('https://www.[yourjoomlawebsite].com/templates/yourtemplate/js/yourscript.js', 'text/javascript', false, true);

The problem is that the above method has been deprecated and will be later removed from the Joomla core. The new (and cleaner) method for loading a JavaScript file asynchronously in Joomla is the following:

JHtml::_('script', 'https://www.[yourjoomlawebsite].com/templates/yourtemplate/js/yourscript.js', array('version' => 'auto', 'relative' => true), array('async' => 'async'));

If you want to defer the loading of a JavaScript file, then you would use:

JHtml::_('script', 'https://www.[yourjoomlawebsite].com/templates/yourtemplate/js/yourscript.js', array('version' => 'auto', 'relative' => true), array('defer' => 'defer'));

Essentially, the last array in the above function call is the attributes array, so any Joomla supported attribute should technically work. If you add a non-supported attribute, such as ‘myattribute’=>’myattribute’ in the last array, then Joomla will add the following at the end of the script HTML tag: myattribute=”myattribute”. This is useful in case you want to load custom attributes that are later processed by an internal or external script.

So, what is the function in the Joomla core that does all the work?

If you really want to know, then it is the function addScript which is located in the Document.php file, which in turn is located in the libraries/src/Document folder.

We hope that you found this post interesting and useful. If you need help loading some JavaScript (or CSS) files asynchronously on your Joomla website, then look no further. Just contact us and we’ll do the work for you very quickly and for very little money.

An exciting project that we have just finished is DFP (now Google Ads) ad targeting using HubSpot on a large Joomla website.

In short, here’s how the system works:

• Someone visits the Joomla website and fills in a HubSpot form – the person is automatically assigned a hubspotutk which is associated to his HubSpot profile.

• When the same person visits the website again, a PHP script is triggered (using Ajax) to get all his information from HubSpot using the hubspotutk cookie. This information is then saved to a local cache. This step is necessary to eliminate the connection overhead to HubSpot and to circumvent HubSpot’s 40K API requests/day limitation. After the information is saved to the local cache, it is then pushed to the local storage of the browser in order to reduce the number of requests to the client’s server.

• When the same person visits the website afterwards, a different script is triggered and passes on all the browser information to DFP using key-value ad targeting (for example, yearlyIncome = >$200,000, businessTraveler = yes, etc…), resulting in accurate ad targeting for the person.

Of course, we faced many challenges while implementing the project, for example, we discovered that the value of the hubspotutk does not have a 1-1 relationship to a a HubSpot profile. In other words, the same HubSpot profile can be associated to 1 or more hubspotutk. In fact, this is the main reason why 1) a HubSpot export does not contain the hubspotutk and 2) there are no HubSpot API calls that return the hubspotutk in their response.

We hope you that you found our process for DFP ad targeting using HubSpot interesting. If you need help with the implementation, then contact us and we’ll do the work for you affordably, cleanly, and elegantly.

A new client called us a couple of days ago telling us that he successfully updated the Joomla website of the company he works in, however, he told us that they had to revert back because of some issues on the website resulting from the update. He told us that after reverting back, he saw the following error on the website:

The parameter platforminfo.class must be defined.

The above error was displaying on both the frontend and the backend, which made debugging the problem a bit hard. A quick research on the subject revealed that this error was generated by the libraries supporting the RocketTheme template because of an incompatibility issue with PHP 7, and that the solution to the problem would be to update the RokCommon plugin (and library) to the latest version, which is kinda painful considering that both the frontend and the backend are down (side note: we never liked the RocketTheme templates, they’re really heavy and they are never fun to debug).

We asked the client whether he updated his PHP version as well, and he said that he didn’t and nobody else was working on the server. Of course, the question was, how come the original website worked with PHP 7, and the reverted website was showing this error. There was only one answer to this question: the revert wasn’t complete; either the filesystem or the database wasn’t reverted properly.

So we emailed the client on whether he reverted both the filesystem and the database and whether he made sure that he used the filesystem and the database from the same backup, and he told us yes, but he said that he noticed something weird thing when trying to revert the database, and it was that when he tried to load the database backup from the shell using the following command…

mysql joomla_database < joomla_database.sql

...he noticed that the command was executed successfully, yet the database in phpMyAdmin was still empty. Of course, that shouldn't happen, and a further investigation into the issue revealed that the problem was that the client had 2 servers, an old one and a new one, and he was restoring the filesystem to one server, while the database to the other. We addressed that and that fixed the problem!

But what if both the filesystem back and the database backup are correct and "in sync"?

In that case, the cause of the problem is a PHP update (to 7.x) that is not compatible with the current version of the Rok Joomla libraries (e.g. the the RokCommon library). The easiest solution to this problem is to have the website use an older version of PHP, which can be done using the MultiPHP Manager tool in WHM.

If you are seeing the error The parameter platforminfo.class must be defined on your Joomla website all of a sudden, then make sure that your website uses PHP 5.6 instead of PHP 7, and then update your RocketTheme template and all the Rok plugins/extensions (especially RokCommon), and then update back to PHP 7. If you are seeing this problem after reverting back to a previous version of your website, then a possible cause of the problem is that the database and the filesystem are not in sync. If you need help anywhere in the process, then you can always contact us. Our work is fast, our quality is top notch, and our fees are super affordable.

A client called us yesterday morning and told us that they were having issues trying to update their Joomla website. They didn’t give us any details, they just told us that they were seeing an error and that they wanted our help. As usual, we gratefully and immediately obliged…

We logged in to the backend of their Joomla website and we clicked on the Update Now button, and then we clicked on the Install the Update button on the Joomla Update page. On the next screen, the one with the progress bar, the following error popped up:

ERROR: JSON.parse: unexpected character at line 1 column 1 of the JSON data

The popup also had a lot of gibberish as you can see from the below image:

Thinking that this was a JavaScript issue, we checked the FireFox console by pressing F12 and then clicking on the Console tab. We saw the following error:

XML Parsing Error: unclosed token
Location: https://www.[ourclientjoomlawebsite].com/administrator/components/com_joomlaupdate/restore.php
Line Number 1, Column 1:

The restore.php mention in the above line immediately reminded us of the Ajax Loading Error: Category not found issue that we addressed some time ago, so, we checked whether the main .htaccess file had any rules blocking the execution of PHP scripts with the exception of the index.php file, and it had none.

We then thought, maybe this is just a FireFox issue, so we tried to do the update on Google Chrome (which is now the dominant browser), but we saw a similar error:

ERROR: Unexpected token in JSON at position 0

Again, the popup had a lot of gibberish following the above error message. The console, however, did not show any errors. Weird…

We knew we could just do the update manually, but we didn’t want to do that, we wanted to know what the problem was, even if it’s at the cost of our own sanity. Luckily, we didn’t have to suffer for long, as we accidentally saw an .htaccess file under the administrator/components folder with the following line:

RemoveType .php

This meant that any PHP file, in the administrator/components directory gets displayed instead of being executed, hence the gibberish that we were seeing as the response to the ajax update request (it was the whole restore.php file). So, we removed that line, and we tried the update again, and this time it worked. Hooray!

So, what’s with the “RemoveType” .htaccess line?

A long time ago, we recommended the use of the RemoveType line in the images folder, to prevent execution of malicious file that were uploaded to the website through an exploit. However, using this .htaccess line anywhere else is not recommended, as it compromises the security of the website by exposing the actual content of all the PHP files. If you are sure that PHP files under a specific folder should never be accessed directly, then you should have the following line in your .htaccess file (the .htaccess file inside the aforementioned folder, that is):

<Files *.php>
    Order Allow,Deny
    Deny from all
</Files>

The above code denies direct access to all the PHP files under that folder. To add an exception for a PHP file, such as the restore.php file, then all you need to do is just add the following to the .htaccess file that you have just created:

<Files restore.php>
    Order Allow,Deny
    Allow from all
</Files>

We hope that you found this post useful. If you are trying to update your Joomla website but you are having issues, then please contact us. Our rates are affordable, our work is clean, and we are always eager to serve new (and old) clients!

We know, we’ve been lazy this month, as this is our first article and it’s now the 31^st – but, we were working on exciting projects so rest assured that we’ll make it up to you in the next month.

One of the exciting projects that we worked on was the sharing of a database table across 3 different Joomla websites. We tried several methods to reach this goal, but only one of them proved to be stable and secure, and it was the MySQL view.

For those who don’t know, MySQL, like almost any other RDBMS, offers an interesting functionality called a view. A view is more or less like a table, except that its content comes from other tables. Also, if there is a one to one relationship between a view and a table, then updating the view can also update the table. Let us explain…

Say you have a table called abc_emails in the database firstdatabase which serves one of your Joomla websites. Say you have another Joomla website which is served by the database secondatabase. If you want to use the table abc_emails on your second Joomla website (that has def_ as its alias), then all you need to do is to run the following query while using the second database:

CREATE ALGORITHM = UNDEFINED DEFINER=`firstdatabase_user`@`localhost` VIEW `def_emails` AS SELECT * FROM `firstdatabase`.`abc_emails`;

Once you do that above, any row that is added to the abc_emails table will automatically appear in the def_emails view, and any row that is added to the def_emails view will automatically be added to the abc_emails table. Nice, isn’t it?

You can do the above with as many Joomla websites that you want, but make sure that you only point a view to a table – do not point a view to a view, for example, do not point the view ghi_emails to the view def_emails, but rather point it to the abc_emails table.

Of course, there are other methods of sharing tables across multiple databases, but they aren’t as simple and as secure. For example, you can create a cron job that copies rows from tables to each other, but we almost guarantee you that you will regret it, and that you’ll run into many obstacles (trust us, we tried it). You can also update a table from another website by modifying the connection parameters to those of the first website, but this is also a non-solid and a non-secure option.

We hope that you found our post fun and exciting. If you need help with the implementation, then please contact us. Our fees are affordable, our work is clean, and we always love working on Joomla challenges!

We know, you’ve always dreamt of truncating a database table using the Joomla database library, but you didn’t know how to do that, so you used the following code instead:

$db = JFactory::getDbo();
$sql = 'TRUNCATE TABLE `mytable`';
$db->setQuery($sql);
$db->execute();

The above code works, but it doesn’t say much about your Joomla skills. A cleaner way of truncating a database table is to use the below code:

$db = JFactory::getDbo();
$db->truncateTable(`mytable`);

With the above code, you will accomplish 2 things: 1) You will achieve your dream of using the Joomla abstraction layer for truncating a table, and 2) your code will look more professional even though, well, you are truncating a table!

Will the truncateTable method work on all database drivers?

Yes. It will work with all database drivers since it is defined in the main database driver abstraction class.

Will the truncateTable method truncate any table, including core Joomla tables?

Yes – it will. In fact, there are no checks whatsoever on the truncateTable method, it can truncate any table, including, for example, the very critical #__assets table. So, be really really careful when using this function and do not rely on any Joomla built-in precautions because there aren’t any. You have been warned!

Is there a dropTable method?

Yes – there is. It works the same way as the truncateTable method, except that it drops (deletes/removes) a table instead of truncating and that it is defined at the used driver level, such as the MySQLi driver (not in the main driver), because of the difference in the implementation of the DROP SQL command between different database engines. For example, Oracle doesn’t have the DROP TABLE IF EXISTS functionality, but MySQL does.

We hope that you found today’s post helpful. If you need help, any help, with your Joomla website, then please contact us. Our work is super quick, our quality is top notch, and our rates are extremely affordable.

A few days ago, a managed client of ours called us and told us that their website stopped working, and that that event coincided when an employee was trying to upload a file to the images folder using FileZilla.

We immediately knew what the issue was, as it happened to another client of ours a couple of years ago: a staff member uploading a file to the images folder accidentally moved the components folder to the inside of the images folder. A quick investigation revealed that this was exactly what happened this time, with the exception that it was the includes folder that was moved to the images folder. Fixing the problem simply consisted of moving the includes folder back to its original location (directly under the public_html folder).

Needless to say, this whole issue made us feel unsafe, so we needed to find a solution to the problem!

Using FTP instead of SFTP was outside the question for security reasons, and, even if we headed that way, we would still run into some risks of moving directories inside each other.

After some very long research, we discovered that the following shell command…

chattr +i dir1

…makes the directory dir1 immutable. Meaning that it cannot be moved or altered in any way.

So, we issued, as root, the following shell commands when inside the public_html directory:

chattr +i administrator/components administrator/help administrator/includes administrator/language administrator/manifests administrator/modules administrator/templates administrator/.htaccess administrator/index.php
chattr +i bin cli components includes language layouts libraries modules plugins templates .htaccess configuration.php defines.php index.php

After doing the above, we tested if we were able to move any of the above folders to a different location from within FileZilla (by dragging and dropping), and it didn’t work. Hooray! We solved the problem.

But, what happened to the cache, media, and tmp folders?

Joomla needs to have write access to the cache, administrator/cache, media, and tmp folders for its basic operation, and that’s why we excluded them from the shell commands above, as an immutable file/folder cannot be written to.

What if someone wants to install an extension?

Installing an extension won’t work with the immutable flag set. So, if you need to install an extension, you will need to first remove the immutable flag from the above folders by issuing the following shell commands (again, you must do that root, as root is the only user that can set/unset the immutable flag):

chattr -i administrator/components administrator/help administrator/includes administrator/language administrator/manifests administrator/modules administrator/templates administrator/.htaccess administrator/index.php
chattr -i bin cli components includes language layouts libraries modules plugins templates .htaccess configuration.php defines.php index.php

How do you know if a file/folder has the immutable flag set?

The lsattr shell command does the trick. So, if you want to know which extended flags are set on a specific file/folder, all you need to do is to issue the following command:

lsattr file-or-folder

For example, to know which extended flags are set on the index.php file, you issue the following shell command (you don’t have to be root to use the lsattr command):

lsattr index.php

Won’t making the folders/files immutable enhance security?

Definitely! As you can see in the first set of shell commands in this post, we are making the index.php immutable, which prevents alteration to this file by anyone. In fact, the only way to modify this file would be to login as root and then unset the i flag.

We hope that you found this post useful. If you need any help with the implementation, then all you need to do is to contact us. We’ll do the above for you in a breeze and we won’t charge you an arm and a leg for it.

When a Joomla developer is asked to add some JavaScript tracking code to a Joomla website, he typically codes a content plugin in order to achieve this result. This, of course, works. But, it’s a very inefficient way of doing things. Why? Because this can be done easily with a custom HTML module and a layout override. Here’s how:

• Open the index.php file under the templates/[your-joomla-template] folder.

• Just before the closing body tag (e.g. just before </body>) add the following code:

  <jdoc:include type="modules" name="code-bottom" style="raw" />

  With the above line, we have added a new position to the template where we can add our JavaScript code.

• Create a nodiv layout override for the Custom (previously called Custom HTML) module. You can find the full details on how to do so here.

• Create a new Custom module, and assign it to the code-bottom position.

• Add your JavaScript to the module that you’ve just created, and then click on the Advanced tab, and finally choose nodiv from the Layout dropdown.

• Save the module and clear your Joomla cache.

• That’s it! Now you will have your tracking code on all pages (you can even easily choose which pages to exclude/include since you are using a module).

But what if the JavaScript code is slightly dynamic?

Well, in this case, you can create a specific module override (instead of the nodiv module override) and then add your logic there.

So, are there real advantages over creating a plugin?

Definitely! First, the more plugins you have installed and activated on your Joomla website, the slower it is. Second, we’re leveraging an existing extension to do what we want (reusability), and third, once the module position and the layout override are in place, a Joomla administrator with no coding experience will be able to add any tracking code to the website.

We hope that you found our short post useful. If you need help with the implementation, then you can always contact us. We are always ready to help, we love working with new clients (and, of course, existing clients), and our rates are very affordable.

Earlier today, we ran into a very weird issue… We were making some changes to a cron PHP file (under the cli folder), but our changes were not taking effect. This was very odd because we were sure that no caching was enabled anywhere on the server, at least so we thought.

What was odd is that if we renamed the file, to something like cron-file-1.php instead of cron-file.php, our changes were taking effect, but only once. So, if we made additional changes, we had to rename the file to cron-file-2.php, cron-file-3.php, cron-file-4.php, and so forth…

A few hours of investigation revealed that the cause of the problem was (unsurprisingly) PHP’s built-in caching system, OPCache, which, to put it simply, translates normal PHP code into machine code and stores it into the server’s memory in order to eliminate the compilation phase each time a PHP file is requested by the web server, which will lead to some performance gain.

So, in short, we just needed to disable OPCache, at least for that particular folder, in order to get our code to refresh.

So, how did we disable OPCache?

We just created a .user.ini file and placed it in the cli folder, and then added the following code to it…

opcache.enable=0

…and that did the trick! The second we created the .user.ini file, the problem was solved. Our PHP changes were taking effect instantly, and we were very exalted!

But isn’t a good idea to leave OPCache enabled?

While we don’t think highly of the advantage that OPCache brings to the performance of a web application, we still believe that any improvement is a good improvement, so it’s a good idea to turn it back on (by removing the above line from the .user.ini file) once you’re done with it. Some claim that the improvement in response time can be as high as 70%, but we cannot confirm this.

So, the next time you make a change to a PHP file, and you don’t see your change taking effect immediately, try disabling OPCache by just adding the above line. If it doesn’t work, or if you need help with the implementation, then please contact us. Our fees are affordable, our work is quick, and we are the Joomla experts!

Lately, we have been working increasingly on marketing (from a very technical perspective), specifically on ad targeting. Essentially, our clients are being pressured by their own clients to target visitors with matching ads. For example, if someone is a big reader of car articles on a news website, then he will be targeted with car ads (instead of just throwing random ads at the person and hoping he’ll be interested in one of them).

Google Adsense does this automatically. DFP (DoubleClick for Publishers), however, needs to be told how to target ads, which is typically done using key-value targeting.

So, what is DFP’s key-value targeting?

In short, key-value targeting in DFP consists of some variables and their values passed from the application (in our case the Joomla website) to the ad server, which searches the inventory for a matching line item, eventually prioritizing (and displaying) the ad belonging to the line item. Yes, we know that it looks like gibberish at first glance, but let us explain by giving you an example…

Let’s say you have a news website, and a client representing a car company who wants to advertise on your website. The client wants you to target people who read a lot of articles about new cars with their ads. So, what do you do? First, you login to your DFP account, and then you do the following:

• Click on Inventory on the left panel, and then click on the Key-values link.

• Click on the “New Key” button, and then fill in the name of the field, such as “CarTarget”, and the possible values, such as “Y” and “N”.

• Go to the line item of the ad (we are assuming that you have already created the ad unit, the line items, and the creatives), and then click on Settings.

• Scroll down to the bottom of the page, and then click on the Key-values link (under the Add Targeting section).

• In the Select a key input box, choose the CarTarget variable that you have just created, and then, in the box just below it, enter “Y” (this means that this line item will only display when CarTarget is set to “Y”).

• That’s it!

Of course, now you will need to pass the CarTarget variable to DFP from the Joomla website (or any web application) using DFP tags. Here’s how we do this:

• We first create a database table called dfp_targeting that has 3 fields: userhash, keyvalue, and counter (you will later understand what each of these fields is).

• For every visit, we check the cookies for a cookie called userhash, if it exists, we increase the counter of the value of the userhash in the dfp_targeting table by 1 where the category field matches the current category being viewed by the user. If such a row doesn’t exist in the table, then we create it, and we set the initial counter to 1.

  If the userhash cookie doesn’t exist, then we create the cookie and we set its value to a 64 character hash, and, on the next visit, we store that cookie in the database (along with the category field and the counter field, which are set to the current category being viewed by the user, and the number 1, respectively).

• Once a user visits the website with, say 50 as the counter for the “Car” category, we set the CarTarget DFP key to Y and we push this information to DFP using the following JavaScript code:

  googletag.pubads().setTargeting('CarTarget', 'Y');

  Note: We implement all the server/client side code in the article layout override of the Joomla website (e.g. the file templates/[template-name]/html/com_content/article/default.php).

• That’s it!

Once the above is done, the Joomla website will start displaying targeted ads to regular readers of the Car category.

We hope that you found this article fun and informative. If you need help with implementing DFP ad targeting on your Joomla website, then please contact us. Our fees are super cheap, our coding is super clean, and we are super fun to work with!

One of our non-regular Joomla clients revealed to us their wish to build their own CMS. They told us that the number one reason why they’re doing this is because maintaining a large Joomla website is very costly. They also cited other reasons, including having to go through the trouble of updating their site every time there is a new Joomla release, the absence of some functionalities (extensions) that they need, the generic Joomla core that is not optimized by default for their website, and the fact that their website is constantly under attack because, well, it’s a Joomla website.

What was surprising to them is that we agreed with everything they said: yes, maintaining a large Joomla website is costly, and yes, updating it is not fun, and yes, Joomla is a generic open source product that must be customized to accommodate a serious website. But again, these issues are inherent to any open source mainstream CMS, including Drupal and WordPress. Naturally, the client told us that since they’re moving to a custom made CMS, they won’t have any of these issues, right? Well, as they say, the grass is always greener on the other side of the fence…

You see, creating a custom made CMS is no fun job, and, to this date, we have yet to witness a case where the client converted to a custom made CMS and not regretted their decision later on (that is, of course, if the custom made CMS reaches the deployment phase, as many of these custom made CMS projects end up being killed after a couple of years because they were only supposed to take 4 months), and this is mainly because of the five following reasons:

1. Being at the mercy of developers: A common problem that most companies do not foresee is that when they switch to a custom made CMS they will instantly become at the mercy of their developers. What if their developers suddenly leave during the project? What if the code is so (unnecessarily) complex that other developers would not be able to decipher it? What if they use obsolete libraries or products? This is by far the most crucial problem to consider if you want to switch to a custom made CMS.

2. The myth of perfect security: Joomla has always been criticized for its security, but this is mostly bad reputation that is lingering from almost a decade ago. Joomla’s security at the moment (we’re in June 2018) is top notch. The same cannot be said about custom made CMS’s, which are always thought to have perfect security, but this is just an illusion. In reality, the security on any custom made software is extremely feeble, but, since it is not open source, its potential exploits are only known to a limited number of people (including the original and the current developers, which make these developers even more dangerous). We have seen the code of quite a few closed source CMS products, and we are confident when we say that their security is worse than that of the worst CMS on the market.

3. The inability to catch up with technology: In most cases, by the time a custom made CMS is finished, the technology it was built on becomes already obsolete, which is not a huge problem. But, the fact that this obsolete technology will not change for many years to come exposes the CMS to many issues, such as sticking with a very old (and exploitable) environment just for the sake of keeping everything working.

4. The very expensive development work: All transactions in this world are subject to supply and demand. If gold, for example, was in huge supply, then it would’ve been very cheap, but it is not, and so are the developers who can work on custom made CMS (we’re not saying that such developers are like gold, which is often not the case, we’re just saying that they’re rare). A very simple task, such as changing a font in the template, can take a couple of weeks and can end up costing the client tens of thousands of dollars. Imagine what would the cost be for introducing a new feature on a custom made CMS.

5. Bugs, more bugs, and a very unfriendly GUI: Ever worked on or seen a closed source CMS? Then you may already know that the 3 prominent characteristics in such a product are bugs, more bugs, and an extremely unfriendly GUI (Graphical User Interface). The thing is, unlike mainstream CMS’s, custom made CMS’s do not get tested by thousands (if not millions) of people, and that’s why they end up really, really buggy and with a horrible GUI, resulting in a very bad experience, and ultimately causing decent staff to quit.

We hope that you found our post helpful and convincing. If you still want to dump Joomla and have a custom made CMS developed for your business, then why not contact us before to discuss your case? We are eager to help you make the right decision. Please note that our super affordable fees apply!

If you want to see something interesting, then do the following:

• ssh to the server hosting your Joomla website.

• Change to the domlogs folder.

• Run the following grep on the main log file of your domain:

  grep 'MobileSafari' yourjoomlawebsitelogfile.txt > mobilesafari-log.txt

After doing the last step, you should have a mobilesafari-log.txt file in your current directory. Open it for editing and you will see something like the following:

[user-ip] - - [31/Apr/2018:10:16:46 -0400] "GET /apple-touch-icon-120x120-precomposed.png HTTP/1.1" 404 20 "-" "MobileSafari/602.1 CFNetwork/808.3 Darwin/16.3.0"
[user-ip] - - [31/Apr/2018:10:16:46 -0400] "GET /apple-touch-icon-120x120.png HTTP/1.1" 404 20 "-" "MobileSafari/602.1 CFNetwork/808.3 Darwin/16.3.0"
[user-ip] - - [31/Apr/2018:10:16:46 -0400] "GET /apple-touch-icon.png HTTP/1.1" 200 4581 "-" "MobileSafari/602.1 CFNetwork/808.3 Darwin/16.3.0"

Of course, the first question that would probably hit your mind is “what is that?” Well, that is what happens when a mobile Apple device accesses your website. It first sends a request with a weird user agent (MobileSafari) to check for “touch icons” – these “touch icons” are later used by the browser to create a shortcut of your website (users can then touch the shortcut to load your website). As you may have guessed, the image of the shortcut is the apple-touch-icon*.png image (the Apple device will choose one of the 3 images and will display it as a shortcut button). This is all fine and dandy, but there is one problem: your website doesn’t have any of these apple-touch-icon* buttons, so, this Apple “feature” ends up slowing down your Joomla website because of the resulting 404s – this is especially the case on high traffic Joomla sites. To give you an example, a high traffic Joomla website that we manage gets about 60K 404s/month just because of the Apple touch icons.

So, what can be done about this?

There are 4 things that you can do to address this issue:

1. Redirect, at the global .htaccess level, any traffic with MobileSafari as its user agent to a png file of your choice (or to a simple HTML page). Note that the option to redirect to a specific png file is a bit tricky, since some Apple devices require different image sizes (e.g. 152×152 instead of 120×120). Also note that it is important that you don’t block the request (or the user agent), as blocking access to those images through an .htaccess rule may result in having the device’s IP automatically blocked!

2. Create a condition in your defines.php to handle this situation. You can just display a simple 404 message (such as die(“Not Found”);) when MobileSafari is the user agent in order to prevent Joomla from continuing to process the request.

3. Create the PNG images that Apple mobile devices want. In short, you will need to create the following images and place them in the root directory of your Joomla website (quick rant: it is just rude of Apple to expect those proprietary images to exist at the root level of the website by default):

   apple-touch-icon-120×120-precomposed.png
   apple-touch-icon-120×120.png
   apple-touch-icon-152×152-precomposed.png
   apple-touch-icon-152×152.png
   apple-touch-icon.png

4. Do nothing. Yes – do nothing. This should be the case when your website has little traffic and you have an abundance of free resources on your server.

We hope that you found this short post useful and informative. Keep in mind that at itoctopus, we don’t offer design services (so we can’t create those Apple buttons for you), but what we can do is help you redirect traffic from the MobileSafari user agent in order to reduce pressure on your server. If you want us to do that for you, then just contact us. Please note that our super affordable fees apply.

One of the most annoying problems with the System – Page Cache plugin is the fact that when you update an article, then that update won’t reflect on the website immediately, as the cached page would still be served to the visitors and won’t be refreshed until it “expires”. In fact, the absolute majority of Joomla news websites completely avoid the “System – Page Cache” core plugin just because of this issue, and instead use a 3^rd party extension to fulfill their caching needs, which is a far-from-ideal solution, as 3^rd party caching extensions have their own “quirks”.

So, what is the ideal solution?

The ideal solution is, of course, to have the System – Page Cache plugin refresh all the affected pages when an article is updated. An almost ideal solution is to develop a plugin that will refresh the affected pages when an article is updated, and this is exactly what we did at itoctopus!

We developed a system plugin that cleaned the following pages when an article was updated:

• The article’s page
• The print version of the article’s page
• The article’s category page
• The article’s category RSS page
• The homepage
• The main RSS feed page

In essence, we did the above when the onContentAfterSave and the onContentChangeState events were triggered, and when the $context was “com_content.article”. We cleaned all the cache entries when $context was not “com_content.article”. We also cleaned all the cache entries when the onExtensionAfterSave event was triggered (so, for example, when a module, any module, is saved, then the whole cache is wiped out).

We deployed the plugin on a major website and the whole thing worked like a breeze: the server’s load decreased substantially (at this very moment, the 15 minute load is 0.84, which is really hard to beat on a website receiving a 100,000 page loads every day). Not only that, the staff was much happier, everything responded quicker than before, and their changes were reflected instantly (so they got the best of both worlds).

We were happy because the client was happy and because of the major performance gain on the server, but we were also concerned. Our heavy and low level work with the Joomla caching mechanism revealed that the latter needs a substantial overhaul as it doesn’t seem to have been planned well. For example, it is very easy to generate and delete a cached version of a page when you are actually on that page, but it is very hard to do anything with the cached version of a page if you are not on that page (especially if you are trying to do it from the backend).

We hope that you found this post, and our concept for efficiently refreshing the cache, useful. If you have any questions, or if you need us to implement the above on your Joomla website, then please contact us. Our fees are right, our work is clean, and we are experts in Joomla performance.

Note: The following post assumes that your Joomla website is powered by a WHM/cPanel server and that the Eximstats service is enabled (you can enable the Eximstats service from WHM’s Service Manager).

At itoctopus, we go at great lengths to monitor the security of our managed Joomla websites, to the point where we check the Eximstats database for anything suspicious. The Eximstats database, in case you’re wondering, is a SQLite database containing a record of every email sent from the server (this is the case for almost all WHM/cPanel powered servers, where Exim is the default mail server).

But how can the stats of an email server signal anything suspicious on the Joomla website?

Well, if your Joomla website usually sends about a 1,000 emails every month, and the Eximstats database contains about 100K entries for that particular month, then this a huge signal that your website is compromised.

So how do you go by analyzing the EximStats database?

As mentioned at the very beginning of this post, the EximStats database uses the SQLite database engine, which is fairly similar to MySQL, but with one major difference: a SQLite database is just one file, unlike a MySQL database which is spread across many files. This makes the whole process of analyzing the database fairly simple. All you need to do is the following:

• Install phpLiteAdmin onto the server (phpLiteAdmin is the phpMyAdmin equivalent for SQLite). This can be done by issuing the following shell commands on the server:

  cd /home/[cpanel-user]/public_html
  mkdir phpliteadmin
  cd phpliteadmin
  wget https://bitbucket.org/phpliteadmin/public/downloads/phpLiteAdmin_v1-9-7-1.zip
  unzip phpLiteAdmin_v1-9-7-1.zip

• Copy the Eximstats database to the newly created phpliteadmin folder:

  cp /var/cpanel/eximstats_db.sqlite3 /home/[cpanel-user]/public_html/phpliteadmin/

  Note: by default, the Eximstats database retains data for one month.

• Point the phpLiteAdmin application to use the Eximstats database that was just copied:

  cd /home/[cpanel-user]/public_html/phpliteadmin
  mv phpliteadmin.config.sample.php phpliteadmin.config.php

  After doing the above, open the file phpliteadmin.config.php and then change the following code:

  $databases = array(
          array(
                  'path'=> 'database1.sqlite',
                  'name'=> 'Database 1'
          ),
          array(
                  'path'=> 'database2.sqlite',
                  'name'=> 'Database 2'
          ),
  );

  to:

  $databases = array(
          array(
                  'path'=> 'eximstats_db.sqlite3',
                  'name'=> 'Database 1'
          ),
  );

• Launch the phpLiteAdmin application by pointing your browser to https://www.[yourjoomlawebsite].com/phpliteadmin.

• Login with the password “admin” (without the quotes).

• Check the sends table for an abnormal volume of sent emails (abnormal, in this context, is fairly subjective).

If you find anything out of the ordinary in the EximStats database (e.g. a huge number of sent emails), then your Joomla website is likely hacked. Try our super quick shell commands to find the hacked files on your Joomla website. If you can’t find anything, then it would be a good idea to contact us. We are experts in Joomla security, we will clean your website, and you won’t have to sell your business to cover our fees!

On any news site, one of the most convenient things for editors is to have an “Edit Article” link on the article page. Such a link will save a lot of time for the editors since without it they will need to go to the backend, search for the article (which will put pressure on the database server if there are many articles on the site), and then click on the article in order to edit it.

Joomla does have frontend editing, but the user must be logged in to the frontend (and not to the backend) to be able to edit articles directly from the frontend, and serious news sites don’t allow frontend logins. So, how can one overcome this limitation?

A straightforward approach would be to add an “Edit Article” link to every single article page, for everyone. That can be done by adding the following code to the article override page in the template (e.g. in the default.php file under the templates/[template-name]/html/com_content/article folder):

<a href='<?php echo(JURI::root()); ?>administrator/index.php?option=com_content&task=article.edit&id=<?php echo($this->article->id); ?>' target='_blank'>Edit Article</a>

The above code will display an “Edit Article” link (in the location where it’s placed) that will direct a backend logged-in-user to the actual edit screen of the article, which is excellent, but is not ideal. The thing is, the “Edit Article” link will be displayed for everyone, and not just the editors, and this is where it gets increasingly tricky…

A casual Joomla developer would probably say: just get the user by using JFactory::getUser(), check if the uses can edit articles, and then display the link. The problem with this approach is that the JFactory::getUser() method will return an empty object on the frontend if the user is logged in from the backend. Try it yourself!

So, how can this problem be solved?

The easiest solution to this problem is to set a site-wide cookie when the onUserLogin event is triggered, and this is done by creating a plugin and then adding the following code to the onUserLogin method of that plugin:

setcookie('edit_article_efpke4uax2fmp789', '1', 0, '/');

So, when someone logs in to the backend of the website, the value of the edit_article_efpke4uax2fmp789 will be set to 1. This will allow us to check for that cookie on the frontend (e.g. in the article layout page) and display the “Edit Link” if that cookie is present and set to 1.

Are there any caveats?

Yes – a few. First, we will have issues with the above approach if Joomla’s native System – Cache plugin is used, as the “Edit Article” link may also appear for site visitors (non-editors). Granted, they will not be able to do anything with that link (it will just redirect them to the login page), but it’s just not professional. This situation will happen when editors check the article page before the visitors, but, if the reverse happens, then the editors just won’t see the link. The solution to this problem consists of using a modified System – Cache plugin that will take into consideration the “Edit Article” link.

Another problem is that the “Edit Article” link will appear for the editors even when they’re logged out, and this is because we are not unsetting the cookie on user logout. Now, while this can be done with the onUserLogout event, it is not enough, because, in the absolute majority of cases, editors do not click on the “Logout” button, but are rather logged out automatically by the system when their sessions expire.

Also, the “Edit Article” link does not take into consideration the fact that the article might be locked (“checked out” in Joomla’s terms) by another editor. However, this is easily remedied with a simple check prior to displaying the link.

We hope that you found our post useful and informative. If you need help with implementing the “Edit Article” link on the frontend of your Joomla website, then please contact us. Our fees are affordable, our work is quick, and our quality is top notch.

Many Joomla administrators think that 127.0.0.1 is equivalent to localhost when it comes to MySQL connections, and they think that they can change the value of $host in their configuration.php file from localhost to 127.0.0.1 and the website will still work as it should. In reality, if they’re using cPanel, then the website will go down, and, in almost all the remaining cases, the website will slow down. Let us explain…

When you create a user in cPanel, such as myuser, cPanel automatically assigns that user to 3 hosts:

• localhost
• Server HostName (such as “host.itoctopus.com”)
• Server IP (such as “173.199.145.150”)

This means that the user will be able to connect to the database only when one of the above is set as the host in the configuration.php file. If you’re the skeptical type and want proof, then try the following:

• ssh to the WHM server as root.

• Type in mysql.

• Type in the following query:

  SELECT User, Host FROM mysql.user;

Once you do the above, you will see a list of all the MySQL users, along with their “Host”, on your database server, and you will notice that each user is added 3 times, once for every host. You will also notice that (with one exception, where “root” is the user) 127.0.0.1 doesn’t exist anywhere in the resultset of the query, which means that you can’t use 127.0.0.1 as your host and expect it to work.

Of course, the above mainly applies to a cPanel/WHM environment. If you’re using a more manual environment, or if you’re using root to connect to the database (which is never a good idea), then you may be able to use 127.0.0.1 as the host. However, the problem with doing that (in the absolute majority of cases), is that any communication with the database server will be through a TCP/IP connection, instead of a local socket. This incurs unnecessary networking overhead which will slow down your Joomla website.

We hope that you found this short post useful. If you are having problems with your Joomla website connecting to your database server, then please contact us. We are very quick to help, our work is ultra clean, and our fees are always competitive!

A somehow controversial subject in the database world is MySQL persistent connections: Some claim they have a performance benefit, others claim they’re bad and they can cause some memory issues on the server. At itoctopus, we have decided to put an end to the controversy by running experiments on a large Joomla website in order to prove either theory.

By default, Joomla sites use non-persistent connections to connect to the MySQL database. A non-persistent MySQL is automatically destroyed by PHP when the script is finished (that’s why there is no reason to explicitly close it). A persistent MySQL connection, on the other hand, is not destroyed at script end and is reused. This should mean that persistent MySQL connections should provide a performance boost to the Joomla website, as they eliminate the overhead of creating a new connection on each page execution (and destroying it when the page has finished loading). Right? Maybe, but let us first explain how to use MySQL persistent connections on a Joomla site before giving a definitive answer to the question.

How to Use MySQL Persistent Connections on a Joomla Site

In theory, switching from a non-persistent to a persistent MySQL connection is simple, all that one needs to do is to prefix the value of the host with a “p:” (without the quotes). So, on a standard Joomla website (where the database server co-exists with the web server), the host should be something like “p:localhost” instead of “localhost”.

However, things are always not that simple, as we saw the following error when we changed the value of $host in the main configuration.php file from “localhost” to “p:localhost”:

0 – Could not connect to MySQL.

This is because the function connect() which is located in the mysqli.php file (which, in turn, is located under the libraries/joomla/database/driver folder) essentially removes all the characters that are deemed “unnecessary” from the host, including the colon (“:”) through regular expressions. In fact, the colon is only useful when you want to explicitly specify the port that the MySQL database server listens to. So, if MySQL listens on the 7777 port, for example, then your host should be “localhost:7777”. Providing something like “p:localhost” will mean that the host is “p”, and the port is “localhost” (which, of course, is wrong).

We tried our best to “trick” the regular expressions into accepting our “p:localhost” host, but we failed. We then tried to override the mysqli_connect function, but we couldn’t, because overriding it meant using the override_function, which belongs to an obsolete PECL library that is no longer maintained (and possibly insecure). Overriding it using namespaces also wasn’t possible, since the Joomla application doesn’t have a global namespace that all the PHP files belong to.

So, the only viable method that we found to have a persistent database connection was to modify the core by changing the following line in the connect() function (which is located in the aforementioned mysqli.php file):

$this->connection = @mysqli_connect(
	$this->options['host'], $this->options['user'], $this->options['password'], null, $this->options['port'], $this->options['socket']
);

to:

$this->connection = @mysqli_connect(
	'p:localhost', $this->options['user'], $this->options['password'], null, $this->options['port'], $this->options['socket']
);

As you can see in the above, we hardcoded the host in the mysqli_connect function. We then loaded the website, and this time it worked!

So, does a persistent connection offer any performance advantage on Joomla sites?

Well, we tested the performance of a huge Joomla website (after switching its connection to persistent) on a staging server, and the numbers were more or less the same when using non-persistent connections. So, in short, and from our tests, persistent connections do not offer any performance advantage over non-persistent connections. So, there is no point in modifying the Joomla core to switch to a persistent connection.

We hope that you found this post useful. If you have any questions about it, or if you need help with optimizing your Joomla website, then please contact us. Our fees are affordable, our quality is top notch, and our fees are super affordable!

If you’re an avid reader of our blog, then you may already know that we have a passion for optimizing Joomla sites. A passion so strong, that we spend the absolute majority of our spare time researching new optimization techniques. More often than not, our research yields interesting results.

For example, about a week ago, we were studying the queries generated by an already optimized Joomla website to see how we can further optimize the website. We were doing that with the help of MySQL’s general_log server system variable (in the my.cnf file, which is located under the /etc folder), which, when set to “on”, will log all the generated queries to the file defined in the general_log_file server system variable. We did that on a dev server by adding the following lines to the beginning of the my.cnf file:

general_log = on
general_log_file=/var/lib/mysql/all-queries.log

The above lines logged every query issued on the MySQL server to the all-queries.log file. After doing the above, we loaded the website, and all the queries issued by the website were logged to the aforementioned file. After a few tests, we noticed something interesting… At the end of each page load, we saw the following line:

74008 Statistics

We did a little research and we discovered that the above line is generated by the stat() function on the mysqli static class. A quick search into the Joomla code revealed the location of the stat() method call. It was in the disconnnect method which is located in the mysqli.php file which, in turn, is located under the libraries/joomla/database/driver folder. It was in the following line:

if ($this->connection instanceof mysqli && $this->connection->stat() !== false)

Obviously, it was the $this->connection->stat() code in the above line that caused that Statistics line to appear in the general log.

Of course, you have 2 questions right now: What does the stat() method do and why should anyone care about the Statistics line?

To answer the first question, the stat() method on the mysqli static class is equivalent to the mysqli_stat function (which takes the connection link as parameters), which returns the output from the following shell command:

mysqladmin status

The output of the above command is something like the following:

Uptime: 2208760 Threads: 1 Questions: 157907223 Slow queries: 557 Opens: 251845 Flush tables: 1 Open tables: 1024 Queries per second avg: 71.491

In short, the output consists of the uptime (in seconds), the number of threads, the number of commands issued on the server (including queries), the number of slow queries, the number of tables that have been opened, the number of flushed tables, the number of tables currently open, and finally the average number of queries per second. All of that is unnecessary information – not only that, even closing the connection is also unnecessary, because Joomla does not use persistent connections. So, in short, the whole disconnect function is pure overhead, and, on large Joomla sites, it is wise to get rid of that overhead by simply adding the following line to very beginning of the disconnect function:

return;

So why should anyone care about the “Statistics” line?

Well, on small sites, nobody should really care. But, on large sites it is just (as mentioned above) an unnecessary overhead, and the website is better off without it (and without the overhead of closing the connection).

Is not closing the connections a good thing?

No, it is not. But, although we are no longer explicitly closing the connection, we’re not leaving it open either. Why? Because, by default, the connection is closed in Joomla by the function destruct which is located in the driver.php file (under the libraries/joomla/database), and the destruct function is called (once) when the script has finished execution. However, PHP automatically closes the MySQL connection when the script has ended, and thus, there is no need for Joomla to explicitly close the connection.

We hope that you found this post useful. If you want help optimizing the large Joomla website of your business, then please contact us. We are always eager and happy to help, our fees are super affordable, and our work is super clean.

We manage quite a few large Joomla sites, and all of these sites get their fair share of bad traffic caused by bad bots. What most system administrators do is that they block these bad bots with an .htaccess rule. At itoctopus, we go the extra mile and we punish those bots.

How do we do that? Well, with an .htaccess rule of course:

RewriteCond %{HTTP_USER_AGENT} ^.*(BadBot1|BadBot2|BadBot3|BadBot4|BadBot5).*$ [NC]
RewriteRule ^/?.*$ "http\:\/\/127\.0\.0\.1" [R,L]

So how does the above rule punish the bad bots?

The above rule checks if the user agent is BadBot1 or BadBot2 or BadBot3 or BadBot4 or BadBot5, and if it is, then it redirects the traffic to 127.0.0.1, which is the IP of the machine the request came from. This means that any attempt to index the website by the bad bot, will end up redirecting the bad bot to the web server on its own machine. This will confuse the bot (not to mention increase the load on the originating server) and will typically require intervention by the system administrator of the bad bot, who will manually blacklist the website from being indexed. A simple, funny, and yet very powerful rule! Don’t you agree?

So, how do you know which bad bots are visiting your Joomla website?

Before answering the question, it is important to explain that the term “bad bot” is subjective. What might be a bad bot for one website might be a good bot for another website. For example, for most of our US clients, we block the “Baidu” and the “Yandex” bots which are the bots of the top Chinese and the top Russian search engines, respectively. We know for sure that sites located in China, for example, would never ever think of blocking Baidu (this would technically mean killing their sites). So, again, the term “bad bot” is subjective.

Now that we have determined that the term “bad bot” is subjective, it is clearly up to you to decide which bots you want to block. But, in order to do that, you will need first a list of the top user agents visiting your website, which you can get using the following shell command (for an explanation about this command, check this post):

awk -F\" '{print $6}' /usr/local/apache/domlogs/yourjoomlawebsite.com | sort | uniq -c | sort -k1nr | awk '!($1="")' | sed -n 1,500p > user-agents.txt

The above command generates a list of the top 500 user agent strings visiting your Joomla website and stores them in the user-agents.txt file. Once you have that file (note that it might take a few minutes for the file to be generated), you can filter out the bots (unfortunately, there is no solid script to filter out the bots, so that must be done manually) and then you can decide which bots you need to block.

Note that if your website uses HTTPS instead of HTTP, then you should use the below command instead (it’s the same as the above, but with the exception that the domain has -ssl_log appended to its end):

awk -F\" '{print $6}' /usr/local/apache/domlogs/yourjoomlawebsite.com-ssl_log | sort | uniq -c | sort -k1nr | awk '!($1="")' | sed -n 1,500p > user-agents.txt

The above 2 commands apply to a WHM based environment. If you use Plesk or Webmin (or something else), then a good idea would be to check with your host for the location of your domain logs. Of course, you can always contact us for help and we’ll take care of those bad bots for you quickly, professionally, and affordably!

One of the (almost unique) features that Joomla has is “locking” articles. So, when someone is editing an article, that article is locked to prevent others from editing it. The lock is released when the article is “closed” by clicking on the Save & Close button, the Save as Copy button, or the Close button.

However, there is one case where the lock is not released when the article is closed, and it is when the server powering the Joomla website has some aggressive caching enabled. In fact, we are getting clients having this problem at least twice a month for a year now: they open an article, they edit it, they click on Save & Close, and then they see that the article is locked on the Article Manager page. The common denominator between clients having this problem is a cheap host.

So, how do we solve this “locking” problem?

In some cases, the problem is solved by simply disabling browser caching on the backend. In some other cases, the host sends us a few lines to add to the .htaccess file in order to disable their aggressive caching engine. In some rare cases, the host refuses to allow us to disable caching and we are forced to move the client to another (more professional) host.

Is the problem always a server problem?

So far, in every case that we have worked one, the “locking” problem was a server problem (and not an application problem). Having said that, it is always a good idea to disable the .htaccess file (by renaming it to .htaccess.old, for example) and see if that fixes the problem. If it doesn’t, then you should check (with your host) the cache settings of the server hosting your website and apply the appropriate remedy. If you need help fixing this problem, then please contact us. We are always eager to help, our fees are super affordable, and we always find a solution.

Almost all Joomla administrators do not think about disabling any core plugins that are enabled by default – which is a fine approach, but not when you’re running a large site. Some of these plugins have very little use for many Joomla sites and, as such, they are better off disabled. Take, for example, the “System – User Log” plugin.

In essence, what this plugin does is that it logs the date (and time), the IP, and the reason of a failed login to the Joomla site. The values are recorded in an error.php file which is located under the $log_path (the $log_path is defined in the configuration.php file). On a WHM/cPanel based environment, the plugin logs invalid logins to the /home/[cpanel-user]/public_html/logs/error.php file or to the /home/[cpanel-user]/public_html/administrator/logs/error.php file.

The question is: do Joomla sites really need that?

For the sites that we manage (at itoctopus), we disable the System – User Log plugin as we don’t need the overhead of loading a whole plugin for logging failed logins, and this is because we use .htpasswd protection for backend logins, and, for the frontend (if a website allows frontend logins), we can track invalid logins (should we really need to) through Apache logs. In the past 5 years, we never ran into a situation where we needed to track an invalid login.

So, is there any noticeable performance gain from disabling this plugin?

In most cases – the performance gain is negligible. For high traffic sites, however, disabling the plugin should be considered necessary by the system administrator who must explore any possible strategy to reduce the load on the underlying server.

Are there any other benefits of disabling this plugin?

Reducing disk space usage on the server is another benefit. Let us give you a real life example: a couple of days ago, a new client contacted us and told us that their (cheap) host is asking them to reduce their disk footprint on the server. So, we examined the filesystem of their Joomla site and we noticed that the error.php file is taking almost 14 GB of disk space (yes, that’s fourteen gigabytes). The file was littered with garbage about unsuccessful logins (likely caused by a dictionary attack). Not only this garbage was in this particular file, it was also backed up every night which was taking even more space on the server. We addressed the problem by 1) adding an .htpasswd protection to the backend, 2) disabling the System – User Log plugin, and 3) deleting the error.php file.

Are there any benefits of leaving the plugin enabled?

The only benefit that we can think of is the fact that a very large error.php file indicates persistent brute force attacks (which is the case of our client above), which, in turn, indicates the fact that you should consider implementing this ModSecurity rule on your server to block such attacks. If you need help with the deployment of this rule on your server, then don’t be shy, just contact us. Our fees are affordable, our work is clean, and we are the friendliest Joomla developers in this solar system.

After moving the website of a client from a shared host to a dedicated server (with another company), we got a call from the client telling us that she’s experiencing a weird situation.

She told us that when she tries to edit an article on the frontend of the Joomla site, it works the first time, but the second time she clicks on the “Edit” link, she sees the following error:

You are not permitted to use that link to directly access that page

We tried to reproduce the problem on our end, but we couldn’t. So, we asked her to use another browser, and so she used Google Chrome (she was using Safari), and she had the same issue: the first time the edit link worked, and the second time it didn’t.

So, we switched to Chrome (we were using Firefox), and tried to reproduce the problem on our end, and this time, we saw the same error that she was seeing. From our experience, when a problem occurs on one browser and not on another, then it is likely a URL caching problem. So, we disabled browser caching on her Joomla website by adding the following code to the very beginning of the .htaccess file:

<FilesMatch "\.(html|htm|php)>
FileETag None
Header unset ETag
Header set Cache-Control "max-age=0, no-cache, no-store, must-revalidate"
Header set Pragma "no-cache"
Header set Expires "Mon, 01 Jan 1990 00:00:00 GMT"
</FilesMatch>

We then tried editing the page on the fronted of the Joomla website, and this time it worked!

Why did the problem start happening when she was moved to the other server?

We are not exactly sure – it might be that there are some global browser caching settings in the httpd.conf file of the Apache web server, but we cannot confirm that.

What if adding the above code doesn’t work?

If the above code doesn’t work, then try disabling the System – Cache plugin and see if it helps. If it still doesn’t, then a good idea would be to analyze the server logs to see how the server is responding to the browser requests. You can also contact us and we’ll fix the problem for you swiftly, professionally, and for a very affordable fee!

Note: This post contains a core modification. Keep in mind that core modifications may cause stability issues and may be wiped out with a simple Joomla update.

A few months back, we’ve written about the Joomla getAuthors function, and how we initially thought it was there to populate the contents of the Author filter field, and how we eventually found out that the function had no purpose whatsoever and wasn’t even responsible for loading the authors.

Needless to say, we were a bit curious back then to know whether loading the authors (that are used for filtering the articles in the backend) used a similar query, but we just didn’t have the time to investigate further…

Fast forward to today, when we found the below slow query while checking the MySQL slow query log on the server of a high traffic Joomla site.

# Query_time: 1.455532  Lock_time: 0.000354 Rows_sent: 33  Rows_examined: 62925
SET timestamp=1521556512;
SELECT u.id AS value, u.name AS text
FROM #__users AS u
INNER JOIN #__content AS c ON c.created_by = u.id
GROUP BY u.id, u.name
ORDER BY u.name;

We immediately thought that the problem was caused by the getAuthors function (which is located in the articles.php file which, in turn, is located under the administrator/components/com_content/models folder), so, we disabled the function by adding a return array(); to its very beginning. Unfortunately, The problem wasn’t fixed as the query still appeared in the slow query log every few minutes.

So, we searched for the files containing the following string “u.id AS value, u.name AS text” in the filesystem of the Joomla site (e.g. under the /home/[cpanel-user]/public_html folder) using the following command:

grep -R 'u.id AS value, u.name AS text' *

And we found that the above query existed in the following 3 files:

1. administrator/components/com_content/models/articles.php
2. libraries/cms/html/list.php
3. libraries/src/Form/Field/AuthorField.php

The first file was clearly not the culprit since we just fixed it. The second one (the list.php file) didn’t have the exact query. So, we were left with the third one, where the query was an exact match.

So, we opened the file AuthorField.php which is located under the libraries/src/Form/Field folder. We scrolled down to the getOptions function, and we changed the following code:

$query = $db->getQuery(true)
	->select('u.id AS value, u.name AS text')
	->from('#__users AS u')
	->join('INNER', '#__content AS c ON c.created_by = u.id')
	->group('u.id, u.name')
	->order('u.name');

to the one below:

$query = $db->getQuery(true)
	->select('u.id AS value, u.name AS text')
	->from('#__users AS u')
	->order('u.name');

We saved the file and then we monitored the site for a few hours and we didn’t see an additional occurrence of the query in the slow query log. Hooray!

So, what is the “AuthorField.php” file?

The AuthorField.php file is the file responsible for displaying the author filter field on the Articles backend page. So you can safely say that the question that we asked ourselves a few months ago (whether the author filter field would cause a performance issue on the site or not) kinda forced us to find its answer.

But, why is the original query slow?

The original query is slow because it only returns the users that have written articles (e.g. the authors), so it does a join operation on the #__content table in order to do that, which is a very expensive operation especially if the #__content table has many articles. Ironically, the purpose of this query is to reduce the load on the browser and also reduce the memory needed to store the users, because some Joomla sites (especially social sites) have thousands of users, and, without this join, all of these users will be returned in the result set and also all of them will be pushed to the browser. That’s why it is not a good idea to implement the above solution on subscription based Joomla sites (please contact us for a customized solution).

We hope that you found our post useful and that implementing the above did speed up your large Joomla site. If you need help with the implementation, then please contact us. We are always eager to help, our fees are affordable, and our Joomla expertise is abundant.

Back in January 2011 (7 years and 1 month ago), people started migrating from Joomla to WordPress or Drupal in droves. The main reason for this mass exodus was the absence of a non-technical migration script from Joomla 1.5 to Joomla 1.6 which left many Joomla administrators with 3 options: 1) pay a lot of money to a professional to migrate their sites to the latest version of Joomla, or 2) migrate to another (more reliable) platform with the help of a professional, or 3) start from scratch on another platform.

What made things worse was the fact that the two subsequent Joomla releases, Joomla 1.6 and 1.7, were both unreliable platforms, and so even some of the people who migrated their sites switched to another platform, as it seemed, back then, that Joomla was self-imploding.

In February of 2012, Joomla 2.5 was released, and while it addressed the stability issues found in Joomla 1.6 and 1.7, it still didn’t solve the major migration dilemma from 1.5 to 2.5. So an additional batch of Joomla administrators left the platform.

At the end of 2012, another major event happened in Joomla, and it was the release of Joomla 3.x (which we are still using today). The migration of Joomla 2.5 to Joomla 3.x was, in most case, a non-seamless experience, and so more people (who couldn’t take it anymore), left the platform to another one.

At around the same time, there were many internal strifes within the Joomla community itself, projecting an impression that Joomla’s future was far from secure. In fact, some serious concerns were voiced back then that Joomla was heading down the Mambo road – in other words, it was going to die and something else was going to be reborn from its ashes. Many enterprise sites using Joomla dumped the platform at that point because nobody wanted to stay in what seemed to be a sinking ship.

But, for some mysterious reasons, Joomla survived (after reaching its lowest point in 2014), albeit with a smaller userbase. The very stable Joomla 3.5 was released, and then Joomla 3.6, Joomla 3.7, and finally Joomla 3.8. Each Joomla iteration since the end of 2014 was better and more secure than its predecessors. There were some exceptions, but these exceptions were immediately addressed and a patched version of Joomla was released typically within a week of the botched version. Never was Joomla that dynamic, and that dynamism has ensured that existing Joomla users were retained, and new ones were acquired.

So what are the factors that lead to Joomla’s revival?

There are several factors that revived Joomla and made it regain its users’ trust:

• Listening to the mainstream Joomla community: In the early days of Joomla, only opinions from important people in the Joomla ecosystem had impact on the actual coding of the CMS. This is no longer the case today. For example, the very active Joomla forum has a place to discuss each Joomla version when it’s released, and any bug reported by anyone is taken very seriously and is addressed almost immediately.

• Genuine commitment from the core developers to the product: Unlike WordPress and Drupal, Joomla’s core developers are paid zilch for their contributions, so one would reasonably expect very little commitment from Joomla’s core developers to the product. This is not the case, in fact, the lead Joomla core developer seems to have assumed ownership of the product, which is a good thing, as that same developer has lead Joomla from the brink of extinction to the solid and reliable product that it is today.

• Sudden (and welcome) attention to performance: A few years ago, not a single core Joomla developer cared about the performance, and the general assumption was that Joomla was mainly used by small websites. We know, for a fact, that Joomla powers some very high traffic websites, and we have complained for ages that Joomla needs some serious optimizations in the core. Our pleads went unheeded, until recently, when Joomla’s core developers started taking performance more seriously. In fact, the negligible performance hit resulting from updating from Joomla 3.7 to Joomla 3.8 (there is a huge technical difference between the 2 versions) is an indication that Joomla is finally on the right track when it comes to performance.

• More robust security: When was the last time you heard that Joomla had a serious security flaw (other than a minor XSS issue)? Compare that to a few years ago, when security flaws from legacy code were abundant.

• Less bugs: While we still think that the testing of the Joomla product is mediocre at best, we have to admit that the number of known bugs is decreasing with each Joomla iteration, and the days where a Joomla iteration is specifically released to fix the problems caused by the previous iteration of Joomla are almost behind us.

• Quick adoption of latest technologies: As of 2015 (possibly even before), Joomla has been a pioneer in adopting new technologies very quickly. The competition, such as WordPress and Drupal, are very slow in that aspect. In fact, neither WordPress nor Drupal have adopted MVC yet, which has been adopted by Joomla since version 1.5.0 (which was released in January of 2008, that’s 10 years ago!).

• Seamless updates: Very few people know that the update from Joomla 3.7 to 3.8 is technically a migration, and not just a normal update. With some exceptions, that update went super smoothly for Joomla administrators. Who would have thought that Joomla, the same Joomla with the messy 1.5 to 2.5 and 2.5 to 3.x migrations, was finally able to do that?

There are many other factors such as cleaner code, cleaner interface, better code management, etc… that indicate that Joomla has indeed reached maturity, and is now, again, a very stable and a very trustworthy CMS that people can rely on for many years to come to power their websites!

Back in August 2014, Google stated on their official webmasters blog that their ranking algorithm will start giving an advantage to HTTPS sites over HTTP sites. Back then, many people simply did not care, and did not switch their sites to use HTTPS. However, in the beginning of the last autumn (“the beginning of the last autumn” should be the title of a book), Google raised their tone significantly, insisting that all pages that have forms must use HTTPS, or else their browser (Google Chrome, which has more than 50% of the current market share) will display a big warning for the visitors (since content transmitted in HTTP mode will be transmitted in clear text, unlike content that is transmitted in HTTPS, which is transmitted in encrypted format) as of October 2017. This, of course, will increase the bounce rate, which will negatively affect the SEO rankings of the site. Naturally, this decision by Google has forced most businesses who are serious about their websites to switch from HTTP to HTTPS. Owners of small Joomla sites had very little work to do: all they needed was to install an SSL certificate (which is provided for free for WHM accounts), and then Force HTTPS on their entire site in the Joomla backend. For large Joomla sites, the process is more complex, and that’s why we, at itoctopus, decided to share our experience on how we switched a large Joomla website from HTTP to HTTPS.

Here’s the process from A to Z:

• We installed the SSL certificate.

  Since the client was using WHM, we installed the SSL certificate that was provided for free with a WHM account. Here’s how:

  • We logged in to WHM as root.
  • We clicked on Manage AutoSSL under SSL/TLS on the left panel.
  • Under AutoSSL Providers, we clicked on the radio button next to cPanel (powered by Comodo).
  • We clicked on the Options tab, and then we clicked on all the checkboxes there.
  • We clicked on the Manage Users, and then we clicked on the Enable AutoSSL for the cPanel user.
  • Finally, we clicked on Run AutoSSL For All Users at the very top.

  We waited a few minutes, and then, it happened! AutoSSL generated a cPanel-Comodo branded SSL certificate for the website. Yoohoo!

• We checked if the Joomla website had any HTTPS redirect plugins.

  In our case, we didn’t find any HTTPS redirect plugins. But, if we did find such plugins, we would have disabled them immediately (not doing that may cause an infinite loop). We also disabled the Joomla Redirect Manager extension since the client didn’t even know how it worked (and also didn’t need it in the first place).

• We updated all the internal links in the #__content table to use HTTPS:

  • We logged in to phpMyAdmin.

  • We selected the database powering the Joomla website, and then we issued the following queries (note that we replaced #__ with the table alias of the Joomla website [you can find the table alias in the configuration.php file]):

    UPDATE `#__content` SET `introtext`= REPLACE(`introtext`, 'http://www.ourclientjoomlawebsite.', 'https://www.ourclientjoomlawebsite.') WHERE `introtext` LIKE '%http://www.ourclientjoomlawebsite.%';
    UPDATE `#__content` SET `introtext`= REPLACE(`introtext`, 'http://ourclientjoomlawebsite.', 'https://www.ourclientjoomlawebsite.') WHERE `introtext` LIKE '%http://ourclientjoomlawebsite.%';
    UPDATE `#__content` SET `fulltext`= REPLACE(`fulltext`, 'http://www.ourclientjoomlawebsite.', 'https://www.ourclientjoomlawebsite.') WHERE `fulltext` LIKE '%http://www.ourclientjoomlawebsite.%';
    UPDATE `#__content` SET `fulltext`= REPLACE(`fulltext`, 'http://ourclientjoomlawebsite.', 'https://www.ourclientjoomlawebsite.') WHERE `fulltext` LIKE '%http://ourclientjoomlawebsite.%';

  Let us explain the 4 queries above a bit. The first query and the third query replace every instance of http://www.ourclientjoomlawebsite. with https://www.ourclientjoomlawebsite. in the introtext and the fulltext fields respectively. The second query and the fourth query replace every instance of http://ourclientjoomlawebsite. with https://www.ourclientjoomlawebsite. in the introtext and the fulltext fields respectively. The reason why we did that is because some links in the database are http://ourclientjoomlawebsite. instead of http://www.ourclientjoomlawebsite.. Note that if your website uses non-www links instead of www links, then you will need to remove every instance of www. from the links above, and add www. to the links not containing www.. On a side note, we prefer using www, since Google themselves use www.

  If you are using K2, then all you need to is to replace `#__content` with `#__k2_items` in the above queries.

• We updated all the internal links in the #__menu table to use HTTPS.

  Essentially, the below queries were used to address menu items of type URL with hardcoded internal links.

  UPDATE `#__menu` SET `link`= REPLACE(`link`, 'http://www.ourclientjoomlawebsite.', 'https://www.ourclientjoomlawebsite.') WHERE `link` LIKE '%http://www.ourclientjoomlawebsite.%';
  UPDATE `#__menu` SET `link`= REPLACE(`link`, 'http://ourclientjoomlawebsite.', 'https://www.ourclientjoomlawebsite.') WHERE `link` LIKE '%http://ourclientjoomlawebsite.%';

• We updated all the internal links in the #__modules table to use HTTPS.

  This step should not be ignored as typically there are quite a few modules (especially modules of type Custom) that have hardcoded links pointing to the Joomla website. Here are the queries that we used:

  UPDATE `#__modules` SET `content`= REPLACE(`content`, 'http://www.ourclientjoomlawebsite.', 'https://www.ourclientjoomlawebsite.') WHERE `content` LIKE '%http://www.ourclientjoomlawebsite.%';
  UPDATE `#__modules` SET `content`= REPLACE(`content`, 'http://ourclientjoomlawebsite.', 'https://www.ourclientjoomlawebsite.') WHERE `content` LIKE '%http://ourclientjoomlawebsite.%';

• We did a search on the whole database for any remaining entries containing http://www.ourclientjoomlawebsite. (or http://ourclientjoomlawebsite.).

  We did that the following way:

  • We clicked on the “Search” button after selecting the database in phpMyAdmin.

  • We entered the search keyword, which is %http://www.ourclientjoomlawebsite.% in the search box.

  • We clicked on the Exact Phrase radio button and then we clicked on Select All to select all the tables.

  • We clicked on the Go button, and then we fixed every item that contained a hardcoded link.

  • We repeated the above process for %http://ourclientjoomlawebsite.%.

• We searched through the filesystem for any hardcoded links the following way:

  • We logged in to the Joomla backend and then we cleared the cache. This step is important as it reduces the time it takes to search the whole folder – it is also necessary, as we just changed all the internal links in the database from HTTP to HTTPS, and we don’t want the search to return some false positives from the cache.

  • We ssh’d to the server.

  • We cd’d to the following directory /home/[cpanel-user]/public_html/:

    cd /home/[cpanel-user]/public_html

  • We ran a grep searching for any internal HTTP links in the filesystem:

    grep -R 'http://www.ourclientjoomlawebsite.' *

  • We fixed any file (including JS and CSS files) that has harcoded HTTP link(s).

  • We cleared the Joomla cache again.

  • We repeated the process for ‘http://ourclientjoomlawebsite.’.

• We instructed the Joomla website to use HTTPS.

  • We went to System -> Global Configuration -> Server in the Joomla backend.

  • We changed the value of Force HTTPS under the Server Settings section to Entire Site.

  • We clicked on Save on the top left and then we cleared the Joomla cache.

• We added an HTTP to HTTPS redirect in the .htaccess file.

  Now while Joomla automatically redirects from HTTP to HTTPS after doing the above step, it is better to let Apache handle such redirection, since the redirection will be much faster (the Joomla engine doesn’t need to be loaded to redirect the traffic). Here’s the code that we added in the .htaccess file in order to do this:

  RewriteCond %{HTTPS} off
  RewriteRule (.*) https://www.ourclientjoomlawebsite.com/$1 [R=301,L]

• We removed the value of $live_site from the configuration.php file.

  This step is important because some backend activities (such as updates) will become unstable if $live_site is set to the HTTP version of the website.

• We fixed the code of the social sharing plugins to use HTTPS instead of HTTP.

  We edited the code of the social sharing plugin and ensured that it was referencing the HTTPS version of the website, and not the HTTP one. Of course, the question is, what about the counters? For example, what if the HTTP version of a link was shared 100 times on Facebook, will that number be preserved after switching to HTTPS? The answer is yes, the number is preserved. Social websites consider the HTTPS and the HTTP version of a link as just one page (unlike Google). In any case, among all 3 major social websites (Facebook, LinkedIn, and Twitter), only Facebook now displays the number of shares. Twitter dropped the counter last year, and LinkedIn dropped it early this month (February 2018).

• We tested the website thoroughly.

  We cleared the Joomla cache and then we tested the website thoroughly, and ensured that all the links still work and are loaded through HTTPS. We also ensured that there were no “mixed content” errors in the console.

• We added the HTTPS version of the website in Google’s Search Console.

  For some odd reason, Google’s search engine (and Google’s search console) still considers the HTTP version of the website and the HTTPS version of that same website to be different websites, which means that we had to re-add (and re-verify) the HTTPS version of the website to Google’s Search Console (formerly Google’s Webmaster Tools). Note: There is no need to do any action in the Google Analytics account when switching from HTTP to HTTPS.

• That’s it! That was the whole process!

We hope that you found our lengthy guide useful for the HTTPS transition of your Joomla website. If you need help with the implementation, then look no further. All you need to do is to contact us and we’ll take care of the whole thing swiftly, professionally, and cleanly. Please note that our fees apply.

A new client called us yesterday and told us that the website of the company he works for is down. He told us that it was displaying a weird error.

We immediately visited the website, and we noticed that it was displaying the below error (in red):

“1881 – Operation not allowed when innodb_forced_recovery > 0.” Error on Joomla Site

Luckily, we knew what innodb_forced_recovery is as we have used it before: it’s a MySQL setting in the my.cnf file (the MySQL configuration) and it is used to recover a MySQL database using the InnoDB storage setting after a crash.

We thought, could it be that the MySQL database crashed, and someone tried to recover the database, and it didn’t recover properly so he forgot the innodb_forced_recovery setting in the my.cnf file? So, we logged in to to phpMyAdmin and, to our pleasant surprise, the database was working and all the tables were OK. So, the prevailing theory here is that the database did crash, someone tried to recover the database, the recovery worked, but that person didn’t even test the website possibly because he thought that the recovery failed because it was taking a lot of time (and, as such, forgot to remove innodb_forced_recovery setting). Now, all we had to do was to comment out (or remove) the innodb_forced_recovery setting from the my.cnf file. We did this the following way:

• We ssh’d to the server as root.

• We opened, using vi, the file my.cnf which is located under the /etc folder.

• We commented out the following line by adding a hash (#) to its beginning (note: you can also remove the line altogether):

  innodb_force_recovery = 2

• We restarted MySQL by issuing the following command:

  service mysql restart

• We checked the website and this time it worked! Yoohoo!

Now, if you’re the curious type, then you might be wondering what it means when innodb_force_recovery has a value of 2, and whether it can have different values. In short, the innodb_force_recovery can have values ranging from 0 to 6.

If innodb_force_recovery = 0, then recovery is disabled (this means normal operation).

If innodb_force_recovery = 1, then recovery is enabled, but it’s the most basic recovery and should be used when only little corruption has occurred to the database.

The higher the value of the innodb_force_recovery, the more aggressive (and dangerous) the recovery is. A value of 6 should only be used when all other options were exhausted, this is because it can make the problem even worse by permanently and irreversibly corrupting the database. You can read more about innodb_force_recovery = 1 on MySQL’s official documentation (note: external link).

We hope that you found this short post useful. If you are having the same problem and if you need help implementing the solution, then please contact us. Our fees are right, our work is professional, we are based in the lovely city of Montreal, and we are Joomla experts!

A regular client who subcontracts work to us emailed us that he wasn’t able to update the website of one of his clients from Joomla 3.8.3 to Joomla 3.8.5. He told us that the update would fail halfway through with the following error:

AJAX Loading Error: error

Of course, the first thing that we thought was a blocked restore.php file, despite the fact that the error was a bit different. So, we checked whether there was a rule in the .htaccess file blocking direct execution of PHP files other than the index.php file, but, not only we didn’t find any blocking rule, we didn’t find any .htaccess file at all. Additionally, all the permissions on the restore.php file were correct as we were able to directly access the file from our browser by using the following URL: http://www.[ourclientjoomlawebsite].com/administrator/components/com_joomlaupdate/restore.php (it was returning the following json ###{“status”:false,”message”:”Invalid login”}###).

So, we tried to update the website ourselves (after backing up the filesystem and the database) and, at first, everything went smoothly. However, at exactly 56% of the update process, the progress stopped. About 30 seconds later, we saw the same error that our client was seeing. Not only that, we were blocked from accessing the website for about 5 minutes (we checked the website through a proxy and it was working there). Hmmm!

We were a bit suspicious of this whole 5 minute block thing, but, at first, we didn’t think of it much as we assumed it was just a coincidence. So, we did a research on the issue (yes, sometimes we rely on other people’s answers to the problem), and some were stating that the presence of a $live_site value in the configuration.php file was causing the same problem for them. So, we checked the configuration.php file under the main directory of the Joomla website, and, to our disappointment, the $live_site value was already empty. Others claimed that clearing the cache may solve the problem, so, we cleared the Joomla cache (there was about 1 GB of cached files, which was quite a lot, even more so when you take into consideration that the website was extremely simplistic), and then we tried the update again, but it didn’t work. Not only it didn’t work, but we were also blocked again. Double hmmm!

Once we were unblocked, we tried the update using Google Chrome while having the console open (you can open the console in Google Chrome by pressing F12), and we saw the following messages:

update.js?73ddd8fdd62da3333508a24283f9db89:171 XHR finished loading: POST "https://www.[ourclientjoomlawebsite].com/administrator/components/com_joomlaupdate/restore.php".
update.js?73ddd8fdd62da3333508a24283f9db89:171 POST https://www.[ourclientjoomlawebsite].com/administrator/components/com_joomlaupdate/restore.php net::ERR_CONNECTION_TIMED_OUT
update.js?73ddd8fdd62da3333508a24283f9db89:171 XHR failed loading: POST "https://www.[ourclientjoomlawebsite].com/administrator/components/com_joomlaupdate/restore.php".

Now, if you read the second line a bit more closely, you will notice that the browser is complaining about timing out while posting to the restore.php file. We immediately thought of 2 things that may have been causing the issue:

• A small value of the max_execution_time PHP setting.
• A weird rule triggered by an application firewall, such as ModSecurity.

We were able to immediately dismiss the first issue above as the Joomla backend was reporting 300 seconds as the value of the max_execution_time. Still, just to be 100% sure, we added a .user.ini file under the main directory of the Joomla website and then we added to that file the following line:

max_execution_time = 300

We tried the update again (just in case), and, unsurprisingly, it failed.

So that left us with the firewall theory, which made the most sense, since our IP was blocked temporarily every time we tried to update the site, which was a huge sign of a firewall’s foul play. Unfortunately, we were not able to confirm that theory because we didn’t have root access to the server, and neither did the client (and nor the client’s client, for that matter). So, we communicated our findings to the client, and asked him to route these findings to the host. The host was immediately responsive, and stated that a security module called OSSEC (which we have never heard of before) caused this issue. They then whitelisted the client’s IP and he was able to finally perform the update. Hooray!

So, our dear, dear reader, if you are seeing an AJAX Loading Error: error popup when you are trying to update your Joomla website, then check your max_execution_time value in PHP’s settings. If it’s too low, then increase it, and see if the problem is solved. If not, then clear your Joomla cache and make sure that $live_site has an empty value in the configuration.php. If that doesn’t solve the problem, then check your server for any security rules being triggered. If that also fails to solve the problem, then you can always contact us. We will fix the problem for you quickly, cleanly, and affordably!

Around 10 AM EST yesterday morning, we got a call from a large company telling us that their Joomla website is down as none of their staff members are able to access it. We checked and we were able to access the website. The first thing that came into our minds was that their company IP was blocked by the firewall. So, we logged in to WHM, and we flushed all the blocks (we clicked on ConfigServer Security & Firewall under Plugins on the left menu, and then we clicked on Flush all Blocks) . Once we did that, we emailed them asking them to confirm that the problem was resolved, and they replied back with a “Yes”.

Around 10:30 AM EST, they called back and told us that the problem happened again. Clearly, we needed to get to the bottom of the problem (we can’t keep flushing their blocks every 30 minutes forever). From our experience, there are 3 things that can cause this problem:

1. Logging in to the server with the wrong SFTP credentials: If you login to SFTP several times from the same IP, then you will get blocked (usually after 5 unsuccessful logins).

2. Logging with the wrong set of credentials on an .htpasswd protected page: If you have an .htpasswd protected page, and you login with the wrong set of credentials for 5 times, then you will get blocked by the firewall.

3. Repeatedly triggering a ModSecurity rule: ModSecurity, the web application firewall, is very picky about the incoming requests to the web server and blocks quite a few false positives. It’s even more picky when additional, custom rules are added to its engine. For example, in the case of one client, a custom ModSecurity rule added by the host blocked OpenOffice users because they were sending some wrong headers to the server.

A quick investigation in the logs revealed that the issue was caused by none of the above, but by something else. The following line in the lfd.log file (lfd stands for login failure daemon) which is located under the /var/log folder provided us with a pointer as to what the actual cause of the issue was:

Feb 12 08:51:33 [server] lfd[28239]: (smtpauth) Failed SMTP AUTH login from [ip-address] (US/United States/[...]): 5 in the last 3600 secs - *Blocked in csf* [LF_SMTPAUTH]

The above line, in case you’re wondering, is telling us that the IP of the client ([ip-address]) was blocked because it had 5 failed SMTP authentications in one hour. Now, of course, the question is, which email address is causing this?

After examining the exim log files, we discovered that the following email was the cause of the problem: noreply@ourclientjoomlawebsite.com. We quickly informed the client about our findings, and they took care of the issue on their end immediately. We then flushed the CSF blocks again and the issue was finally resolved.

If your company IP is blocked from accessing your Joomla website, then try flushing the CSF blocks in WHM. If that doesn’t fix the problem permanently, then check your logs for the cause of the issue in order to address it. If you need help doing that, then please contact us. We will know what’s causing the block, we will fix it, and your company won’t go bankrupt paying our fees.

Increasingly, we are getting requests from clients asking us to increase the Google PageSpeed Insights (PSI) score of their Joomla sites. Some of them even ask us to get them a score of a 100, which can only be possible if the Joomla website is an extremely basic one.

We usually respond that any score of about 80 (or even 75) is a good score, and that while a very high PSI score may provide a very slight boost to the SEO rankings of the site, it is usually not worth it, because, at best, that score will only be temporary, and, at worst, the website’s will become a complete mess. Let us explain…

First, Google PageSpeed Insights has some really unrealistic expectations when it comes to the page response time. We have seen the tool complaining about a 240 millisecond response time as too slow and asking us to “reduce it”. In our opinion and in the opinion of many, such a response time is excellent. So, administrators with slow Joomla sites (according to PSI) resort to using all kinds of caching tools (including Joomla’s own System – Cache plugin, possibly the most horrible caching plugin ever) to provide a quick remedy to the problem. This typically results in weird issues, such as layout issues, broken functionality, etc…

Second, PSI insists on having only the extremely necessary CSS and JS code loaded “above-the-fold”. “Above-the-fold” means being loaded in the head tag (before the page is rendered), and this requirement is there to ensure that the HTML content loads even if there are problems loading the CSS/JS files. This concern is valid for 2 reasons: 1) if the dependencies (the CSS and JS files) are loaded externally and there are connection problems with the remote site(s), and 2) if the SYN_FLOOD firewall value is set to a very small value, preventing the quick loading of assets. Joomla administrators typically install system plugins that will add defer or async to various assets being loaded – the problem is, however, that in many cases, these extensions result in a hard-to-fix mess.

Third, PSI expects the site administrator to have control over external websites. For example, if the website has AdSense ads on it, then PSI insists that the website must “leverage browser caching” on the AdSense assets (images, CSS, JS), which is not possible. This makes the lives of Joomla administrators extremely hard, especially those having many JavaScript tracking snippets and widgets on their Joomla sites. These administrators are typically left with the not-so-pleasant task of giving the marketing department (or the site owner) a choice: Higher PSI score or JS tracking snippets/widgets, but not both.

Fourth, even if a Joomla website is lucky enough to reach a high PageSpeed Insights score, then that score will gradually decrease with time, this is because almost every install and every update of an extension will negatively affect the score, and because new images that will be uploaded (and possibly processed by an extension, such as the images that are uploaded in K2) will most likely be unoptimized.

Fifth, it is really absurd to meet the technical standards that Google itself is not meeting in most of its products. Take a look at YouTube’s score for example, do you see a 100? How about Gmail’s score? How about that of Alphabet, Google’s parent company? None of these have perfect score, and one of them (Gmail) has a really, really low one.

So, should people ignore Google PageSpeed Insights?

Not at all – it does promote some best practices that are excellent to implement. However, it is important not to get obsessed with having a high score. A moderate to good score of 75-80 is more than enough.

PSI is a good tool, but it is ideal and far from realistic. With its idealism, Google’s PSI is inadvertently promoting some bad practices on the Internet, and is ironically standing against the #1 recommendation that Google has for the webmasters: make websites for humans, and not for (ro)bots. Because of PSI’s requirements, some webmasters are now doing the opposite.

Now, if you, our dear reader, are having problems raising the Google PageSpeed Insights score of your Joomla website to an acceptable level, then all you need to do is to contact us. We are always there for you, our work is professional, our attitude is right, and our prices are affordable.

Note: The solution in this post is a core modification. Proceed with caution.

Another note: This post has been modified on January 31^st, 2018 to correct an error. The post has originally been published the day before, on January 30^th, 2018.

While analyzing the performance of the backend of a Joomla 3.8 website that has over 200K articles, we noticed something interesting; we noticed that the following query was taking some time:

SELECT u.id AS value, u.name AS text
FROM #__users AS u
INNER JOIN #__content AS c ON c.created_by = u.id
GROUP BY u.id, u.name
ORDER BY u.name;

The above query generates a list of all the users who have created at least one article. It was taking about 14 seconds and it was examining about 500,000 rows. Tracing the query, we discovered that it was triggered by the getAuthors function which is located in the articles.php file (which is in turn located under the administrator/components/com_content/models folder).

So, which page triggers the “getAuthors” function?

Mainly it is the Articles page (under Content -> Articles) that triggers the getAuthors function. At first, we thought that it does that to populate the Select Author filtering dropdown (under the Search Tools), so, when someone is searching for an author, they won’t have to search through the users that don’t have any articles published. However, on closer look, we discovered that this wasn’t the case. The function getAuthors had no use, it was just loaded by the administrator articles view (the file articles.php which is located under the folder administrator/components/com_content/views) by the following line:

$this->authors = $this->get('Authors');

Now we searched the whole Joomla website for a single usage of this function, but we couldn’t find any. Yes, the function was utterly useless. So, we just added return array(); to the very beginning of the getAuthors function and we got rid of that heavy and unnecessary query.

We hope that you enjoyed this light post and that you found it useful. If you need help with the implementation, then just contact us. Our fees are right, our work is clean, and we are experts in optimizing Joomla websites.

A client of ours uses a Joomla article for listing internal announcements (the Joomla website is used as an intranet). The Joomla article contains announcements from 2005 and, each time the company has a new announcement (which literally happens at least one time every business day), that announcement is added to the very beginning of the article. Needless to say that with time, the article became very very large, but still this process worked fine for the client all of these years. But, last Friday, something happened: the client tried to add a new announcement, and, to their surprise, what has been working for 13 years suddenly broke, and the Joomla site displayed a 404 error. The client immediately emailed us for help and we promptly started the investigation.

The first thing that we tried to do was to recreate the problem, which wasn’t really hard: we logged in to the backend of the Joomla website, we opened the “Announcements” article, we added the new announcement (which was sent to us by the client), we saved the article, and we saw the 404 error. At first glance we thought it was something that had to do with the text that the client was trying to add (maybe a pattern was triggering a ModSecurity rule), but it wasn’t, because we changed the text to something entirely different, and we still had the same issue.

So, we did a tail on Apache’s error log (for that particular website, as the client had 2 other websites on the same server) in order to have a better idea what went wrong:

grep '[our-client-joomla-website].com' /usr/local/apache/logs/error_log | tail -500

And here’s what we saw:

[Fri Jan 26 09:39:58.597094 2018] [:error] [pid 4079] [client [our-ip]:38805] [client [our-ip]] ModSecurity: Request body no files data length is larger than the configured limit (1048576).. Deny with code (413) [hostname "www.[our-client-joomla-website].com"] [uri "/administrator/index.php"] [unique_id "WmegrFt7NF@XqHcDe6IAywAAABw"], referer: https://www.[our-client-joomla-website].com/administrator/index.php?option=com_content&view=article&layout=edit&id=82

So, it was ModSecurity (which is nearly always the culprit when a weird issue suddenly happens), but it wasn’t anything in the pattern of the text, it was the length of the text. The no files data length (e.g. the HTML length) is larger than the allowed maximum, which is 1 MB. But where is that set?

A quick research revealed that this limit was defined by the ModSecurity global directive SecRequestBodyNoFilesLimit, which defaults to 1 MB (or 1024 kilobytes, or 1048576 bytes). So, we resolved the problem by increasing the limit to 2 MB (which should give our client another 13 years). We did that the following way:

• We opened the file custom.conf file which is located under the /etc/apache2/conf.d folder.

• We added the following line to it (the file was originally empty):

  SecRequestBodyNoFilesLimit 2097152

• We saved the file and then we restarted Apache.

• We logged in to the Joomla backend and tried saving the “Announcements” article again after adding the new content.

• It worked!

Note that the location and the name of the custom ModSecurity rule file may not be the same on your server. So, you will need to make sure of the right location before implementing the solution (or else the solution will be useless).

But, does the above solution compromise security?

Well, not really, but it not ideal either… By limiting the maximum size of non-file content to 1 MB, ModSecurity lessens the impact of a DoS (Denial of Service) attack on your server. Increasing that limit to 2 MB is not bad, but it is not as good (security wise) to having that limit set to 1 MB. So, preferably, you should consider splitting a very large article into smaller articles instead of increasing the SecRequestBodyNoFilesLimit value.

So, the next time you see a 404 error when saving a large article on your Joomla site, investigate the Apache error logs, as it might be ModSecurity. If you need help fixing the problem, then please contact us, we are always ready to help, our work is ultra clean, and our our fees are super right.

Note: This post is advanced and is mainly aimed at system administrators, but any Joomla administrator can still read it and understand the main points as we have simplified it as much as we can.

One of our customers constantly emailed us about people getting randomly blocked from accessing their Joomla website. At first, our response was as simple as manually unblocking the blocked IPs in CSF (ConfigServer Security & Firewall) as we thought it was just something that they mistakenly did on the website that caused them to get blocked. However, the situation became more of an issue when some of these people became blocked on a daily basis (and, in some cases, on an hourly basis). We needed to investigate the issue…

So, we checked the modsec_audit.log file (which is the ModSecurity log file) that is located under the /usr/local/apache/logs folder for one of the blocked IPs and here’s what we saw:

--c39ced72-A--
[31/Dec/2017:13:05:01 --0800] WklQ-dEvE6fYw87vIOCzMwAAAVU [offending-ip] 54353 [server-ip] 443
--c39ced72-B--
PROPFIND /media/kunena/emoticons/smile.png HTTP/1.1
Host: our-client-joomla-website.com
User-Agent: Apache OpenOffice/4.1.3
Accept-Encoding: gzip
Depth: 0
Content-Type: application/xml
Transfer-Encoding: chunked

--c39ced72-C--
<?xml version="1.0" encoding="utf-8"?><propfind xmlns="DAV:"><prop><resourcetype xmlnx="DAV:"/><IsReadOnly xmlnx="http://ucb.openoffice.org/dav/props/"/><getcontenttype xmlnx="DAV:"/><supportedlock xmlnx="DAV:"/><lockdiscovery xmlnx="DAV:"/></prop></propfind>
--c39ced72-F--
HTTP/1.1 500 Internal Server Error
Content-Length: 679
Connection: close
Content-Type: text/html; charset=iso-8859-1

--c39ced72-H--
Message: Access denied with code 500 (phase 2). Match of "rx ^$" against "REQUEST_HEADERS:Transfer-Encoding" required. [file "/etc/apache2/conf.d/modsec2.liquidweb.conf"] [line "183"] [id "340004"] [rev "1"] [msg "Dis-allowed Transfer Encoding"] [severity "CRITICAL"]
Apache-Error: [file "apache2_util.c"] [line 271] [level 3] [client [offending-ip]] ModSecurity: Access denied with code 500 (phase 2). Match of "rx ^$" against "REQUEST_HEADERS:Transfer-Encoding" required. [file "/etc/apache2/conf.d/modsec2.liquidweb.conf"] [line "183"] [id "340004"] [rev "1"] [msg "Dis-allowed Transfer Encoding"] [severity "CRITICAL"] [hostname "our-client-joomla-website.com"] [uri "/media/kunena/emoticons/smile.png"] [unique_id "WklQ-dEvE6fYw87vIOCzMwAAAVU"]
Action: Intercepted (phase 2)
Stopwatch: 1514754301176673 671 (- - -)
Stopwatch2: 1514754301176673 671; combined=117, p1=35, p2=82, p3=0, p4=0, p5=0, sr=12, sw=0, l=0, gc=0
Producer: ModSecurity for Apache/2.9.2 (http://www.modsecurity.org/).
Server: Apache
Engine-Mode: "ENABLED"
--c39ced72-Z--

If you look closely at the first paragraph, you will notice that the whole cycle of events started with a request from OpenOffice to the smile.png emoticon in a Kunena forum. The request was innocent, looked innocent, but it didn’t act as innocent: the Transfer Encoding value sent by OpenOffice was not allowed, and, as such, it was blocked by a custom ModSecurity rule developed by LiquidWeb. Looking at the file modsec2.liquidweb.conf which contained the blocking rule, we found the following:

# Don't accept transfer encodings we know we don't handle
# (and you don't need it anyway)
SecRule HTTP_Transfer-Encoding "!^$" "id:340004,rev:1,severity:2,msg:'Dis-allowed Transfer Encoding'"

The above lines define the rule 340004 which blocks unknown transfer encodings. So, a quick solution to the problem was to remove the above rule (or comment it out with a #) from the modsec2.liquidweb.conf file and then restart Apache, and this is what we did. However, it wasn’t the ideal solution, since that rule was there for a reason, and it was additional protection. Then again, it was happening so often to the members of the site that it threatened the very existence of the business.

But, why did OpenOffice try to load the image?

An analysis to the problem, as well as statements from members, revealed that the problem started with the affected members copying content from the forum and then pasting that content into an OpenOffice document. That content contained emoticons (images), which only had their references copied, and so OpenOffice tried to load them (using the wrong headers) from the remote site in order to display them in the document.

Isn’t there a better solution to the problem than to remove the rule?

Well, yes. The best solution is that OpenOffice sends valid requests (including a valid Transfer Encoding value). Another solution would be to tell members not to copy content from the forum into OpenOffice, and use another tool instead to copy content. Admittedly, both solutions are a bit unrealistic, but we do hope that the people at OpenOffice resolve the problem.

So, the next time, our dear reader, your clients complain about being randomly blocked from accessing your website, think about checking your ModSecurity log; it might be OpenOffice. If you need help uncovering the culprit, then please contact us. We are Joomla security experts, we are based in Montreal, we are hard workers, and our rates are always affordable.

For a while now, we were noticing intermittent load spikes on the server powering a high traffic Joomla website of a client of ours. These spikes were troubling, because they were caused by neither the MySQL database nor the PHP instance; they were caused by something else, something that we didn’t know what it was until this morning…

At around 9:15 AM we noticed, by chance, that the cxswatch process was erratically appearing and disappearing in the top -d shell command during the load spikes. The process had huge processor usage, often close to 20%.

For those who don’t know, the cxswatch process is short for the ConfigServer eXploit Scanner, which is, as the name suggests, a Linux application that is used to scan directories for exploits. It is now installed by default on a WHM server, and it is somehow instrumental in finding malicious/hacked files as soon as they are uploaded/modified. Of course, this means that we cannot just disable the application in order to address the load issue, so our best option was to check the logs for “issues”. So, we logged in to WHM, and then clicked on ConfigServer eXploit Scanner in the left menu (in the Plugins section), and then we clicked on the Tail cxs Watch Log button. Here’s what we saw:

Jan 16 09:16:39 pro01 cxswatch[37473]: WARNING: '/home/[cpanel-user]/public_html/cache/com_content' scanned 6 times in the last 30 seconds, you might want to ignore this resource

The above message was there hundreds of times, but with different dates. Apparently, the Joomla cache folder is so dynamic that it is scanned very frequently, leading to unnecessary overhead, so we needed to tell cxswatch to just ignore this folder. Here’s how we did it:

• We ssh’d to the server.

• We opened the file cxs.ignore (which is located under the /etc/cxs/ folder) using vi:

  vi /etc/cxs/cxs.ignore

• We added the following line to it (note that, by default, the cxs.ignore file doesn’t exist; in that case editing it and saving it through vi will just create it):

  dir: /home/[cpanel-user]/public_html/cache

• We saved the file and then we restarted ConfigServer eXploit Scanner by going back to its settings in WHM and then clicking on the Restart cxs Watch button.

• That’s it! Once we did the above, the load was reduced substantially, especially during high activity hours when cache files were constantly being (re-)generated.

But, doesn’t disabling the cxs for the “cache” folder compromise security?

Well, not particularly, and this is because we have an .htaccess file with a deny from all line in that folder. So, even in the unlikely case where a malicious file gets uploaded to that folder, it can’t be executed. Additionally, we do have a midnight scanner that can catch malicious files and that is run on all folders.

But, what if a malicious user uploads a hacked file that is masqueraded as a legitimate cache file?

Well, doing that is not easy. You see, the names of cached files in Joomla follow a certain hash, and it is probably (much) easier to know the root password of the server than to figure out the “cache hash”.

We hope that you found this post interesting and useful. If you need help with the implementation, then we are always there for you. All you need to do is to contact us and we’ll do the work for you swiftly, professionally, and at affordable fees!

Note: This post is somehow technical. Some PHP knowledge is required to understand it.

After updating PHP to 7.2 on the server powering the Joomla website of a major client, we noticed that one of the modules became broken. The module was supposed to display some images with a specific size, and, while the images were being displayed, the size was not correct, and some of the images were overlapping (because the layout assumed that all the images had the same, specific width and height).

We investigated the issue, and we were able to determine that it was caused by this line:

$$arrSingleItemPreText[0] = $arrSingleItemPreText[1];

The above line contains $$, which PHP refers to as variable variable. So, what is a variable variable?

Let us explain it with an example… Let’s say that you have the following PHP code:

$myVariable = "itoctopus";
$$myVariable = "Joomla Experts";

The first line is obvious. The second line is translated the following way:

$itoctopus = "Joomla Experts";

As you can see, we replaced $myVariable to its value in the first line, and we left the first $ sign. So, $$myVariable became $itoctopus, which means that $itoctopus will have the value of “Joomla Experts”.

Now, going back to the above line we were having a problem with, we examined it thoroughly, and we were sure that there was nothing wrong with it. It was just another variable variable, so why is that a problem after updating to PHP 7.2? Has PHP suddenly decided to stop interpreting variable variables? We researched the issue, and, as we had expected, the root cause turned out be something different…

It was the interpretation of the line that was different. In versions prior to PHP 7, PHP had a mixed interpretation of variables, in most cases it interpreted them left to right, but, in some cases, it interpreted them right to left. In PHP 7, for consistency reasons, variable interpretation is strictly left to right (see the section “Changes to variable handling” on the official PHP 7 backward incompatibility page). So, in PHP 5, the above line was interpreted as:

${$arrSingleItemPreText[0]} = $arrSingleItemPreText[1];

In PHP 7, however, it was interpreted the following way:

($$arrSingleItemPreText)[0] = $arrSingleItemPreText[1];

The first line above (the PHP 5 interpretation) means that if the value of $arrSingleItemPreText[0] is “myVariable”, then $myVariable will be set to $arrSingleItemPreText[1]. The second line (the PHP 7 interpretation) will try to find a value for the variable $arrSingleItemPreText (which is an array), and then set the variable made out of that value to $arrSingleItemPreText[1]. This is a completely different thing, and it will certainly fail.

So, how did we solve the problem?

We solved the problem by replacing the problematic line with this one:

${$arrSingleItemPreText[0]} = $arrSingleItemPreText[1];

This forced PHP 7 to emulate the same behavior of PHP 5 for our particular variable variable.

Is the Joomla 3.8.3 core affected by PHP 7.2 update?

The short answer is no. The core (in the absolute majority of cases) is not broken by an update to PHP 7.2 . Having said that, there are quite a few extensions that are not compatible with PHP 7.2.

We hope that you found this post useful. If your Joomla 3.8.3 site (ideally, you should only update to PHP 7.2 if you’re running the latest version of Joomla) gets broken after updating to PHP 7.2, then try disabling your extensions one by one (plugins, modules, switch to a core template) in order to locate the problematic extension. If you need help with that, then please contact us. We will solve you problem swiftly, cleanly, and affordably.

As of January 1^st, 2018, itoctopus will no longer be accepting work from individuals, or from very small businesses (less than 5 employees). Let us explain why…

As you may already know, itoctopus has been serving businesses of all sizes for over a decade now. That long period of time has provided us with some interesting statistics about individuals/very small businesses:

• They generate less than 2% of our gross revenue.

• They take just over 30% of our time and they are generally stressful to work with.

• Some of them are not very pleasant to work with, and all of the unpleasant clients that we have had were individuals/very small businesses.

• 100% of the billing problems that we have had were caused by them.

• Their retention rate is very low despite us doing our best to please them.

According to our statistics and observations, the disadvantages of serving individuals/very small businesses far outweigh the advantages. In fact, we firmly believe that if we continue serving them we will hurt our business, and we will hinder our huge potential for growth. As such, we came to a conclusion last Friday (December 22^nd, 2017) that we will no longer accept tasks/projects from individuals/very small businesses as of January 1^st, 2018. This decision was tough, because some of these clients are really nice, but the prosperity and the future of our business is at stake.

Will this decision affect existing clients?

Unfortunately, yes. All existing clients who are individuals or very small businesses will no longer be served by itoctopus as of 2018. There is an exception, however, and it’s those developers who send us subcontracting work.

Won’t that cause itoctopus to lose money?

Definitely not! In fact, we estimate a 30% growth because of this decision. And even if that’s the case (that we will lose money), we think that our health and sanity are much more important than 2% of our revenue.

We hope that we have objectively clarified our decision, and we hope that our very small clients will find something similar to the service they were getting here elsewhere.

Yesterday morning, a client emailed us that he wasn’t able to update his Joomla website from version 3.7.5 to version 3.8.3. He told us that he was seeing an “an ajax error of some sort” after clicking on the Update Now button. We immediately thought that he had permission issues on the restore.php file, but we quickly realized that this wasn’t the case.

The “ajax error” that the client was seeing was this one:

Error: Could not open archive part file /home/[cpanel-user]/public_html/tmp/Joomla_3.8.3-Stable-Update_Package.zip for reading. Check that the file exists, is readable by the web server and is not in a directory made out of reach by chroot, open_basedir restrictions or any other restriction put in place by your host.

Additionally, the following warning was being displayed:

JFolder::create: Could not create folder.Path: /home/[cpanel-user]

We quickly examined the permissions (and the ownership) on the logs and the tmp folders, and they were both OK. What could it be, we wondered? We couldn’t think of anything, and so, in a moment of despair, we copied the value of the $tmp_path variable from the configuration file, and we tried to cd to it in the shell, but we got the following error:

-bash: cd: /home/[cpanel-user]/public_html/tmp: No such file or directory

How could it be? So we carefully examined the path, and it turned out there was a typo in [cpanel-user] (for example, itoctopus was spelled itoctopos). We fixed the typo, and then we refreshed the update page (by pressing F5), but still, we saw the same error. We were sure that what we did should have fixed the problem, but, for some reason, it didn’t.

After a few minutes of extreme frustration, we remember that we had to browse off the update page, and then go back, and so we did that: we went to the System – Control Panel page, and then we clicked on the Update Now button, and this time, it worked!

We hope that you found this post useful, and that it helped you update your Joomla website. If it didn’t, or if you need help, then please contact us. We will fix your problem quickly, cleanly, and affordably!

Note: This post assumes that you are using WHM and EasyApache 4. If you aren’t using them but are seeing the same problem, then the heart of the solution is correct (you will need to install the proper MySQL extension, but the implementation is different).

It was around 2:00 AM last Saturday when were scheduled to update PHP on the server powering a major website that we manage to the latest version, which is PHP 7.2. We expected a smooth update, but, unfortunately, after updating to PHP 7.2 (we used EasyApache 4), we saw the following error:

Error displaying the error page: 0 – The MySQL adapter mysqli is not available

Hmmm… It seems that our update to PHP 7.2 did not include the appropriate MySQL libraries; we needed to install them. So, we did the following:

• We logged in to WHM.

• We clicked on the EasyApache 4 link in the left panel (just under Software).

• We clicked on the Customize button next to Currently Installed Packages.

• We clicked on the PHP Extensions link in the left middle tab.

• We enabled the php72-php-mysqlnd extension.

• We clicked on Next at the bottom several times and finally we clicked on Provision.

• We restarted Apache.

We then visited the website, and, unsurprisingly, this time it worked! We were very happy because the whole thing consisted of a few minutes of downtime during off-hours.

But why wasn’t the MySQL extension “checked” by default in EasyApache 4?

We don’t know. We know that, in the absolute majority of cases, PHP is used in conjunction with MySQL, so the fact that EasyApache 4 is not selecting the php72-php-mysqlnd extension by default is a bit disappointing.

Will doing the above always fix the “0 – The MySQL adapter mysqli is not available” error?

It should, but, if it doesn’t fix the problem for you, then make sure that:

• You restarted Apache. If you don’t restart Apache, then your changes to the PHP instance will not take effect and your problem won’t be solved.

• You are really using PHP 7.2. Installing PHP 7.2 (or any version of PHP) through EasyApache 4 (or EasyApache 3) doesn’t necessarily mean that you are actually using PHP 7.2 on your Joomla site. In order to force PHP 7.2 on all your cPanel accounts, then you will need to login to WHM, click on MultiPHP Manager on the left tab (under Software), and then, under System PHP Version, choose ea-php72 and then click on Apply. You will need to restart Apache after doing this.

  Alternatively, you can set the PHP version to 7.2 at the application level, by opening the .htaccess file and adding the following line to its very beginning:

  <IfModule mime_module>
    AddType application/x-httpd-ea-php72 .php .php5 .phtml
  </IfModule>

  The above line will instruct Apache to use your chosen PHP version instead of the global PHP version that is set in the MultiPHP Manager.

We hope that this post helped you resolve the “0 – The MySQL adapter mysqli is not available” error on your Joomla website. If it didn’t, or if you feel that fixing the problem is a bit over your head, then please contact us. Our fees are super affordable, our work is extremely clean, and we are really friendly!

A new client called us and told us that they are seeing an Internal Server Error (Error 500) after moving their site from one server to another. They told us that they’re sure that all the files were properly moved (they zipped the Joomla website on the origin server and extracted it on the destination server), and that the database was properly created, populated, and linked to from the configuration.php file.

Luckily, we ran through this issue before, and so we knew that the problem may be either caused by some wrong file permissions or some .htaccess rules that are not compatible with the new environment.

Our first step was to check the files’ permissions, so we ssh’d to the website and we cd’d to the /home/[cpanel-user]/public_html/ folder, and we noticed that all the files were owned by root. So, we fixed this problem by issuing the following command (when under the /home/[cpanel-user]/public_html/ folder):

chown -R [cpanel-user].[cpanel-user] *

Note: [cpanel-user] is the username of the cPanel account owning the website, such as mydomain.

We loaded the website and it was still displaying Internal Server Error. So, our next step was to check the .htaccess file, and it didn’t take us long to find the problem…

The following code was at the very beginning of the .htaccess file:

php_flag magic_quotes_gpc off

But, the server was using PHP 7.2 and, not only PHP’s magic quotes are deprecated in that version; they just no longer exist! So, we removed that line from the .htaccess file and we loaded the website again, and this time, it worked!

Now, if you, our dear reader, are seeing an Internal Server Error on every page of your Joomla site, then check if the permissions/ownership on your Joomla files are correct, also check if there is anything fishy in your .htaccess file. If you feel that you need help, then please contact us. We’re always anxious to help, we love working on Joomla, and we won’t charge you much!

As of Joomla 3.8.0 (and so far until Joomla 3.8.2), the articles model in the com_content extension (both frontend and backend) uses DISTINCT in the main query that returns the articles (in the getListQuery function). For the untrained eye, such a change is harmless, but, to anyone who has some experience in MySQL (and any other relational database), this is a dangerous change that can have devastating effects on large Joomla sites with high traffic. Why? Well, before explaining why, let us first examine how DISTINCT was introduced to the Joomla core…

On August 22^nd, 2017, a GitHub user called JannikMichel (who is a user with a very small number of contributions), proposed this pull request. In short, the pull request was meant to redo/enhance this 3 year old pull request that added article filtering on multiple access levels/authors/categories/tags. So, when you are in the Articles page in the backend of a Joomla site, you will be able to filter on multiple categories (or authors, or tags, etc…).

Now, since Joomla allows only one access level/author/category per article, you won’t have a problem if you filter on multiple access levels/authors/categories. For tags, it’s a different story: Joomla allows an article to have multiple tags (tags are nothing but trouble by the way), so, if you filter on multiple tags, the underlying query can return the same article multiple times. To address this issue, the following code in Joomla 3.7.5 (in the articles.php file which is located under the components/com_content/models folder)…

$query->select(
	$this->getState(
		'list.select',
		'a.id, a.title, a.alias, a.introtext, a.fulltext, ' .
			'a.checked_out, a.checked_out_time, ' .
			'a.catid, a.created, a.created_by, a.created_by_alias, ' .
			// Published/archived article in archive category is treats as archive article
			// If category is not published then force 0
			'CASE WHEN c.published = 2 AND a.state > 0 THEN 2 WHEN c.published != 1 THEN 0 ELSE a.state END as state,' .
			// Use created if modified is 0
			'CASE WHEN a.modified = ' . $db->quote($db->getNullDate()) . ' THEN a.created ELSE a.modified END as modified, ' .
			'a.modified_by, uam.name as modified_by_name,' .
			// Use created if publish_up is 0
			'CASE WHEN a.publish_up = ' . $db->quote($db->getNullDate()) . ' THEN a.created ELSE a.publish_up END as publish_up,' .
			'a.publish_down, a.images, a.urls, a.attribs, a.metadata, a.metakey, a.metadesc, a.access, ' .
			'a.hits, a.xreference, a.featured, a.language, ' . ' ' . $query->length('a.fulltext') . ' AS readmore'
	)
);

…was changed to the following in Joomla 3.8.0:

$query->select(
	$this->getState(
		'list.select',
		'DISTINCT a.id, a.title, a.alias, a.introtext, a.fulltext, ' .
		'a.checked_out, a.checked_out_time, ' .
		'a.catid, a.created, a.created_by, a.created_by_alias, ' .
		// Published/archived article in archive category is treats as archive article
		// If category is not published then force 0
		'CASE WHEN c.published = 2 AND a.state > 0 THEN 2 WHEN c.published != 1 THEN 0 ELSE a.state END as state,' .
		// Use created if modified is 0
		'CASE WHEN a.modified = ' . $db->quote($db->getNullDate()) . ' THEN a.created ELSE a.modified END as modified, ' .
		'a.modified_by, uam.name as modified_by_name,' .
		// Use created if publish_up is 0
		'CASE WHEN a.publish_up = ' . $db->quote($db->getNullDate()) . ' THEN a.created ELSE a.publish_up END as publish_up,' .
		'a.publish_down, a.images, a.urls, a.attribs, a.metadata, a.metakey, a.metadesc, a.access, ' .
		'a.hits, a.xreference, a.featured, a.language, ' . ' ' . $query->length('a.fulltext') . ' AS readmore'
	)
);

By using DISTINCT, the SQL code in Joomla 3.8.0+ ensures that no row is returned twice by the query, which is a good thing. Unfortunately, this results in a much bigger problem than the original problem: DISTINCT is heavy, very heavy, especially when it is combined by GROUP BY (which is the case in the getListQuery function), in which case it will most likely need to create a temporary table (which results in a huge performance hit). A quick testing for the above query on a large Joomla site revealed that the query is, on average, 5-6 times slower when using DISTINCT. Since the above query is one of the most executed queries on a Joomla site, then it may will substantially increase the load on the server, making the site unusable especially during peak hours.

Not only is the usage of DISTINCT very harmful, it is also disappointing for several reasons:

• It was approved by multiple Joomla core developers who consider themselves experts. We can count 3 (yes, three) expert core developers who checked this pull request and tested it successfully. This is alarming, since it means that core Joomla developers do not really read the code of rookie developers, which, in most cases, is unoptimized and buggy.

• It wasn’t added in a condition. There is no need whatsoever to use DISTINCT when there is no multiple filtering on the tags. There must be an if condition for the usage of DISTINCT.

• It was added to the backend and the frontend articles model. While one can understand the reasons behind adding DISTINCT to the backend, there is no explanation whatsoever on why it was added to the frontend. We have yet to see a Joomla site that allows multiple tag filtering on the frontend.

For our large clients, we are removing DISTINCT from the articles model (both from the frontend and from the backend). We always try our best to avoid core alterations, but in this situation, we have no choice. It is either a core modification or an extremely slow site.

We hope that you found this post useful. If you have any questions about it, or if you need us to optimize your Joomla website, then please contact us. We are experts in Joomla optimization, our work is very delicate, and our fees are always affordable.

There are many recipes for a disaster out there, our favorite one is the following:

• One three-pound European white truffle.
• Eight ripe fruits from a jellyfish tree.
• 7 drops of fairy tears.
• One medium sized hair from the tail of a yellow (#ffff00) unicorn.
• One onion. Just a regular, medium sized onion.
• A sprinkle of moon dust.
• 2 liters of Acqua Di Cristallo Tributo a Modigliani water.
• One eighteenth century Qianlong porcelain vase.

In a pot, boil the truffle in the Acqua Di Cristallo Tributo a Modigliani water (throw the empty water bottles as you won’t need them anymore). About 10 minutes from when the water begins to bubble, add the jellyfish fruits and wait until the water turns slightly red. When that happens, add the 7 drops of fairy tears. Wait exactly 10 seconds between each drop (otherwise, you will have to throw all the ingredients and start over).

Slice the onion (very thin slices) and then fry it in a pan on medium temperature. Once brown, add the unicorn hair, and stir until the mix smells like a gardenia flower. Sprinkle the moon dust over the mix, and then add the whole thing to the pot of truffles (while the latter is boiling).

Serve in the eighteenth century Qianlong porcelain vase, and, for good bad luck, break the vase (Turkish restaurant style) when serving. This recipe for a disaster serves 6 and the results are guaranteed.

Admittedly, the above recipe is a bit daunting, and can be expensive if you don’t know the right people. An easier (and possibly cheaper) alternative is to just turn on progressive caching on your high traffic Joomla site. This will ensure that your Joomla website stops working within 24 hours. The problem is the same as before, only, for some reason, it is much much worse. We are getting many clients complaining about their sites suddenly failing, only to discover the root cause of the problem is the usage of progressive caching.

By the way, in all of the websites that we have examined, none of them actually needed the features of progressive caching. In fact, the reason why progressive caching was chosen for these sites is that the admins thought that progressive caching provided better results than conservative caching, and this is because progressive caching was ordered last in the System Cache drop down (which wrongly implied that it was the best caching mechanism).

But why is progressive caching that bad?

Well, because of 2 things: 1) it floods the filesystem with files, and 2) there seems to be something wrong in the algorithm generating the cached files (we cannot confirm the latter point yet, but we know for sure that progressive caching was less destructive on Joomla 2.5). Typically, high traffic websites hosted on non-SSD drives suffer most.

So, unless you want to try a quick recipe for a disaster, steer away from Joomla’s progressive caching. Most likely you don’t need it, and, even if you do, there are much better alternatives out there. If you need to know what your options are, then just contact us. We are always ready to help, our work is professional, and our fees are affordable.

Most people love mystery movies, there is something thrilling about them. We also love mystery movies, but there is something that we love even more, and it is Joomla mysteries, and yesterday we were fortunate enough that we were assigned to investigate one of these mysteries…

At around eleven hundred hours yesterday morning, a regular client of ours told us that they were seeing the following error on their Joomla site:

Error displaying the error page: Application Instantiation Error: Could not connect to MySQL.

Of course, the above error is probably the most common fatal error on Joomla sites, and we have written about it before, yet this time, it was different, because a few minutes later, the client called back and told us that the site was now working. Hmmm… It was the first time where we see this issue resolve itself. That was weird, we thought. We asked the client if they wanted to investigate further, but they told us that they were OK; it was just a hiccup…

A couple of hours later, they emailed us, and told us that they were seeing the same error message that they saw in the morning. By the time we received the email and checked the site again, the error was gone. So, we emailed the client back, and we recommended that they authorize an investigation about this issue, because instability issues such as these are never a good sign. The client immediately authorized the investigation, and the first thing that we checked was the MySQL error log which is located under the /var/lib/mysql folder (the file name typically ends with .err). Here’s what we saw:

2017-11-07 11:03:29 82870 [Note] Plugin 'FEDERATED' is disabled.
2017-11-07 11:03:29 82870 [Note] InnoDB: Using atomics to ref count buffer pool pages
2017-11-07 11:03:29 82870 [Note] InnoDB: The InnoDB memory heap is disabled
2017-11-07 11:03:29 82870 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2017-11-07 11:03:29 82870 [Note] InnoDB: Memory barrier is not used
2017-11-07 11:03:29 82870 [Note] InnoDB: Compressed tables use zlib 1.2.3
2017-11-07 11:03:29 82870 [Note] InnoDB: Using Linux native AIO
2017-11-07 11:03:29 82870 [Note] InnoDB: Using CPU crc32 instructions
2017-11-07 11:03:29 82870 [Note] InnoDB: Initializing buffer pool, size = 64.0M
2017-11-07 11:03:29 82870 [Note] InnoDB: Completed initialization of buffer pool
2017-11-07 11:03:29 82870 [Note] InnoDB: Highest supported file format is Barracuda.
2017-11-07 11:03:29 82870 [Note] InnoDB: Log scan progressed past the checkpoint lsn 65178299172
2017-11-07 11:03:29 82870 [Note] InnoDB: Database was not shutdown normally!
2017-11-07 11:03:29 82870 [Note] InnoDB: Starting crash recovery.
2017-11-07 11:03:29 82870 [Note] InnoDB: Reading tablespace information from the .ibd files...
2017-11-07 11:03:29 82870 [Note] InnoDB: Restoring possible half-written data pages
2017-11-07 11:03:29 82870 [Note] InnoDB: from the doublewrite buffer...
InnoDB: Doing recovery: scanned up to log sequence number 65178300463
InnoDB: 1 transaction(s) which must be rolled back or cleaned up
InnoDB: in total 1 row operations to undo
InnoDB: Trx id counter is 379714816
2017-11-07 11:03:29 82870 [Note] InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percent: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
InnoDB: Apply batch completed
InnoDB: Starting in background the rollback of uncommitted transactions
2017-11-07 11:03:30 82870 [Note] InnoDB: 128 rollback segment(s) are active.
2017-11-07 11:03:30 7f6cf1e16700  InnoDB: Rolling back trx with id 379714508, 1 rows to undo
2017-11-07 11:03:30 82870 [Note] InnoDB: Waiting for purge to start
2017-11-07 11:03:30 82870 [Note] InnoDB: Rollback of trx with id 379714508 completed
2017-11-07 11:03:30 7f6cf1e16700  InnoDB: Rollback of non-prepared transactions completed
2017-11-07 11:03:30 82870 [Note] InnoDB: 5.6.38 started; log sequence number 65178300463
2017-11-07 11:03:30 82870 [Note] Server hostname (bind-address): '*'; port: 3306
2017-11-07 11:03:30 82870 [Note] IPv6 is available.
2017-11-07 11:03:30 82870 [Note]   - '::' resolves to '::';
2017-11-07 11:03:30 82870 [Note] Server socket created on IP: '::'.
2017-11-07 11:03:30 82870 [Note] Event Scheduler: Loaded 0 events
2017-11-07 11:03:30 82870 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.6.38'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Community Server (GPL)
2017-11-07 13:11:21 11675 [Note] Plugin 'FEDERATED' is disabled.
2017-11-07 13:11:22 11675 [Note] InnoDB: Using atomics to ref count buffer pool pages
2017-11-07 13:11:22 11675 [Note] InnoDB: The InnoDB memory heap is disabled
2017-11-07 13:11:22 11675 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2017-11-07 13:11:22 11675 [Note] InnoDB: Memory barrier is not used
2017-11-07 13:11:22 11675 [Note] InnoDB: Compressed tables use zlib 1.2.3
2017-11-07 13:11:22 11675 [Note] InnoDB: Using Linux native AIO
2017-11-07 13:11:22 11675 [Note] InnoDB: Using CPU crc32 instructions
2017-11-07 13:11:23 11675 [Note] InnoDB: Initializing buffer pool, size = 64.0M
2017-11-07 13:11:23 11675 [Note] InnoDB: Completed initialization of buffer pool
2017-11-07 13:11:23 11675 [Note] InnoDB: Highest supported file format is Barracuda.
2017-11-07 13:11:23 11675 [Note] InnoDB: Log scan progressed past the checkpoint lsn 65220975371
2017-11-07 13:11:24 11675 [Note] InnoDB: Database was not shutdown normally!
2017-11-07 13:11:24 11675 [Note] InnoDB: Starting crash recovery.
2017-11-07 13:11:24 11675 [Note] InnoDB: Reading tablespace information from the .ibd files...
2017-11-07 13:11:27 11675 [Note] InnoDB: Restoring possible half-written data pages
2017-11-07 13:11:27 11675 [Note] InnoDB: from the doublewrite buffer...
InnoDB: Doing recovery: scanned up to log sequence number 65221104018
2017-11-07 13:11:30 11675 [Note] InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percent: 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
InnoDB: Apply batch completed
2017-11-07 13:11:33 11675 [Note] InnoDB: 128 rollback segment(s) are active.
2017-11-07 13:11:33 11675 [Note] InnoDB: Waiting for purge to start
2017-11-07 13:11:33 11675 [Note] InnoDB: 5.6.38 started; log sequence number 65221104018
2017-11-07 13:11:33 11675 [Note] Server hostname (bind-address): '*'; port: 3306
2017-11-07 13:11:34 11675 [Note] IPv6 is available.
2017-11-07 13:11:34 11675 [Note]   - '::' resolves to '::';
2017-11-07 13:11:34 11675 [Note] Server socket created on IP: '::'.
2017-11-07 13:11:35 11675 [Note] Event Scheduler: Loaded 0 events
2017-11-07 13:11:35 11675 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.6.38'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Community Server (GPL)

You can clearly see that within 2 hours, MySQL restarted twice, by itself! But why? This is where this whole thing started to spin into a fun Joomla mystery! We looked at all the logs, for a sign that can lead us to the real cause of the problem, and after many hours of investigation, we found a clue. It was the following lines in the messages file which is located under the /var/log folder:

Nov  7 11:03:21 host kernel: [42008818.160677] Out of memory: Kill process 39364 (mysqld) score 27 or sacrifice child
Nov  7 11:03:21 host kernel: [42008818.161356] Killed process 39364, UID 498, (mysqld) total-vm:1402564kB, anon-rss:16576kB, file-rss:252kB
Nov  7 11:03:22 host kernel: [42008818.858715] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0

If you carefully read the above lines, you will understand what was happening. The system ran out of memory, and it had to make a decision: either to kill the mysqld process or to sacrifice its child(ren). In the next line, you can see that the kernel decided to kill the mysqld process and immediately went ahead with this decision, which resulted in restarting the MySQL server.

OK, now we know it’s a memory issue. But why? Doesn’t their server have enough memory? We issued the following command in the shell to know the amount of memory the server has…

free -m

… and it returned the following:

             total       used       free     shared    buffers     cached
Mem:          1815       1545        270          0        130       1004
-/+ buffers/cache:        409       1405
Swap:         1999        159       1840

Aha! So this server has less than 2 GB of RAM (it is obviously a VPS, and not a server), and 0% of this RAM was free. Of course, Linux tends to allocate unused memory for cache to speed things up, but still, 2 GB is obviously too small for this website, as the server is running out memory.

We offered 2 solutions to the client: 1) We can optimize the queries on the site in order to reduce the memory usage on the database server, or 2) They upgrade to a real dedicated server with at least 16 GB of RAM.

The client opted for the latter solution, which is, in our opinion, the ideal solution.

So there you have it, if you have intermittent downtimes on your MySQL server, then check your logs (especially the /var/log/messages file) to see if your server is running out of memory. If that’s the case, then the best route that you can take is to add resources to your server (or migrate to a more powerful server that has more RAM). If you want to take an alternative route, then just contact us. We are always ready to help, our fees are right, and our work is ultra professional!

You haven’t done anything on your Joomla website for 4 weeks, and you are the only authorized person that has access to the backend of the website and to the site’s filesystem/database. Yet, you wake up this morning and you find out that your website (frontend and/or backend), is displaying a blank page. You call your host, and they tell you they know nothing about it. You panic, you google this issue, and you land on this post. You did the right thing, because in this post, we’ll reveal the top 7 reasons that may have caused the blank page that you’re now seeing on your Joomla site.

1. Your website got hacked: In the majority of cases, a sudden blank page on a Joomla website is caused by a hack that didn’t go very well; the hacker tried to inject malicious code on your site, but instead, crashed the website. Look at the bright side of this, if the hack fully succeeded and your website still worked, then it would have taken you a long time to discover that your website was hacked, which may cause Google to penalize your website (it is not fun to get out of a Google penalty, and sometimes, you are stuck there for a very, very long time).

2. Your environment has changed: The second common reason that can cause a sudden blackout (or a blankout) on your Joomla site is an environment change. For example, a MySQL update and/or a PHP update that is (are) not supported by your current version of Joomla. From our experience, asking the host to revert the update is a futile endeavor (although some hosts do have some tools that will allow you to use a different version of MySQL and/or PHP), so the best solution is to either update your website to the latest version of Joomla, or hire someone to resolve the incompatibilities between your Joomla website and the updated environment. Keep in mind that at times the issue is caused by one ore more 3^rd party extension that are incompatible with the new version of MySQL and/or PHP.

3. Your system/database ran out of space: On every page load, Joomla writes something to the database (typically in the #__session table). If Joomla cannot write to the disk for one reason or the other, then it will fail and will display a blank page (unless you have error reporting turned on, in which case Joomla will display the actual error). The solution to this problem is to delete the temporary/unnecessary files (especially obsolete backups) and reduce the size of the database by removing any backup tables/databases. You should also consider asking your host to increase your disk space or your database limits.

4. You have some corruption at the database level: As stated in our last point, Joomla writes to the database on every page load. If there is a corruption at the database level or at a table level, then Joomla will display a blank page. There are several causes behind a database/table corruption, including, but not limited to, disk failure and bad sectors (which are mainly caused by power outages and forced shutdowns). InnoDB databases tend to fix themselves on database restart (if the cause of the issue is not a disk failure). MyISAM databases, on the other hand, must be repaired manually (which is another reason why you should switch the storage engine of your Joomla database from MyISAM to InnoDB).

5. You server has hardware issues: Hardware issues include, but are not limited to: disk failure, motherboard failure, bad RAM, and overheating. Only the data center can fix any hardware issues you may have on your server. If you want to minimize the risk of having hardware issues, then you need to migrate to another, newer server every 12-18 months. Unfortunately, we know that most readers will take this advice with a grain of salt, and that they will only remember reading about it on the itoctopus blog when it’s too late.

6. One of your Joomla plugins went berserk: One of our clients had a system plugin that loaded some content from a remote site. All of a sudden, the content format on the remote site changed and the plugin went berserk, causing the whole Joomla website to crash. Fixing the problem consisted of disabling the plugin (the client didn’t want it anymore). Admittedly, we only had this one case where a system plugin was the cause of the issue, so the probability for this happening on your site is very low. Still, it is not a bad idea to review your system plugins if the cause of your problem is none of the above.

7. You have once worked with a wicked developer: It’s a wicked, wicked world and there are some wicked, wicked Joomla developers out there. It might be that a developer added some code somewhere on your website a long time ago that will make it die on a specific date. Fortunately, debugging this issue is not hard if you hire the right people to do it for you (such as us).

We hope that you found this post useful, and that it helped you address the sudden blank page issue on your Joomla website. If, however, your problem is not caused by anything in the above list, or if you need help with the implementation, then please contact us. We are here to help, our fees are always affordable, we are very reliable, and our work is very clean!

While optimizing an already optimized K2 powered Joomla website yesterday, we noticed the following query in the slow query log:

SELECT i.*, c.name as categoryname,c.id as categoryid, c.alias as categoryalias, c.params as categoryparams FROM #__k2_items as i RIGHT JOIN #__k2_categories AS c ON c.id = i.catid WHERE i.published = 1 AND i.access IN(1,1,5) AND i.trash = 0 AND c.published = 1 AND c.access IN(1,1,5) AND c.trash = 0 AND ( i.publish_up = '0000-00-00 00:00:00' OR i.publish_up <= '2017-10-31 14:53:20' ) AND ( i.publish_down = '0000-00-00 00:00:00' OR i.publish_down >= '2017-10-31 14:53:20' ) AND c.id IN (1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17) AND i.featured = 1 AND i.id > 25000 ORDER BY i.featured DESC, i.created DESC LIMIT 0, 10;

The above query typically runs on the homepage of the high traffic Joomla site, and is used to return a list of featured K2 items belonging to (K2) categories 1 to 17. The query, as you may have already noticed, is already optimized. However, it is slow, and this is because of the following:

c.id IN (1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17)

As you can see in the above, the query uses MySQL’s IN, which is very expensive. Well, not as expensive as buying a whole watermelon in Iqaluit, Nunavut (this is the in North Pole, in case you just don’t have time to google it), but still, really expensive. The reason why the query uses IN is to restrict the search to a number of specific categories. But, is this really necessary?

In the case of this website, it was not. First, all of the K2 categories on the Joomla site consisted of the 17 categories listed above, and second, even if there were other K2 categories than the ones listed above, the filtering on the featured field was enough (the featured field is mainly used to filter the items to be displayed on the homepage). So, we logged in to the backend of the website, and we went to the Home menu item (e.g. the menu item where the Home flag is set), and we clicked on options, and we removed all the categories from Select Categories. We saved the menu item and then we cleared the cache. We then verified that the website was showing the same content that it was displaying before on the homepage (just to make sure that we didn’t break something in the process).

After doing the above, we monitored the slow query log for that website for 24 hours, and, guess what, there were no more slow queries! Hooray!

So, in case you are having some performance issues on your K2 powered website, then make sure you are not filtering on categories when you don’t need to do so (if you are displaying only featured items on the homepage, then it is almost a certainty that you don’t need to have any filtering on the categories). If the K2 performance issues are caused by something else, then please contact us. We are experts in Joomla optimization, our work is super clean, and our fees are super affordable!

This past weekend, a client of ours from Kuwait emailed us and told us that the backend of his Joomla website is displaying a blank page. He told us that the login page to the backend was working, but, as soon as you login, you will see a blank page. He also told us that the problem happened after updating the obRSS extension.

Naturally, the first thing that we did was to disable the obRSS system plugin by renaming its folder under the plugins/system folder, but, unfortunately, that didn’t fix the problem, so the problem must have been caused by something else.

We then enabled error reporting on the Joomla website by setting the value of the $error_reporting variable to maximum (in the global configuration.php file), but we still saw a blank page. This meant that the “@” operator was used and/or output buffering was enabled somewhere, so we tried to disable output buffering through various means, but that didn’t work. We still saw a blank page.

After our 2 failed attempts to unveil the actual error, we decided to debug the issue the good old way. We renamed the whole modules folder (under the administrator folder) to modules_old and we tried to load the backend, and this time, it did work (albeit with no modules whatsoever). This meant that the problem was caused by one of the administrator modules. So we renamed the modules_old folder back to modules, and then we renamed the folders (modules) inside that folder one by one, and we tested the website after each rename. Once we renamed the mod_menu folder to mod_menu_old, the problem was no more! This was weird, we thought, since the problem turned out to be with a core module (we thought that this was caused by a 3^rd party module/plugin).

After discovering that the problem was caused by the mod_menu folder, which is located under the administrator/modules folder, we needed to display the actual error. So, we added the following 3 lines to the very beginning of the menu.php file (which is located under the administrator/modules/mod_menu folder):

ini_set('display_errors', 1);
ini_set('display_startup_errors', 1);
error_reporting(E_ALL);

After doing the above, we loaded the backend of the Joomla website, and this time we saw an error instead of a blank page. It was this one::

Fatal error: Class 'Joomla\CMS\Menu\Tree' not found in /home/[cpanel-user]/public_html/administrator/modules/mod_menu/menu.php on line 63

Hmmm! The above error felt like it was caused by an incomplete Joomla 3.8.1 update, as it seems that the backend menu module was trying to use dependencies from a Joomla 3.8 instance despite the fact that the Joomla website was running 3.7.0. So, we renamed the libraries folder to libraries_old, and recopied the libraries folder from a fresh copy of Joomla 3.7.0, but that didn’t fix the problem.

So we thought, maybe the problem was only in the aforementioned menu.php file, so we copied that file from a fresh Joomla 3.7.0 install, and we loaded the backend of the site. This time, we saw a different error, it was this one:

Fatal error: Call to undefined method ModMenuHelper::getComponents() in /home/[cpanel-user]/public_html/administrator/modules/mod_menu/preset/enabled.php on line 297

So, we did the same thing for the enabled.php file (we copied it from a fresh Joomla 3.7.0) install. We then reloaded the backend, and yet another different error was displayed:

Fatal error: Call to undefined method JAdminCssMenu::getTree() in /home/[cpanel-user]/public_html/administrator/modules/mod_menu/tmpl/default.php on line 17

At this point, we were no longer having fun. So we copied the whole administrator/mod_menu folder from a fresh copy of Joomla 3.7.0 to the current site (after deleting the old mod_menu folder), and this time, it worked! There were no errors being displayed.

Despite fixing the problem, we didn’t feel satisfied, because we felt that the website was in an unstable state. From our experience, a failed Joomla update doesn’t only affect one folder. So, we asked for permission from the client to update the website, and we were granted that permission immediately.

We initiated the Joomla update by clicking on the Update Now button on the home screen of the backend of the Joomla website (of course, we backed up both the filesystem and the database of the website before proceeding), and, instead of having a smooth update, we saw the following error:

Error: Could not open /home/[cpanel-user]/public_html/cache/index.html for writing.

So, we loaded the backend again, and this time, we saw a blank page. We immediately suspected that the blank page was caused by the administrator menu module, and disabling that module by renaming the mod_menu folder (under the administrator folder) to mod_menu_old confirmed our suspicions. Not only that, this failed update allowed us to understand the original cause of the problem: the client tried to update the Joomla website, and it failed on him, but, it did that after updating the administrator menu module to its Joomla 3.8.1 code, which made the module incompatible with the current website, thus triggering the error.

So, we reverted the administrator menu module to Joomla 3.7.0 (again), and we checked the permissions on the index.html file (which is located under the cache folder). They were the following:

-rwxrwxrwx 1 root root 0 Apr 26 2017 index.html*

Obviously, the permissions and the ownership of the file were both wrong, so we fixed them by issuing the following 2 commands in the shell:

chmod 644 index.html
chown [cpanel-user].[cpanel-user] index.html

We then reattempted the update and this time it worked! The website was updated to Joomla 3.8.1 and was working without a hitch. Hooray!

We hope that you found this post useful and that it helped you resolve your problem. If it didn’t, or if you need help with the implementation, then please contact us! Our fees are affordable, our work is professional, and we will be your true Joomla friends!

At itoctopus, we have a strong passion for optimizing Joomla websites, that’s why we always research new ways that may improve their performance. We mostly focus on optimizing the database because sometimes a simple database optimization can have a huge impact on the overall performance of the Joomla website. This time it was no different.

However, the database optimization that we attempted wasn’t about optimizing a query, or indexing a field, or modifying the my.cnf file (the MySQL global configuration file). It was about answering the million dollar question that many have asked but no one has given a straight answer to: “Does closing the connection to the MySQL database cause a performance hit”? Many have provided theories and opinions about this, but none (as far as we know) has given an answer based on an actual experiment. We decided to be the first ones to provide a real, experiment-based answer to this question, not only because it hasn’t been answered before, but also because we were very curious.

Joomla does explicitly close the connection to MySQL in the mysqli.php file which is located under the libraries/joomla/database/driver folder. So we decided to do the following test:

• Run the following ab (Apache Benchmark) on an unmodified Joomla 3.8.1 website (with test data installed) with n concurrent connections, where n is 10, 20, 30, 40, and 50:

  ab -c n -t 10 "http://www.joomla381.com/"

• Remove the following line from the aforementioned mysqli.php file (note: the line is in the disconnect function):

  mysqli_close($this->connection);

• Run the above ab test again on the modified Joomla website, where the MySQL connection is no longer closed.

Unlike our comparison between the performance of Joomla 3.7.5 and Joomla 3.8.1, we didn’t get any exciting data. The data was more or less the same. A few milliseconds here and a few milliseconds there. Nothing of significance was observed. There was no constant advantage for one over the other.

In fact, we thought (hoped?) that not closing the connection would provide a boost (albeit a negligible one) to the performance of the Joomla website, and this is because there is overhead in opening and closing MySQL connections, but it seems that such an advantage (if it really existed in the first place) was offset by something else.

So there you have it! There is no advantage in not closing the connection to the MySQL database on a Joomla website. If you have a different experience, then please comment below. If you need help optimizing your Joomla website, then please contact us. We are experts in Joomla optimization, our work is clean, and our fees are right!

Yesterday evening (or very early in the morning today), we updated the Joomla website of a new client from Joomla 3.1.5 to Joomla 3.7.5 (we are waiting for 3.8.2 to be released to update to 3.8). The update was really nothing out of the ordinary: we did see some issues but we fixed them immediately as all of these issues were déjà vu. We were going to email the client and tell them that we were done, but, as we were about to do that, we noticed a weird error on a custom component, which was more or less like the core Joomla content component, but, it had departments instead of articles. The problem was that when we clicked on a department to edit it (or when we tried to create a new department), we saw the following error:

You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ‘WHERE `catid` = 0 ORDER BY ordering’ at line 3

We analyzed the view, and we were able to narrow down the problem to the following field in the department.xml file under the administrator/components/com_department/models/forms folder:

<field
	name="ordering"
	type="ordering"
	class="inputbox"
	label="JFIELD_ORDERING_LABEL"
	description="JFIELD_ORDERING_DESC"
/>

Naturally, we searched for the form field of type ordering defined at the component level, and we found it, but we didn’t find any mention of catid anywhere in the extension, so the fatal query wasn’t coming from that field. Strangely though, when we removed the field from the XML file, everything worked as it should (but, of course, without the ordering).

So, we enabled debugging in the global configuration of the Joomla website by setting the value of Debug System to “Yes” under System -> Global Configuration -> System. We then loaded the page and we immediately discovered the cause of the problem: the ordering field was being loaded from another location. It was being loaded from the ordering.php core file which is located under the libraries/cms/form/field folder (note that in Joomla 3.8+, the ordering field is located in the OrderingField.php file, which in turn is located under the libraries/src/Form/Field folder). But, what is that field? And why isn’t documented on the Joomla official website?

Further investigation revealed that this field was introduced in Joomla 3.2.0, and that it is used in 3 extensions: com_banners, com_contact, and com_newsfeeds (by the way, we just noticed, why doesn’t Joomla have some standard when it comes to singular/plural naming in its core extensions; why isn’t com_banner and com_newsfeed instead of com_banners and com_newsfeeds?). The core ordering field does exactly what you expect it to do: it displays a drop down of all the content items belonging to the category of the current content item, which allows you to set the latter’s ordering. The field expects the category name of the current content item to be called catid. If it doesn’t find such field, then it crashes, which fully explains the problem that we were having on the client’s site.

Here’s an example usage of the ordering field from the com_banners extension:

<field
	name="ordering"
	type="ordering"
	label="JFIELD_ORDERING_LABEL"
	description="JFIELD_ORDERING_DESC"
	table="#__banners"
/>

You can see that all that you need to pass as a parameter is the table field, which is #__banners in the case of the com_banners extension (note that in previous versions, such as Joomla 3.2.0, you needed to pass the content_type, such as com_banners.banner).

But why wasn’t the “ordering” Joomla form field type documented?

We’re not really sure but we think it was just that the Joomla documentation team forgot about it (we will report this omission to the Joomla core team) – this is plausible because Joomla officially added a whopping 8 fields in Joomla 3.2 (so the ninth one could have easily fell through the cracks). It might also be due to the fact that this field type feels unfinished and non-generic. For example, why is catid hardcoded in the field? Still, the ordering field has its merits, especially for extensions taking advantage of the #__content and the #__categories tables.

So, what did we do to fix the problem?

We fixed the problem by appending the word department to every instance of the word ordering for that field type. Here’s how:

• We renamed the file ordering.php which is located under the administrator/components/com_department/models/fields to departmentordering.php.

• We then opened the departmentordering.php file, and we changed the following line:

  class JFormFieldOrdering extends JFormField

  to:

  class JFormFieldDepartmentOrdering extends JFormField

  We also changed the following line:

  protected $type = 'Ordering';

  to:

  protected $type = 'DepartmentOrdering';

• Finally, we opened the department.xml file which is located under the administrator/components/com_department/models/forms folder and we changed the following:

  <field
  	name="ordering"
  	type="ordering"
  	class="inputbox"
  	label="JFIELD_ORDERING_LABEL"
  	description="JFIELD_ORDERING_DESC"
  />

  to:

  <field
  	name="ordering"
  	type="DepartmentOrdering"
  	class="inputbox"
  	label="JFIELD_ORDERING_LABEL"
  	description="JFIELD_ORDERING_DESC"
  />

Once we did the above, the problem was solved! Woohoo!

Now, if you, our dear reader, are having the same problem and you need help with implementing the solution, then fear not, we are your Joomla super heroes! Just contact us and we’ll fix your website quickly, professionally, and affordably!

Note: This a very lengthy post. So, it’ll be a good idea to read it first thing in the morning when your mind is fresh and while you’re having your wake up coffee (or tea, or milk, or cold water, or nothing).

Another note: This is a very technical and a very advanced post. It is mainly aimed for developers, but non-developers can still enjoy it nonetheless, or can just scroll down to the bottom of this post for the chart.

So far, we haven’t upgraded any of our customers to Joomla 3.8.x, and this is because we learned our lesson a few years ago to wait a couple of months before updating to the latest version of Joomla (unless, of course, the update is a major security patch). However, we are very close followers of the Joomla community, and we have read and heard many comments about the bad performance of Joomla 3.8 when it is compared to Joomla 3.7. We did communicate these concerns to the Joomla developers, who discovered that some events were triggered twice in Joomla 3.8.0 (leading to a major performance issue on large Joomla sites), and who fixed that problem in 3.8.1.

So, is Joomla 3.8.1 faster or slower to Joomla 3.7.5? In this post, we decided to benchmark Joomla 3.8.1 against Joomla 3.7.5, in order to provide a definitive answer to the question.

The Setup

We used an idle dedicated server for our test. The server had the following technical specifications:

• Processor: Intel Xeon E3-1271 v3 Quad-Core
• Memory: 16 GB DDR3 SDRAM (SDRAM, in case you’re wondering, stands for Synchronous Dynamic Random Access Memory)
• Disk: 250 GB SSD Drive

The following software was installed on the server:

• Operating system: CentOs 6 – 64Bit
• Web server: Apache 2.4
• Database server: MySQL 5.5
• PHP version: PHP 5.6

Since we didn’t have any benchmarking tool on the server, we installed ab, the Apache HTTP server benchmarking tool, which is developed by the Apache Software Foundation. Here’s how we installed it

• We edited the yum.conf file which is located under the /etc folder to remove httpd from the exclusion list. We opened the file using vi:

  vi /etc/yum.conf

  From the exclude line (which is the second line in the file), we removed the httpd* entry, to allow the install of the ab tool through yum. We saved the file and exited vi.

• We then issued the following command in order to ensure that we can now install ab through yum:

  yum provides /usr/bin/ab

  (Note: If you see No matches found when you issue the above command, then try contacting your host).

• Once we were sure that we were able to install ab, we installed it using the following command:

  yum install httpd-tools

Now that we have installed ab, te next step was to create 2 cPanel accounts, joomla375.com and joomla381.com (this was easily done through WHM). As you might have guessed, the first account was created in order to install a Joomla 3.7.5 website, while the latter was used for a Joomla 3.8.1 website. But, before installing Joomla on either account, we had to modify the hosts file on our PC and on the server so that these domains resolve to the test server. For our PC, we modified the hosts file which is located under the C:\Windows\System32\drivers\etc folder to include the following 2 lines:

[ip-address-of-the-server] joomla375.com www.joomla375.com
[ip-address-of-the-server] joomla381.com www.joomla381.com

For the server, we added the same lines above in the /etc/hosts file.

At this point, we were ready to install Joomla. We installed Joomla 3.7.5 on joomla375.com and Joomla 3.8.1 on joomla381.com (we will spare you the boring installation details), and – we chose to include the blog test data for both. joomla375.com and joomla381.com ended up being identical sites.

We opened the file my.cnf file under the /etc folder and we added the following 2 lines immediately after [mysqld]:

general_log = on
general_log_file=/var/lib/mysql/all-queries.log

We then restarted MySQL. In case you’re wondering, the above 2 lines will log any query that hits the database server to the all-queries.log file (located under /var/lib/mysql/ folder).

The Benchmarking

Typically, benchmarking is done by issuing a number of concurrent connections to the website for a specific amount of time using a benchmark tool such as ab, then examining the results and comparing them to a different website. However, since we are working on 2 identical websites using the same CMS with a different version, we thought that it’s a good idea to add another metric to the test, which is the size of the query file generated by a page load (or a number of pages loads). In other words, we load the homepage of each Joomla website, with the general_log on, and we compare the size (in bytes) of the all-queries file generated by each Joomla website to the one generated by the other. We will consider that smaller is better (we agree that it is not an accurate benchmark, since the complexity of a query is not always proportional to its string length).

We started with the all-queries benchmark first. Here’s how we did it:

We first loaded the homepage of the Joomla 3.7.5, and we examined, the size in bytes, of the all-queries.log file, and it was 21473 bytes. We emptied the all-queries.log file, and we did the exact same test on the Joomla 3.8.1 website. The size of the file was 18552 bytes. For us, it was a shocking surprise, because we expected the complete opposite. We didn’t expect Joomla 3.8.1 to have about 3 KB less queries than Joomla 3.7.5, so naturally, we were curious. We compared the 2 files and it turned out that Joomla 3.8.1 got rid of all the unnecessary (and never used) references to the #__content_rating table. We did mention the unnecessary references to the #__content_rating table before, and it seems that someone from the Joomla core team was listening then. Other than the removal of the #__content_rating table, we didn’t really find much difference between the 2 files. They were almost identical.

Our next test scenario was slightly more elaborate, as it consisted of doing the following (in the exact sequence below) on the 2 sites:

• Logging in to the Joomla website.

• Adding a user with default privileges.

• Adding an article to the Blog category.

• Viewing the homepage.

In this test, the results were more in line with our expectations: the size of the all-queries.log file was 161394 bytes for Joomla 3.7.5 and 209505 bytes for Joomla 3.8.1, which amounted to almost exactly a 30% increase in the size of the query file in Joomla 3.8.1. Again, we were curious as to what was different, and so we compared the 2 files. This time, we discovered something very interesting…

The all-queries.log file of the Joomla 3.8.1 website was littered with the following query:

SELECT COUNT(`extension_id`) FROM `#__extensions` WHERE `element` = '[com_extension]' AND `type` = 'component'

com_extension was the name of a component, such as com_content, com_menu, etc…. There were hundreds of these weird queries (375 to be exact) – many were repeated! For example, the following query…

SELECT COUNT(`extension_id`) FROM `#__extensions` WHERE `element` = 'com_menu' AND `type` = 'component'

…was literally 45 times in the log file.

These 375 queries didn’t exist in Joomla 3.7.5 (375 queries didn’t exist on Joomla 3.7.5, is it really just a coincidence, or were the developers trying to tell us something?), and these 375 queries resulted in the 30% increase in the size of the log file.

Now, this query logging method is not scientific, and we didn’t really expect to learn much from it, but we did. We did learn that there is a huge overhead (about 50 additional queries per page) in Joomla 3.8.1, and it’s all because of these weird queries…

The next step was to use the classical method for benchmarking, which is the ab method. So, we issued the following 2 commands in the Linux shell:

ab -c 10 -t 10 "http://www.joomla375.com/"

which returned the following:

This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking www.joomla375.com (be patient)
Finished 737 requests


Server Software:        Apache/2.4.23
Server Hostname:        www.joomla375.com
Server Port:            80

Document Path:          /
Document Length:        18052 bytes

Concurrency Level:      10
Time taken for tests:   10.000 seconds
Complete requests:      737
Failed requests:        0
Write errors:           0
Total transferred:      13676509 bytes
HTML transferred:       13304324 bytes
Requests per second:    73.70 [#/sec] (mean)
Time per request:       135.688 [ms] (mean)
Time per request:       13.569 [ms] (mean, across all concurrent requests)
Transfer rate:          1335.57 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:    56  133 139.6    115    1482
Waiting:       56  132 139.6    115    1482
Total:         56  133 139.6    115    1482

Percentage of the requests served within a certain time (ms)
  50%    115
  66%    124
  75%    131
  80%    135
  90%    146
  95%    162
  98%    183
  99%   1306
 100%   1482 (longest request)

…and…

ab -c 10 -t 10 "http://www.joomla381.com/"

…which returned the following:

This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking www.joomla381.com (be patient)
Completed 5000 requests
Completed 10000 requests
Completed 15000 requests
Completed 20000 requests
Completed 25000 requests
Completed 30000 requests
Completed 35000 requests
Completed 40000 requests
Completed 45000 requests
Completed 50000 requests
Finished 50000 requests


Server Software:        Apache/2.4.23
Server Hostname:        www.joomla381.com
Server Port:            80

Document Path:          /
Document Length:        328 bytes

Concurrency Level:      10
Time taken for tests:   2.643 seconds
Complete requests:      50000
Failed requests:        0
Write errors:           0
Non-2xx responses:      50000
Total transferred:      28000000 bytes
HTML transferred:       16400000 bytes
Requests per second:    18918.25 [#/sec] (mean)
Time per request:       0.529 [ms] (mean)
Time per request:       0.053 [ms] (mean, across all concurrent requests)
Transfer rate:          10345.92 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       1
Processing:     0    0   0.1      0       1
Waiting:        0    0   0.1      0       1
Total:          0    1   0.1      0       1
ERROR: The median and mean for the total time are more than twice the standard
       deviation apart. These results are NOT reliable.

Percentage of the requests served within a certain time (ms)
  50%      0
  66%      1
  75%      1
  80%      1
  90%      1
  95%      1
  98%      1
  99%      1
 100%      1 (longest request)

The first command was used to benchmark the Joomla 3.7.5 website, and the second command was used to benchmark the Joomla 3.8.1. So, which one did better?

Before examining the results of the ab command (which are very interesting), let us dissect and explain the ab command line that we issued:

ab -c 10 -t 10 "http://www.joomla381.com/"

The above means that we are sending 10 concurrent (or simultaneous) connections (-c 10) for a maximum of 10 seconds (-t 10) on the website http://www.joomla381.com/ (Note: adding -k to the above command will force the use of the HTTP KeepAlive feature).

Going back to the results, at first glance, it is very obvious that Joomla 3.8.1 did much, much better than Joomla 3.7.5. In fact, Joomla 3.8.1 was able to process 50,000 requests in 2.643 seconds, while Joomla 3.7.5 only processed 737 requests in 10 seconds. The time per request for Joomla 3.8.1 was a a very impressive 0.529 milliseconds, while that of Joomla 3.7.5 was a very slow 135.688 milliseconds. So, if we were to end this article now and we were to use this benchmark data, then we can conclude that Joomla 3.8.1 is 257 times faster than Joomla 3.7.5. This is just too good to be true – more like unbelievable (and rightly so, as you will learn later in this post). So we closely examined the above numbers and we noticed that there was some kind of cheating going on. You see, while the Joomla 3.7.5 website responded with 18052 bytes (this is the Document Length), Joomla 3.8.1 responded with a mere 328 bytes. Clearly, the Joomla 3.8.1 website wasn’t giving the full response.

As you might have guessed, we investigated further…

So, we grabbed the Joomla 3.8.1 website using wget with the following command…

wget http://www.joomla381.com/

…and we got the following response:

Resolving www.joomla381.com... [ip-address]
Connecting to www.joomla381.com|[ip-address]|:80... connected.
HTTP request sent, awaiting response... 403 Forbidden

Aha – the website was returning a 403 error, and then it hit us, we denied, in the .htaccess file of the Joomla 3.8.1 website, all IPs from accessing it with the exception of our IP; so we needed to add the IP of the server as well. The Joomla 3.7.5 website didn’t have the problem because the server IP was already authorized in an allow statement in the .htaccess file.

So, it wasn’t really a conspiracy or cheating or foul play or anything (it’s always easy to blame the poor Joomla development team for anything), it was just us being tired. So we decided to continue our experimentation the next morning.

The next morning (well, at 3 AM to be exact), we continued our work. The first thing that we did was to add the server IP to the .htaccess file of the Joomla 3.8.1 website, and then we re-executed the following ab statement in the shell:

ab -c 10 -t 10 "http://www.joomla381.com/"

This time, it returned totally different results:

This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking www.joomla381.com (be patient)
Finished 714 requests


Server Software:        Apache/2.4.23
Server Hostname:        www.joomla381.com
Server Port:            80

Document Path:          /
Document Length:        18691 bytes

Concurrency Level:      10
Time taken for tests:   10.038 seconds
Complete requests:      714
Failed requests:        0
Write errors:           0
Total transferred:      13705944 bytes
HTML transferred:       13345374 bytes
Requests per second:    71.13 [#/sec] (mean)
Time per request:       140.583 [ms] (mean)
Time per request:       14.058 [ms] (mean, across all concurrent requests)
Transfer rate:          1333.45 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:    58  137 146.1    117    1530
Waiting:       58  137 146.1    117    1530
Total:         58  137 146.1    118    1530

Percentage of the requests served within a certain time (ms)
  50%    118
  66%    127
  75%    134
  80%    138
  90%    153
  95%    168
  98%    187
  99%   1332
 100%   1530 (longest request)

Now it’s more like it. The document length was 18691 bytes, which is very close to the 18052 bytes returned by the Joomla 3.7.5 website (by the way, it would be interesting to know why there is a 639 bytes difference between the 2 – but it’s really outside the scope of this post). The number of completed requests is 714 requests, with a mean time of 140.583 milliseconds. The Joomla 3.7.5 website, in case you don’t feel like scrolling up a bit, handled 737 requests with a mean time of 135.688 milliseconds.

The Chart

At this point, the whole thing became very interesting. We had meaningful and close results. So, we decided to play with the number of connections: we issued a series of ab commands on each website with increasingly higher number of connections (e.g. we changed -c 10 to -c 20, -c 30, etc… – note that we waited a few minutes after each command for the load to return back to 0), and we got this beautiful chart (probably what you have been waiting for since the beginning of this post):

You can see in the above chart 2 things: 1) Joomla 3.7.5 consistently performs better than Joomla 3.8.1, but only slightly, and 2) the 2 lines diverge more with the number of connections. In fact, Joomla 3.8.1 is 3% slower than Joomla 3.7.5 with 10 concurrent connections, but with 50 concurrent connections, it is 6% slower.

So, there you have it. Joomla 3.8.1 is slower than Joomla 3.7.5, and we’re totally not lying with stats charts (please trust us!). In fact, you can conduct the same test yourself and you will likely get similar results.

We hope that you found this post informative and that you had fun reading it. If you have any questions about it, or if you need help optimizing your Joomla 3.8.1 website, then all you need to do is to contact us. Our fees are affordable (and haven’t increased for the past 7 years), our work is clean, and we are the friendliest Joomla developers in the Milky Way (unless, of course, Mars turns out to be populated by extremely friendly Joomla developers).

A new client emailed us that he was seeing the following error on his Joomla website:

Error displaying the error page

He told us that he started seeing the above error when the hosting company moved his website from one server to another. He told us that the hosting company debugged the issue for a long time, but couldn’t determine the problem.

So, we logged in to the cPanel account of the website, and then we went to the File Manager, and from there, we edited the configuration.php file and changed the value of $error_reporting from default to maximum (we couldn’t do that through the backend of the Joomla website since both the backend and the frontend were showing the same error).

When we visited the website again, it displayed the following error:

Error displaying the error page: Application Instantiation Error: No database selected

Aha, now we have a real, meaningful error, and we know that it has to do with the database. But, are the connection parameters incorrect? No, because if they were, then we would see the following error:

Error displaying the error page: Application Instantiation Error: Could not connect to MySQL.

So, we know for sure that the database host, username, and password (these are all set in the configuration.php file) are all correct. But, is the database name correct? So, we checked the name of the database in the configuration.php and we matched it to the one in phpMyAdmin and it did.

We were confused – we were confident that the database parameters were correct, yet we were sure that the problem was with the database parameters (yes, we know, it doesn’t make any sense!). So, we went back to the cPanel homepage, we clicked on the MySQL databases icon, we created another database user, and we assigned that user to the database (we granted the user all the privileges). We then modified the configuration.php values for $user and $password to match those of the user that we just created. We loaded the website and this time it worked. But why wasn’t it working for the old user?

We investigated the issue by comparing the user that we have created and the previous user, and it didn’t take us long to see the different: the old user didn’t have any privileges on the database. It seems that the user was created by the hosting company, but it was not assigned any privileges to the database. So, under the Add User To Database section on the MySQL Databases page in cPanel, we selected the old user under User, and then we selected the Joomla database under Database, we then clicked on the Add button, and on the next page, we clicked on the checkbox ALL PRIVILEGES to select all possible database privileges, and finally we clicked on Make Changes at the bottom. We reverted back the configuration.php to what it was and, unsurprisingly, the website worked!

So, if you have the same problem on your website, then make sure that the right privileges are assigned to the database user on the Joomla database. If you have already done that and it didn’t fix the problem, or if you need help doing that, then please contact us. Our fees are right, our work is clean, and our Joomla experience is unquestionable!

A few weeks ago, we wrote a post in which we admitted that we haven’t discovered fire. This morning, however, we had a new discovery akin to discovering fire, and it was a very clean, very elegant, very lovely method to check if a user is a super user. It all happened by coincidence when were checking the contents of the UserHelper.php file which is located under the libraries/src/User folder. We believe that our discovery is a defining moment in the history of mankind!

We know you’re anxious to know what the method is, so, here’s the code:

JUserHelper::checkSuperUserInUsers(array('5'));

The above checks if 5 is the ID of a super user. If it is, then the function will return true, if not, then it will return false.

What happens if you supply an array of users to the function?

As you can see from the above code, the checkSuperUserInUsers static method accepts an array of users (not just one user), so what happens if you pass an array of users to the function? Well, the function will return true as soon as it detects a user in the array that is a super user. If there are no super users in the array, then the function will return false.

Why does this method exist?

The checkSuperUserInUsers is mainly used to block batch actions on super users when the user performing the action is not a super user. For example, if a “non super user” selects a group of users in the Joomla backend, and clicks on the Delete button on the top, then Joomla will not allow the deletion when one or more selected users are super users. The function checkSuperUserInUsers is used to do the checking.

We hope that you enjoyed our little post this Saturday morning. If, by any chance, you have a Joomla development project, and you need help with it, then please contact us. Our prices are right, our code is clean, and we have over a decade of Joomla experience.

A new client approached us yesterday and told us that they were seeing a blank page on the frontend and on the backend of their Joomla website after updating it to 3.8.0. Since a blank page is a sign of a fatal error, we set the error reporting on the client’s site to “maximum” in the configuration.php file, and then checked the frontend of the website, which was displaying the following:

Fatal error: Call to undefined method JApplicationSite::isClient() in /home/[cpanel-user]/public_html/plugins/system/languagefilter/languagefilter.php on line 94

The backend of the website was displaying the following error:

Fatal error: Call to undefined method JApplicationAdministrator::isClient() in /home/[cpanel-user]/public_html/plugins/system/languagefilter/languagefilter.php on line 94

(Note: Enabling debugging displayed the following error on the backend and on the fronted of the Joomla website: Fatal error: Call to undefined method JProfiler::setStart() in /home/[cpanel-user]/public_html/administrator/index.php on line 45)

If you look closely at the above errors, you will notice that the problem is that 2 basic Joomla classes, JApplicationSite and JApplicationAdministrator, are not being loaded. But why?

We investigated the problem very thoroughly. We looked at all the files that were used to load core classes or support the loading of core classes, notably, the following 2 files:

• The file loader.php which is located under the libraries/cms/class/ folder.
• The file ClassLoader.php which is located under the libraries/vendor/composer folder.

We couldn’t find anything.

We then thought it could be a problem with the PHP version that the client was using, which was PHP 5.4, and so we switched to 5.6, and then 7.0, and then 7.1. None worked!

We then started thinking about the hosting environment, could it be? Well, the client was using GoDaddy, which should cause any decent developer to become a bit skeptical at the very least (in fact, we could just blame GoDaddy for the heat wave we’re having right now in Montreal and everyone would believe us!). So, we copied the website over to one of our servers, and we tested it there. Same exact problem!

Eventually, we decided to copy a fresh copy of Joomla 3.8 onto the website. So, we downloaded Joomla 3.8, we then removed the images and the templates folder from the downloaded zip file, we extracted the zip file onto the website, and we tried loading the website, still, the same error.

Then it suddenly hit us, what if the problem is similar to this other problem that we resolved a while ago which was caused by Joomla loading old library files instead of the new ones? So, we renamed the existing libraries folder to libraries_old, and we copied a fresh copy of the libraries folder onto the website (from the Joomla 3.8 zip file that we just downloaded) and then we tested it. This time it worked!

Aha! So, the problem was caused by the wrong library file(s) being loaded? But which one(s)? We were determined to find out the answer to our question, so, we printed out all the included files on the fixed site, and then, we reverted back and printed out all the included files on the broken site, and we compared the list of files.

There were, in fact, many, many files that were included from the wrong places on the broken Joomla website. For example, the factory.php file was included from the libraries/joomla folder instead of the libraries/src folder. There were also many files included from the libraries/cms folder, and these files were included from folders that should no longer exist under the libraries/cms folder. In fact, the libraries/cms folder in Joomla 3.8 has only 3 folders inside it (Joomla 3.7.5 has 30 folders inside the libraries/cms folder).

It seems that the client has either updated the site manually (e.g. by overwriting the files from a fresh Joomla install), or the client has updated the site from the backend but from an old Joomla website where the update process does not include removing these old library files/folders. In any case, we think the problem is really caused by Joomla, which should take into consideration any update scenario. The problem is also ironic, since it highlights the fact that there is a huge change in the file structure in Joomla 3.8, completely contradicting a statement by a core Joomla developer that the “the difference from 3.7.5 is relatively minor”.

So, if you are seeing the Call to undefined method JApplicationSite::isClient() error on your Joomla website, then try renaming the libraries folder to libraries_old and then copying the libraries folder from a fresh Joomla 3.8 install. If it doesn’t work, or if you are seeing a different error, then please contact us. We are always ready to help, our fees are very reasonable, and we really love working on Joomla websites!

Note: This integration consists of a core override. As usual, please keep in mind that core overrides can lead to stability issues and can be wiped out with a future Joomla update.

A client of ours asked us to integrate Joomla’s “Email this Link to a Friend” (or “Send Article to a Friend”) with HubSpot. What they wanted to do was to add the person sending the article as well as the person receiving it as contacts. As you already know, when someone sends an article to a friend, he is asked to fill in his name, his email, and his friend’s email. So, in other words, we will have the name and the email of the first contact (the person who sends the article), while we will have only the email of the second contact (the person who receives the article).

The first thing that we did was creating the form in HubSpot. The form contained 3 fields: “Email”, “First Name”, and “Last Name”. The internal field names for those fields were “email”, “firstname”, and “lastname”, respectively. Only the email was set to required. Once we created the form, we clicked on the Embed tab on the left, and we saw the following in the textarea just under Copy the snippet below onto your site:

<!--[if lte IE 8]>
<script charset="utf-8" type="text/javascript" src="//js.hsforms.net/forms/v2-legacy.js"></script>
<![endif]-->
<script charset="utf-8" type="text/javascript" src="//js.hsforms.net/forms/v2.js"></script>
<script>
  hbspt.forms.create({ 
    portalId: '111111',
    formId: '1a1a1a1a-2b2b-3c3c-4d4d-5f5f5f5f5f5f'
  });
</script>

We then took note of the value of portalId (which was 111111) and formId (which was 1a1a1a1a-2b2b-3c3c-4d4d-5f5f5f5f5f5f) in the code above. (Note: These are not the real values and you should replace portalId and formId with your form’s values).

After creating the form and taking note of the values of portalId and formId, we moved over to the fun stuff: it was time for sending the contacts from the Send Article to a Friend Joomla form to HubSpot. So, we opened the file controller.php which is located under the components/com_mailto folder, and we added the following function at the very beginning of the MailtoController class:

private function addToHubSpot($email, $name=''){
  //note - you will need to replace "111111" with your portalId, and you will need to replace "1a1a1a1a-2b2b-3c3c-4d4d-5f5f5f5f5f5f" with your formId.
  $url = 'https://forms.hubspot.com/uploads/form/v2/111111/1a1a1a1a-2b2b-3c3c-4d4d-5f5f5f5f5f5f';

  $pageURL = JUri::current();
  $pageName = "Send Article to a Friend";
  
  $name = trim($name);
  if (!empty($name)){
    $arrName = explode(' ', $name);
    if (count($arrName) > 0){
      $firstname  = $arrName[0];
      array_shift($arrName);
      $lastname = implode(' ', $arrName);
    }
    else{
      $firstname = $arrName[0];
      $lastname = $arrName[0];
    }
  }
  
  $fields = array(
    'firstname' => $firstname,
    'lastname' => $lastname,
    'email' => $email
  );
  $strFields = http_build_query($fields);

  jimport('joomla.user.helper');
  $randomPassword = JUserHelper::genRandomPassword(32);
  
  $hsContext = array(
      "hutk" => $randomPassword,
      "ipAddress" => "192.168.1.12", 
      "pageUrl" => $pageURL,
      "pageName" => $pageName
  );
  $strContext = json_encode($hsContext);
  $strContext = urlencode($strContext);
  $strFields = $strFields.'&hs_context='.$strContext;
  
  $ch = curl_init($url);
  curl_setopt($ch, CURLOPT_POST, true);
  curl_setopt($ch, CURLOPT_POSTFIELDS, $strFields);
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
  curl_setopt($ch, CURLOPT_TIMEOUT, 2); //wait a maximum of 2 seconds
  curl_setopt($ch, CURLOPT_HTTPHEADER, array(
      'Content-Type: application/x-www-form-urlencoded'
  ));  
  $result = curl_exec($ch);
}

Then, just before $this->display(); near the very end, we added the following code:

//add the sender to HubSpot
$this->addToHubSpot($from, $sender);
//now add the person receiving the email only if it is different than the person sending the email
if ($from != $email){
  $this->addToHubSpot($email, '');
}

As you can see in the above code, we are only adding the receiver's email if the email of the person sending it is different from that of the receiver. This is because many people send articles to themselves.

Now, after doing the above, we filled in an "Email this Link to a Friend" form, and to our pleasant surprise, it worked as it should: it made 2 form submissions in HubSpot and created 2 contacts! Hooray!

We hope that you enjoyed this post. If you need help with this integration or with any HubSpot integration on your Joomla website, then please contact us. We have implemented many HubSpot integrations on many Joomla websites, our fees are super affordable, and our work is super clean!

We often get calls/emails from clients stating that the response time of their Joomla websites is suddenly high. They say that the day before everything was fine, and all of a sudden in the morning, the website became super slow. Of course, the first thing that we ask them is: “Has anything changed between yesterday and today?”, and their answer is typically “Not that I know of”. But, when we work on the problem, we discover that something did change, and that “something” was the cause of the problem. Since we have worked on many of these websites, we have compiled a list of 7 reasons that can cause a Joomla website to become super slow all of a sudden:

1. Hacked website: When a website is hacked, it typically attempts to grab the contents of a malicious URL using PHP’s curl library or using PHP’s file_get_contents function (the latter can only work when the extremely unsafe allow_url_fopen PHP setting is enabled). This process is typically instantaneous, however, when the malicious URL gets blocked by the host or when the malicious website is hosted on a very slow (and cheap) host, the PHP function will wait for x seconds (where x is determined by the max_execution_time PHP setting) before timing out and serving the page. Obviously, the solution to this problem is to cleanup the website.

2. Draconian firewall settings: Many hosts resort to SYN_FLOOD protection in order to minimize the impact of a DoS (Denial of Service) attack. While there is nothing wrong with that, some cheap/inexperienced hosts set the SYN_FLOOD value (at the firewall level) to a very low number, causing a very slow response time on the website. This is somehow ironic, because the firewall ends up causing the very same problem it was installed to protect the website from. The fix to this problem simply consists of asking the host to increase the SYN_FLOOD value.

3. A switch to a slower hard drive: What most people do not know is that hard drives have a huge, huge impact on the performance of the website. In fact, when the website of a client of ours was moved from an SSD drive to an HDD drive, the performance degraded substantially, and the website suddenly became mostly unresponsive (at best it was super slow). Of course, the fix to the problem in this case was to switch back to an SSD.

4. Disabled caching: There are many Joomla administrators who give super user access to people who don’t deserve it – typically, this is done because these people outrank the Joomla administrators. The problem with this is that some of these people are a bit technical, and, even though the word bit is the essence of programming, it becomes a huge curse when combined with the word technical or knowledge. Essentially, these super users will not only have enough rope to hang themselves, but also the whole website. For example, when a problem happens on the site, and they read that fixing the problem consists of clearing the cache, they go ahead and disable it altogether thinking that they fixed the problem forever. Shortly afterwards, the website becomes very slow or crashes, with nobody having the slightest idea of what might have caused the issue. A super quick fix to this problem is to re-enable caching on the site.

   Another problem causing this issue is the switch from one caching mechanism to another, for example, after switching a client site from File to Memcache caching, we experienced an increase in response time (although it wasn’t substantial) and we reverted back quickly.

5. A Joomla plugin/extension: Some time ago, we investigated a Joomla website that was taking exactly 10 seconds to respond. At first, we thought it was hacked, but eventually, it turned out that there was a plugin called pingomatic which was using PHP’s curl to grab the contents of a remote URL that was no longer accessible, and so the request was timing out. Disabling the pingomatic plugin addressed the problem.

6. High server load on shared hosting: If your website is on a shared host (or even a VPS), then keep in mind that other websites running on the same server can affect the performance of your website. If a website on the same server is resource hungry, and if not enough monitoring/control is done by the host, then this will become your problem and your website will become very slow. An ideal solution to this problem is to move to a dedicated server, a not-so-ideal solution is to tell your host about it and ask them if they can move the offending website elsewhere.

7. Networking issues: In some instances, we have concluded that high latency was caused by networking issues/DNS issues. Naturally, fixing issues with routing/networking is outside the scope of developers, and is 100% the responsibility of the host. So, check with your host if you think that the sudden slowness on your Joomla website is caused by a networking issue.

We hope that you found this post informative and light, and that it helped you get to the bottom of the sudden performance issue on your Joomla website. As always, we are always here for you if you need professional help. Just contact us and we’ll ensure that the problem on your Joomla website is solved swiftly (well, as fast as we can) and affordably.

If you’re into Joomla plugin development, then you are most likely familiar with the onContentBeforeSave and the onContentAfterSave events. The first event is triggered just before someone saves any Joomla content item, and the second event is triggered (you’ve guessed it) just after someone saves a Joomla content item.

The onContentBeforeSave event is typically used to alter the data that is about to be saved to the database, or to do something just before the data is saved.

The onContentAfterSave event is typically used to do something after the data is saved, for example, redirect to a specific page, or perform one or more database activities.

Now, if you read the first paragraph very carefully, you will understand that there is a problem. Since these events are run when any content item is saved, then this means that they can run at the wrong time. For example, if you have code that should be run when an article is saved, then this code will also run when you save other content items (such as categories or content items of non-core extensions)… In the absolute majority of cases, this is not a desirable behavior. So, in order to avoid this problem, developers add a condition in the beginning of the event to tell it which context it should run in. For example, if a developer wants to have the onContentBeforeSave run only when an article is saved, then he adds the following line to the beginning of the onContentBeforeSave function:

if ($context !== 'com_content.form') return true;

The above line ensures that the event runs only when an article is being saved.

What the absolute majority of Joomla developers don’t know is that there is another, more elegant solution to the problem, and it is by assigning different names to the above events based on the content type. The Joomla core already takes advantage of this hidden functionality: let us take a look at the model of the plugin extension, which is the file plugin.php which is located under the administrator/components/com_plugins/models. In particular, let us look at the constructor of that model:

public function __construct($config = array())
{
	$config = array_merge(
		array(
			'event_after_save'  => 'onExtensionAfterSave',
			'event_before_save' => 'onExtensionBeforeSave',
			'events_map'        => array(
				'save' => 'extension'
			)
		), $config
	);

	parent::__construct($config);
}

You can see that the event_before_save and the event_after_save configuration parameters were set to onExtensionBeforeSave and onExtensionAfterSave respectively. By default, these 2 configuration parameters are set to onContentBeforeSave and onContentAfterSave (this default is set in the admin.php file which is located under the libraries/legacy/model folder). So, when a Joomla plugin is saved, it triggers the onExtensionBeforeSave and onExtensionAfterSave events instead of the onContentBeforeSave and onContentAfterSave events. What does this mean? Well, it means that you can avoid triggering the default before save and the after save events by explicitly setting the values of the event_before_save and the event_after_save configuration parameters in your extension’s model.

We haven’t really discovered fire in this post, but we have demonstrated one best practice in the implementation of Joomla extensions, a best practice that we believe can benefit some Joomla developers out there by saving them some headache.

Now if you, our dear reader, need help with your Joomla events or with the development of Joomla extensions, then please contact us. We are experts in Joomla coding, our work is super clean, and our fees are really affordable!

We love optimization work, there’s always something new to discover, and there’s always something new to learn. Our love for optimization led us to discover a gem, a little optimization gem that literally halved the load of a server powering an extremely large and high traffic Joomla website. As usual, we are always thrilled to share our discoveries with our readers, so here goes…

While investigating load issues on a huge Joomla website that we manage and that we previously optimized, we noticed that the following query was taking a relatively long time:

SELECT a.id, a.title, a.alias, a.catid, a.created, a.created_by, a.created_by_alias, a.created as publish_up,a.publish_down,a.state AS state FROM #__content AS a WHERE a.state = 1 AND (a.catid = 55 OR a.catid IN ('300', '301', '302', '303')) AND (a.publish_up = '0000-00-00 00:00:00' OR a.publish_up <= '2017-09-07 18:09:39') AND (a.publish_down = '0000-00-00 00:00:00' OR a.publish_down >= '2017-09-07 18:09:39') ORDER BY a.publish_up DESC, a.created LIMIT 20

The above query is a modification and an optimization of the core Joomla query that is used to generate the list of articles for pages/modules. The modification consists of displaying articles from multiple categories (instead of a specific category), while the optimization consists of an oldie but goldie optimization process that we first implemented on Joomla 2.5, and that is still valid until today (the website in question is Joomla 3.7.5). Having said that, the heart of the query is still representative of the Joomla core query, which means that any problem in the above query is also a problem in an unmodified and an unoptimized core Joomla article selection query.

In order to learn more about the issue, we went to phpMyAdmin, and we added an EXPLAIN directive to the beginning of the above query, and then we executed it. The query executed was as follows:

EXPLAIN SELECT a.id, a.title, a.alias, a.catid, a.created, a.created_by, a.created_by_alias, a.created as publish_up,a.publish_down,a.state AS state FROM #__content AS a WHERE a.state = 1 AND (a.catid = 55 OR a.catid IN ('300', '301', '302', '303')) AND (a.publish_up = '0000-00-00 00:00:00' OR a.publish_up <= '2017-09-07 18:09:39') AND (a.publish_down = '0000-00-00 00:00:00' OR a.publish_down >= '2017-09-07 18:09:39') ORDER BY a.publish_up DESC, a.created LIMIT 20

Here’s what we saw:

If you look at the key in the above image, you will notice that the MySQL engine chose idx_state (which is the state index) for the key, which is far from ideal. The key should be a combination of all the condition fields and all the order fields, which means that in our query above, the key should consist of the following fields: state, catid, publish_up, publish_down, and created. So, we created an index called idx_articles which consists of all those fields using the following query:

ALTER TABLE `#__content` ADD INDEX `idx_articles` ( `state`, `catid`, `created`, `publish_up`, `publish_down`);

After creating the above index, we re-issued the EXPLAIN query and here’s what we saw:

As you can see in the above image, the MySQL database is now choosing the right index that is fully optimized for the article selection query. Additionally, the number of rows examined has decreased considerably.

So, how did this simple optimization affect the server load?

Marvelously, we can safely say! The load was more than halved. It was at 2.2, and it became 0.8, yes, it was that good! If you think it’s too good to be true, then try it and see for yourself (and let us know)! While we don’t guarantee that you will see the same extraordinary result that we saw (especially if you are using extensive caching on your Joomla website), we have the feeling that you will thank us for this.

Now, if you need help with creating the index or if you didn’t see any noticeable gain after creating the index, then please contact us. We are experts in optimizing Joomla websites, and our fees are super reasonable!

A client of ours wanted to integrate webinar purchasing on her Joomla website. She had a GetResponse account where she creates webinars, and she wanted to give her clients the ability to purchase her GetResponse webinars through her website.

Unfortunately, GetResponse does not technically allow either directly or indirectly the purchase of webinars. So, in order to have someone join a webinar, the person with access to the GetResponse account must manually create the contact (if it wasn’t created previously) and then add the contact to the webinar. Of course, this can only be done after charging the contact for the webinar, which was done manually over the phone. Obviously, we needed to automate all that, and so we leveraged the GetResponse API in order to do it.

Our vision was to have the contact visit a form, that (among other fields) displays a dropdown of all the GetResponse webinars, and then, have a payment method (on the same form) that the contact can use to pay for the webinar. Once payment is made and is successful, then the contact is added automatically to the webinar using the API.

So, the first thing that we did was to generate an API key on the GetResponse website (naturally, you will need to have a valid login information in order to do that).

We then downloaded the GetResponse API Client Library from here, and we modified it to accommodate our needs (we mainly added a few functions). The modified client library can be downloaded here. Once downloaded, we extracted the API file to the api folder (a folder which we manually created under the main directory of the Joomla website).

The next step was to download and install RSForm Pro onto the client’s website, and then download and install the RSForm Pro PayPal Plugin. Once that was done, we started creating the form. The form had the following fields: Name, Email, Webinar, rsfp_product, Payment Type, Total to be paid, PayPal, and Pay for Webinar. Here’s a quick explanation of all the fields:

• The Name and the Email field are self-explanatory. They are both of type Textbox.

• The Webinar was of type Dropdown and it had the following code in the Items field:

  if (!function_exists('getResponseWebinars')){
  	function getResponseWebinars(){
  		require_once(JPATH_ROOT.'/api/getresponse.php');
  		$objGetResponse = new GetResponse('your_getresponse_api_key');
  		$objWebinars = $objGetResponse->getWebinars();
  		$arrResult = array('|Select Webinar');
  		foreach ($objWebinars as $webinar){
  			$arrResult[] = $webinar->webinarId.'__'.$webinar->campaigns[0]->campaignId.'__'.$webinar->name.'|'.$webinar->name;
  		}
  		return $arrResult;
  	}
  }
  
  $ignoreThisValue='<code>';return getResponseWebinars();

  The above code dynamically populates the dropdown with all the webinars created in GetResponse. Note: If you are intrigued by the last line in the above code, then check this post for an explanation.

• The rsfp_product field was of type Single Product and it was used to set the price of the webinar (note: all the webinars had the same price).

• The Choose Payment field was of type Choose Payment, and it contained an automatically generated list of payment methods. In our case, the only payment method was PayPal.

• The PayPal field was of type PayPal, and it was necessary to have PayPal as a payment method. Note that the PayPal payment settings are defined in RSForm Pro‘s configuration page (which can be accessed by going to Components -> RSForm! Pro -> Configuration).

• The Pay for Webinar button was just a submit button.

Once we created the form above, we viewed it (without submitting any data) and it was indeed displaying the webinars in the dropdown.

Now, what remained was to perform the proper actions when a payment is successful. So, we created a plugin called RSFP Payment Success, which implements the rsfp_afterConfirmPayment event, which is triggered when a payment is successful. We added the following code in the rsfp_afterConfirmPayment function:

//get submission information
$db = JFactory::getDbo();
$sql = "SELECT FieldName, FieldValue FROM #__rsform_submission_values WHERE SubmissionId='".$SubmissionId."'";
$db->setQuery($sql);
$arrResult = $db->loadAssocList();
$arrFinalResult = array();
for ($i = 0; $i < count($arrResult); $i++){
	$arrFinalResult[$arrResult[$i]['FieldName']] = $arrResult[$i]['FieldValue'];
}

//now use the API to add the contact and then tag it
require_once(JPATH_ROOT.'/api/getresponse.php');
$objGetResponse = new GetResponse('your_getresponse_api_key');

$strName = $arrFinalResult['Name'];
$strEmail = $arrFinalResult['Email'];
$strWebinar = $arrFinalResult['Webinar'];
$arrWebinar = explode('__', $strWebinar);
$strWebinarID = $arrWebinar[0];
$strCampaignID = $arrWebinar[1];
$strWebinarTitle = preg_replace("/[^A-Za-z0-9]/", '', $arrWebinar[2]);

// Add the contact and associate it with a campaign
$result = $objGetResponse->addContact(array('email'=>$strEmail, 'name'=>$strName, 'campaign'=>array('campaignId'=>$strCampaignID)));

//Add the webinar name as a tag if it doesn't exist
$webinarGetResponseID = $objGetResponse->getOrCreateTagByName($strWebinarTitle);
$resultUpdateContact = $objGetResponse->updateContact($contact_id, array('tags'=>array('tagId'=>$webinarGetResponseID)));

The above code creates the contact if it doesn't exist, then it creates a tag generated from the title of the webinar (also if it doesn't exist), and then tags the contact with the webinar tag. Unfortunately, GetResponse doesn't have any means to add a contact directly to a webinar through the API, hence the workaround with using tags (this means that a slight manual work is required by our client in order to add all contacts with a certain tag to a certain webinar).

Have we had any hiccups with the integration?

Yes - we did. The first major hiccup is the one that we just mentioned, and it is the fact that GetResponse does not have an API method to add a contact to a webinar. We overcame this problem by resorting to tags. It wasn't the most elegant solution, but it was sufficient to the client.

The other major hiccup that we had, is that when a contact is first created in GetResponse, he must verify his account by clicking on a link in a welcome email. If he doesn't do that, then a contact ID will not be generated, and the tagging won't work. This problem, however, was somehow erratic, but, in order to avoid it altogether, we added a row in a table called #__cron_tag_contacts containing the email of the contact and the tag ID when a payment was successful. We then created a small file called tag_contacts.php, which was run by a cron every 15 minutes, and that checked the table for any contacts that were properly verified (e.g. they had an ID in GetResponse). Once a contact gets verified, the script tags it and then deletes its associated row from the table.

We hope that you found this post useful, and that it helped you with your integration with GetResponse. We have tried to provide as much information and guidance as we can about the subject, but, we reckon that the complexity with this integration is a bit high and that some readers may need help with the implementation. If you're one of those readers, then fear not, we are here for you. Just contact us and we'll implement this for you quickly, efficiently, and affordably!

Most administrators check the cron jobs that they have on their server by just logging into their cPanel and clicking on the Cron Jobs link under the Advanced section.

The more advanced administrators check their cron jobs by typing the following command in the shell:

crontab -e

So, in the absolute majority of cases, an administrator does not directly open a physical file to check the cron jobs, he checks them through an interface whether through cPanel or the Linux crontab (or a different interface altogether). Now, the question is, where does the cron information come from? In other words, where is it stored?

Well, on a CentOS server (which is the typical server used for a WHM environment), cron jobs are stored in files under the directory /var/spool/cron. Each file is named after a user (each user is typically the username of a cPanel account), and each file contains the cron jobs for that particular user.

Now, you might be wondering, is there any point to this? Why should anyone check the actual physical file instead of using an interface to view the cron jobs? Well, because, in some cases, some information is not displayed in the interface, and, in most of these cases, this information is malicious. Let us give you an example…

Yesterday morning we were hired to add a cron job to a Joomla website, and, for some reason, we were having problems creating this cron job using the cPanel interface and using the crontab -e command, so, we went straight to editing the physical file containing the cron. Here’s what we did:

• We ssh’d to the server as root.

• We opened the file cpanel-user which is located under the /var/spool/cron folder (you will need to change cpanel-user to your cPanel username) for editing the following way:

  vi /var/spool/cron/cpanel-user

• We saw this code:

  SHELL="/bin/bash"
MAILTO=""
DOWNLOAD_URL="http://[malicious-domain].com/u/w.gz"
LOCAL_FILE_PATH="/home/[cpanel-user]/public_html/libraries/joomla/client/client.php"
TMP_DIR="/var/tmp"

As you can see in the above code, only the first line is benign, the remaining lines are malicious. On the bright side, however, the code had no effect because it was missing the actual cron job. Its seems that the Joomla website we were working on had a cron job hack identical to the one we described here (a malicious file was being downloaded from DOWNLOAD_URL, and was being extracted to the client.php core Joomla file), and whoever fixed it didn’t fully cleanup the cron file (he just removed the cron job from the interface). So, we cleaned up the file and, after doing that, we were able to add the cron job through the interface.

So, it’s always a good idea to regularly check the contents of the files located under the /var/spool/cron folder. If you find anything suspicious and you are not sure what to do, then please contact us. We are experts in Joomla security, our work is quick, and our fees are always affordable!

A regular client of ours came to us yesterday evening with an interesting task: she wanted to set the Last-Modified HTTP header on an article page to the actual modified date of an article. We thought it was an easy task, until we started working on it.

You see, by default, Joomla sets the Last-Modified HTTP header to the current date, unless of course, it is using caching, in that case it sets the Last-Modified HTTP header to the cache time of the article. Logically, the Last-Modified date should be the last modification date of the article, but, in most scenarios, nobody cares about this. However, our client was using a tool which relied on the Last-Modified HTTP header in its data collection and display.

So, we dug in the code to find where the Last-Modified HTTP header was set, and we quickly (well, not so quickly) discovered that it was in the file web.php which is located under the libraries/joomla/application folder. In fact, the whole thing was done in this line:

$this->setHeader('Last-Modified', gmdate('D, d M Y H:i:s') . ' GMT', true);

As you can see in the above line, the Last-Modified HTTP header is set to the current date and time; it completely ignores the modification time of any item the user is currently viewing. If we want to change the Last-Modified value to the actual modification time of the article, we’ll have to replace the above line with the following code:

$currentView =JFactory::getApplication()->input->get('view');
$currentOption =JFactory::getApplication()->input->get('option');
if ($currentView == 'article' && $currentOption == 'option'){
	$currentArticleID = JFactory::getApplication()->input->get('id');
	$sql = "SELECT `modified` FROM `#__content WHERE `id`='$currentArticleID'";
	$db->setQuery($sql);
	$strDateModified = $db->loadResult();
	$objDateModified = new JDate($strDateModified);
	$strDateModified = $objDateModified->format('D, d M Y H:i:s').' GMT';
	$this->setHeader('Last-Modified', $strDateModified, true);
}
else
	$this->setHeader('Last-Modified', gmdate('D, d M Y H:i:s') . ' GMT', true);

The above code will work, however, there is one little problem with it: it is a core modification, and that’s why, at itoctopus, we have devised a better method to do this job, we have developed a plugin, and you can download it here! All you need to do is to just download the plugin, install it, enable it, and that’s it! Your article pages will display the actual last modification date of the article, and not the current date.

Now, here are some FAQs about the plugin:

• Is it really free?

  Yes – it is free. You can use it anywhere and for anything. You can modify it and do anything with it. An attribution to itoctopus would be nice, but not necessary.

• Is it supported?

  No – we do not offer free support for any of our extensions, since all of our extensions are free. If you would like support for any of our extensions then please keep in mind that our fees apply.

• Does the plugin only work with Joomla articles? Can it work with K2 items?

  As it is, the plugin only works with Joomla articles, but, it can easily modified to accommodate other types of content items, such as K2 items. If you need help in this area, then please contact us.

Finally, we would like to stress the point that the plugin is provided as is, without any explicit or implicit warranty/responsibility. It may or may not do what we claim it does, and it may or may not destroy your website or our entire solar system if used, we don’t know, so use at your own risk! Again, we don’t assume a shred of responsibility for this plugin, so use at your own risk!

Oh, and by the way, we have a quick note to the Joomla team, the onAfterRespond can never ever be triggered. Please check the code in the respond function; it took us ages to discover that this event does not work!

Note: The solution in this post is a modification to a core file. Proceed at your own risk, and keep in mind that a Joomla update may wipe out your changes.

We got a request from a client of ours to display articles from multiple categories on an existing category blog page. As a Joomla administrator, you probably know that a basic installation of Joomla can’t do this. In fact, this is the main reason (in most cases) why a Joomla website uses K2 for content management instead of Joomla’s core.

Luckily, we have done this before for another client, so we processed the request swiftly. Here’s what we did:

• We logged in to the Joomla backend and we opened the existing menu item pointing to the category blog page.

• We checked the string in the Link field, which was this one:

  index.php?option=com_content&view=category&layout=blog&id=50

• We took note of the number “50” (which is the ID of the category) in the above string.

• We then gathered the IDs of all the categories which the client wanted their articles displayed under the above category blog page. Here are the IDs: 77, 81, 107, 155.

• We opened the file articles.php which is located under the components/com_content/models folder.

• Just below the following line:

  $categoryEquals = 'a.catid ' . $type . (int) $categoryId;

  We added the following line:

  if ( (int) $categoryId == '50'){
  	$categoryEquals .= ' OR a.catid IN (\'71\', \'81\', \'107\', \'155\')';
  }

  Note: Of course, you will need to replace “50” with your category number. You will also need to replace “71, “81”, etc… with the IDs of the additional categories you want to display articles from.

• That’s it! After doing the above, the category blog menu item displayed articles from all the above categories.

Note that you will need to create hidden menu items of type Category Blog, each pointing to a category in the above list to ensure that SEF links are properly generated for your articles.

But, what about using subcategories to display articles from multiple categories?

Well, you can do that, provided the multiple categories are immediate children of the main category (e.g. the category with ID “50” in the case of our client). If this is the case, then all you need to do is to open the menu item pointing to the main category, and then click on the Blog Layout tab, and finally set the value of Include Subcategories to “Yes”.

However, if you are stuck with dispersed categories that are not children of the main category, then the quickest solution would be to use our core modification above.

We hope that you enjoyed our simple post for displaying articles from multiple categories on a category blog. If you need help with the implementation, then just contact us. We always love having more clients, our work is super fast, our expertise is top notch, and our prices are super affordable!

Note: The second solution presented in this post consists of a core modification, which means that a future update can overwrite your changes. Always document your core modifications so that you can re-apply them after an update.

During the past weekend, we updated the website of one of our clients to Joomla 3.7.5 from Joomla 3.6.5 as we thought the former was stable enough. Yesterday, the client emailed us and told us that whenever they upload a PDF file through the media manager, they were seeing the following error…

Invalid mime type detected.

…and, of course, the upload was failing.

We investigated the issue and it turned out that there was a change in Joomla’s 3.7.x file checking process: In Joomla 3.6.5, a function called canUpload (in the media.php file which is located in the libraries/cms/helper folder) checks if the file uploaded is legal (e.g. it is allowed). If it is, then it allows the upload. In Joomla 3.7.x, the same function, canUpload, is triggered, but, in a default scenario, it also tries to guess the MIME type of the uploaded file using the function getMimeType. If it can’t (for any reason), then it returns false, and will display the above error.

Now the question is, what is a MIME type?

MIME is acronym for Multipurpose Internet Mail Extension, and it is just a string representing the type of a file and is typically used by the browser to know which type of software to launch to display a certain file. For example, if the MIME type of a file is application/pdf, then the browser will know that it needs to open the Acrobat Reader software to display the file. On the server end, the MIME type is often used to know the type of the file being uploaded, in order to decide whether to allow the upload or not.

However, the PHP function that is used to return the MIME type, which is mime_content_type, is an unstable function. In fact, this function was deprecated and then un-deprecated. There is, however, another method to return the MIME type, and it is by using the function finfo_open to return a fileinfo object out of a file and then using the function finfo_file to return the MIME type of the file. Now the problem with the last method is that it requires the PECL PHP library installed, which is not installed by default. The function getMimeType which is called by the function canUpload, first uses the first method to determine the MIME type of the uploaded file, and then it uses the second method. If it can’t determine the MIME type of the uploaded file using either method, then it returns false, leading to the Invalid mime type detected error.

So, what is the solution to the problem?

Since the MIME type check is done only when the Restrict Uploads setting in the Media Manager configuration is set to “Yes”, then a very quick solution would be to change that setting to “No” by logging in to the backend of the Joomla website, and then going to Content -> Media, and then clicking on Options on the top right, and then flipping the value of Restrict Uploads from “Yes” to “No”. Now, if someone hovers on Restrict Uploads, then he will see the following message:

Restrict uploads for lower than manager users to just images if Fileinfo or MIME Magic isn’t installed.

The problem is, however, is that the above message isn’t accurate. In fact, when Restrict Uploads is set to “Yes”, then it’ll be applied for all users, even super users, which is wrong. There is no check anywhere for the user type in the actual code. In order to fix this bug (which will also fix the original problem and will address any security implications resulting from setting the value of Restrict Uploads to “No”), you will need to do the following (note that if you change the Restrict Uploads value to “No” then you don’t need to do the below):

• Open the file media.php which is located under the libraries/cms/helper folder.

• Change the following line:

  if ($params->get('restrict_uploads', 1))

  to the following line:

  if ($params->get('restrict_uploads', 1) && !JFactory::getUser()->authorise('core.admin'))

• Save the file and upload it back.

• The problem should be solved!

Of course, there is another solution that consists of installing the proper PHP libraries, but this solution is dependent on the type of the hosting environment, so check with your host/system administrator about this (have them install the PECL PHP library or the MIME Magic library).

We hope the you found our post useful and that it did solve your problem. If it didn’t, then please contact us, we’ll help you address this problem for you in no time and for a very, very reasonable fee!

Unlike WordPress, Joomla has the unique ability that allows an administrator to label an article as a featured article, which means that the article will display on a page where the menu item is of type Featured Articles (typically, this page is the homepage). You might think, well, that’s not so special, but there is more to this: Joomla allows the administrator to have a different order for featured articles, and that order is independent from their order within their respective categories.

If you’re the skeptical type, then you are probably thinking: “Well, OK! But that’s not really much!” And it isn’t, except that the underlying database structure and the code gets signficantly messy because of this. You see, in order to support the above functionality, Joomla queries an additional table on a page with a Featured Articles menu item (which, again, is typically the homepage), which creates a huge load on the server when that table is of considerable size. The solution to this problem is to regularly trim the #__content_frontpage table and ensure that it doesn’t contain more than 100 entries. This can be done the following way (please backup your database before proceeding further; if you don’t want to then at least backup the #__content_frontpage table):

• Login to phpMyAdmin.

• Click on the #__content_frontpage table on the left pane (of course, you will need to replace #__ with the database alias of your Joomla website).

• Click on the SQL tab on the top and type in the following query:

  DELETE FROM `#__content_frontpage` WHERE `ordering` > 100;

• Click on the Go button on the bottom right.

• That’s it!

Once you do the above, you will feel a noticeable gain particularly if your #__content_frontpage table is quite large.

We hope that you found this post useful. If you need help with the implementation, or if you need to implement the above in a cron, then please contact us. We are always happy to serve, our prices are super affordable, and our work is super clean.

A few days ago, a client of ours emailed us and told us that some of the pages on her company website were displaying content that was not theirs. The content was not technically malicious, but it consisted of ads, which had nothing to do with the client’s business. Clearly, the website was hacked.

Since only a few pages were affected, we had a hunch that it was a database hack, so we searched the database for the modified content, and, oddly enough, we didn’t find the modified content in the #__content table, but we found it in a table called #__finder_data. We’ve never seen this table before. To the untrained eye, the table seemed to be part of the Smart Search extension, but, as Joomla experts, we knew that it couldn’t be the case, as the Smart Search does not use any such table.

So we looked at the content of the table, and we noticed that most of the rows were benign and were just a copy of some rows in the #__content table. A few rows, however, had some altered (hacked) content.

We quickly (and wrongly) assumed that the #__finder_data table was used by a 3^rd party extension for some caching reasons (although it seemed pretty useless, as it was just a copy of the #__content table), but we needed to know which extension it was used by. So, we did a grep on the filesystem which quickly revealed that the table was used in the file query.php which is located under the libraries/joomla/database folder. Yes, you’ve guessed it, the query.php is a core file, which means that the whole #__finder_data is likely part of the hack. So, we opened the query.php, and we searched for finder_data, and we found the following code:

if (!isset($GLOBALS['issebot'])) $GLOBALS['issebot']=$this->is_sebot();
if ( @$GLOBALS['issebot'] AND preg_match('/#__content(\s|$)/', $tables) AND in_array($db_px.'finder_data', $this->db->getTableList() ) ) {
    $tables=str_replace('content', 'finder_data', $tables);
}

The above code was located in the from function, and it checked if the traffic was coming from a bot (the method is_sebot was also added to the query.php file), and if it was, then it displayed content with matching ID from the #__finder_data table. Clearly, the method to check if the traffic was coming from a bot or not wasn’t exactly working properly, as the website was displaying the modified content for regular, non-bot traffic.

We quickly did the following:

• We overwrote the hacked query.php file with a clean query.php file from a matching Joomla version (actually, we used our super fast method to clean Joomla sites and overwrote the whole core).

• We deleted the #__finder_data table as it had nothing to do outside the hack.

Doing the above solved the problem, but, of course, it didn’t answer the question, how did this happen in the first place even though the site was using a very secure version of Joomla? We don’t know for sure, but we suspect the host. We’re not saying that the host did this, but what we are saying that the host that the client uses is notorious for bad security (the host has the word giant in its name, you figure the rest).

If your Joomla website displays (even slightly) different content than the real content, then likely your website is hacked. Try our simple cleanup instructions above and see if that fixes the problem. If it doesn’t, then please contact us. We will clean your website, we will secure your website, and we won’t charge you much.

One of the nice features in Joomla is that you can update the CMS from either the backend or from the filesystem. We typically update our clients’ websites from the backend, but, when we encounter the minor of issues, we update it using the filesystem. Here’s how:

• We ssh to the server as root and we cd (change directory) to a temporary directory.

• We download the latest Joomla install using wget.

• We extract the install in the temporary directory and then we remove the installation folder (and the templates folder if the site uses a built-in Joomla template).

• We zip everything back and we move it to the root directory of the Joomla site.

• We extract everything there and then we use a recursive chown to change the ownership and the group of each of the files to the right one.

• We login to the Joomla backend and then we go to Extensions -> Manage -> Database and then we click on the Fix button on the top left.

• That’s it! Typically, the above process takes a few minutes, and we only do it if the current Joomla website is a few revisions behind.

Lately, however, we are having a minor hiccup with the above process (especially when updating from Joomla 3.6.5 to Joomla 3.7.x), as we are seeing the following warning when we go the Article Manager page (or any backend page having to do with the articles):

“Error loading component: com_fields, Component not found.”

The above warning is caused by the fact that the new article system in Joomla has been modified to use the custom fields (which has been introduced in 3.7.x), but the custom fields were not properly installed onto the website. The solution to this problem is to use the “Discover” functionality in the Joomla backend. Here’s how to do this:

Login to the Joomla backend and then go “Extensions” -> “Discover”, and then click on the “Discover” button on the top left. Once you do that, you should see the following Fields extensions that should have been installed on your Joomla website (but were not):

• Content – Fields (Site Plugin)
• Fields (Administrator Component)
• Fields – Calendar (Site Plugin)
• Fields – Checkboxes (Site Plugin)
• Fields – Colour (Site Plugin)
• Fields – Editor (Site Plugin)
• Fields – Imagelist (Site Plugin)
• Fields – Integer (Site Plugin)
• Fields – List (Site Plugin)
• Fields – Media (Site Plugin)
• Fields – Radio (Site Plugin)
• Fields – SQL (Site Plugin)
• Fields – Text (Site Plugin)
• Fields – Textarea (Site Plugin)
• Fields – URL (Site Plugin)
• Fields – User (Site Plugin)
• Fields – Usergrouplist (Site Plugin)
• System – Fields (Site Plugin)

Select them all by clicking on the top checkbox (the one next to “Name”). Once you have them all selected, click on “Install” button on the top left. When the install finishes (it should take a few seconds), you should see the following message Discover install successful. You can then verify that the problem is fixed by going to the Article Manager (you shouldn’t see the warning any more).

We hope that you found our little post useful and we hope it helped you solve your problem. If it didn’t, then please verify that your server doesn’t have any type of aggressive caching. If you still need help, then you can always contact us. We are eager for new clients, our current clients love us, and our fees are always right.

A few weeks ago, we discussed the main advantages that WordPress has over Joomla. One of these advantages is the fact that WordPress has full time developers and a huge testing team. Of course, for those of us working with Joomla, we know that this isn’t the case for our beloved CMS.

Joomla is supported by volunteer developers (who are highly dedicated) and by a volunteer testing team. Now any programmer worth his salt knows 3 things: 1) he cannot say that his code is “fully tested” when he’s the only one who tested it, 2) any code should never ever be made live without thorough testing, and 3) a simple line of code may unknowingly impact many areas of a product.

If you know how development on the Joomla core works, then you will know that the heart of the problem is the word “thorough” in the previous sentence, simply because the testing is anything but thorough. Let us explain the issue with an example of how a bug is handled:

• Someone reports a bug to the Joomla development team.

• A Joomla developer addresses the bug.

• Other developers test the feature that the bug fix addressed.

• The bug fix is packaged and prepared for the next Joomla release.

• The Joomla release is tested shortly before its launch. The testing consists of installing the new Joomla version with some demo data onto someone’s PC and tinkering with that install.

Obviously, there are some serious flaws in the above process:

• The bug fix may cause issues in areas other than the area addressed by the bug.

• The testing done does not take into consideration the impact of the bug fix on the product as a whole.

• The testing is oblivious to the concept of “3^rd party extensions” in Joomla and is done on a pure Joomla website.

• The testing completely ignores the impact of this simple bug fix on large datasets, because the demo data that is used for the testing consists of just a few articles.

We can give you a proof for each of the points mentioned above. For example, a few revisions ago, Joomla changed the ordering algorithm of articles. The change was meant to address a performance issue caused by the old article ordering algorithm. Some developers tested the change and it was made live in the next revision. Less than an hour after the release, many Joomla administrators flooded the developers with complaints about the new ordering algorithm, as it confused them by pushing newer articles to the very bottom of the articles’ list. Obviously, a consensus was reached shortly after and the change was reverted.

There are many, many other examples, but we don’t think it’s necessary to list them all to get our point across.

But what is the solution?

The solution is for Joomla to hire a full testing team, and to give testing a priority, and to test with real data (e.g. copy real large websites onto a dev environment and then test these websites there). Obviously, more money is needed to accomplish this, and so Joomla must be slightly less conservative when it comes to its tight techniques to raising money.

We hope you found this post useful. If you have any questions about it, then please leave a comment below. If you need help with your Joomla website, then please contact us. Our work is clean, our skills are solid, and our fees are affordable!

A new client called us yesterday afternoon and told us that his company website was displaying a weird page instead of his normal page (his company sells restaurant equipment). The weird page consisted of some gibberish in random languages. It was a no-brainer that the website was hacked.

Since the client was in a hurry, we decided to clean the website using our super quick method for cleaning Joomla websites, but, to our surprise, it didn’t work: the website still displayed the page with gibberish content.

We thought, it might be that the template chosen was hacked, so we switched to the Beez 3 standard Joomla template (that was definitely fixed by the code overwrite), but that didn’t work either.

Naturally, the next step was checking the .htaccess file, maybe there was a malicious rule somewhere, but even after deleting the .htaccess file the site was still hacked. We also checked for malicious .htaccess files in higher directories, and we found none.

Finally, we decided to add a PHP die(‘under maintenance’); statement to the beginning of the main index.php file and see how the website reacts. To our surprise, the website still displayed the hacked page.

There were 3 scenarios in our mind:

1. The website was located in a different directory than the one we were working on.

2. The website was located on another server than the one we were working on.

3. The server hosting the site had some aggressive caching techniques.

While checking for the above possibilities, we noticed something peculiar: there was an index.html file lurking in the root directory of the Joomla website. We opened the file in our text editor, and, unsurprisingly, the file contained the hacked HTML code. Deleting the index.html file immediately solved the problem.

But how did the hacked index.html file got there in the first place?

We’re not exactly sure. The website was using the latest Joomla version (Joomla 3.7.3) and all the 3^rd party extensions were up to date, so, most likely the issue was an undiscovered vulnerability in one of the installed extensions. The client elected not to proceed further with the investigation so we can never be sure.

So, how can one protect his website from a similar issue?

Addressing the result (and not the cause) can be done by adding some .htaccess rules to block access to the following files in the root directory of the Joomla website:

• default.html
• default.php
• index.html

A simpler way to address the problem consists of adding the following line to the very beginning of the .htaccess file:

DirectoryIndex index.php index.html default.html default.php

The above line ensures that the index.php file has priority over all the other files and is always processed first by the web server.

We hope that you found this post useful. If you have some questions about it or if your website ran into the same issue and you want us to do some forensic work done in order to unveil the root cause of the problem, then please contact us. We are security experts in Joomla, our work is very clean, and our fees are extremely affordable.

WordPress, the most used CMS in the world, with around 9 times the market share of Joomla, is not better than Joomla. In fact, we think that Joomla is a much better product, and this is why:

• Joomla uses MVC in its code, WordPress doesn’t.

  The term Spaghetti Code is a great fit for WordPress. Joomla has always adopted a clean Model-View-Control framework which is very easy to work with.

• Joomla is way better structured than WordPress.

  In WordPress, everything is called “a plugin”, with the exception of templates, which are called “themes”. In Joomla, you have plugins, modules, and components.

• Joomla is generally a more polished product.

  Compare a Joomla backend to a WordPress backend and you will know exactly what we mean.

But, even though we think that Joomla is better than WordPress, we cannot ignore the fact that Joomla lags behind when it comes to market share. We believe that this can be attributed to the following reasons:

• Updating WordPress from almost any version to the most current version is a seamless operation.

  Case in point: A year ago we updated the itoctopus website (yes, guilty as charged, we use WordPress on the itoctopus website, mainly because our website is mostly a blog) from a very old version of WordPress to the most recent one, and it literally took us a few minutes to do so. Imagine trying to migrate a Joomla 1.5 website to Joomla 3.7.2, will you be able to do it in a few minutes? We can’t, and if you can, then please send over your CV right now!

• SEF links are automatic with WordPress.

  In Joomla, you have to create menu items, and then point these menu items to articles/categories/etc… in order to get a working link. Sure, you have full control over your links in Joomla, but this extra process is what keeps many webmasters from adopting Joomla. Not many people grasp the idea of menu items. On the bright side, however, Joomla will, in the near future hopefully, have a seamless integrated SEF system.

• WordPress is a dictatorship.

  WordPress was created by one person, and that one person has a final say in almost any decision involving WordPress, including technical and financial decisions. Essentially, WordPress represents the unwavering vision of just one person. Joomla is a totally different story. Joomla’s financial, structural, and technical decisions are taken by different people with different opinions and different agendas. Naturally, this means that many of these decisions are incoherent and incompatible with each other, and what makes things worse is that the Joomla decision makers are often replaced, further weakening the long term vision of the product.

• WordPress is wealthy.

  WordPress employs many monetization techniques that generates a lot of money. Joomla, because of its complicated decision making process (and because of some people with hidden agendas), was never really monetized properly. In fact, the same people making a killing out of Joomla are the staunchest opponents to a real monetization of the Joomla brand.

• WordPress has real – full time developers. Joomla doesn’t.

  Since WordPress is wealthy, then it can afford to hire real good quality developers to support its products. So far, Joomla doesn’t have full time paid developers, it only has volunteer developers. Even the lead Joomla developer is a volunteer developer. Of course, that it not a bad thing, but it is definitely not as good as when you have full time developers supporting the product, and a huge testing team testing the product.

• The WordPress plugins directory is better maintained than Joomla’s JED

  The JED (Joomla Extensions Directory) is just non-comparable to the WordPress plugins directory, and this is because the JED is maintained by very few people. So, the approval, rejection, and the testing of an extension takes a long time and is often done just to “empty the plate”. This means that in many cases, no real testing is done, and this is especially the case for large extensions.

Due to the above factors, WordPress has a much larger community than Joomla, but, on the other hand, Joomla’s community is much more fanatic about Joomla (which can be a good thing and a bad thing). We know that Joomla is a better product in its core, but, we also know that without addressing the above advantages that WordPress has over Joomla, our beloved CMS will, at best, be in second place.

The above post reflects our opinion which is based on our experience with both products. If you have have something to say about this, then feel free to comment below. If you want to hire us to advise you on which CMS is best for you, then please contact us. Our fees are right, our work is honest, and we really know what we are talking about!

One of the common bottlenecks on any website, and not just a Joomla website, is large database tables. Large tables, especially those with significant and constant CRUD (Create, Read, Update, and Delete) activities, can cause serious load issues on websites. This means that it’s always a good idea to check your Joomla websites for large tables and see if you can trim them.

For example, if you are using Smart Search, then you will notice that adding/editing Joomla (or K2) articles gets slower and slower with time, and this is caused by the exponential growth of the finder tables (check point #2 here for more information about the harm that is caused by Smart Search).

Another example is the ACYMailing tables, especially those that are used for storing stats. If you are an avid user of ACYMailing, then you probably know what we mean: a load spike when you are sending out emails (because of the new rows added to the ACYMailing stats table), and a load spike when people start opening their emails (all these database updates happening during a relatively short period of time).

Now, the question is, how can you check your database for large tables? Simple! You just go to phpMyAdmin, select the database powering your Joomla website, and then click on the Rows header. This will display the tables sorted by the number of rows descending (e.g. the tables with the most rows will display first). Once you know which tables have the most number of rows, you will be able to formulate a strategy in order to address that (of course, your options are very limited for some tables, such as the #__content table).

If you need help with those large tables, then all you need to do is to contact us. Our prices are right, our work is professional, and we always have a solution for any Joomla issue.

A new client emailed us back on Saturday (today is Monday), and told us that he was seeing the famous (and dreaded) Invalid Token error whenever he tried to login to his Joomla website. He told us that the whole thing happened all of a sudden. He assured us that he didn’t install a new extension, he didn’t modify the settings of an extension, and he didn’t even add/update anything on his website in the past few weeks.

Of course, our first guess was that the website was hacked, so we did a thorough scan to the website, and it was as pure as the driven snow. Our second guess was a recently installed plugin, but there were none. So we disabled all the non-core plugins, and we still had the problem. Our third (and obvious) guess was Joomla caching, but there was no type of Joomla caching enabled.

So, we started debugging the problem by echoing messages in core files (just to narrow down the problem), and while doing that, we noticed something interesting, our changes were not reflecting immediately. Could it be? So, we did additional tests, and we were able to confirm that aggressive caching was being used by the host. So, we called the host and we were told that Varnish Caching was being used for that website, and so we asked them to disable it (they reluctantly agreed). Once Varnish Caching was disabled, everything worked perfectly, and the Invalid Token issue was resolved.

If you are having an Invalid Token error when you are trying to login to your website, then check our previous post about it. If it doesn’t work for you, then check with your host whether they are using server caching (such as Varnish Caching) for your website. If they do, then ask them to disable it. If they refuse to disable it, then maybe it’s time to move to a new host. If the problem has nothing to do with server caching, then all you need to do is to contact us. We are eager to help, our fees are super affordable, and we always find a solution.

Every few months, we get a request to execute a Joomla administrator view from a cron job. For example, a recent request consisted of running an administrator view from a cron job in order to grab data from other sites and save them into the Joomla website. Now, if you have dealt with cron jobs before, you will probably immediately answer that it’s possible, but (as always), there is a catch: How will you run the administrator view without logging in? If you can do that, then there is a major security exploit in Joomla that must be addressed immediately, but you can’t, at least not before reading this post!

So, how do we do run a Joomla administrator view from a cron job at itoctopus?

Well, we do it the following way:

• We login to the backend of the Joomla website and then we create a super user called admincron with a random password.

• We leverage the power of the defines.php file – that is also loaded by the index.php file located under the administrator folder to do the following:

  • Temporarily disable any security plugins (such as AdminTools, RS Firewall, etc…) by renaming the plugin’s folder to a different name.

  • Load the Joomla environment.

  • Login the user admincron to the Joomla backend.

  • Execute the view.

  • Halt further execution of the script.

• We create the cron job to execute the administrator view.

Here’s the code that we add to the defines.php file:

<?php 
	$hash = $_GET['hash'];
	if ($hash == 'averystronghash'){

		// Optional: Disable any firewall plugin by renaming its folder to *_old
		rename('../plugins/system/firewallplugin', '../plugins/system/firewallyplugin_old');

		//Load the Joomla environment
		if (!defined('_JDEFINES'))
		{
			define('JPATH_BASE', __DIR__);
			require_once JPATH_BASE . '/includes/defines.php';
		}

		require_once JPATH_BASE . '/includes/framework.php';
		require_once JPATH_BASE . '/includes/helper.php';
		require_once JPATH_BASE . '/includes/toolbar.php';
		$app = JFactory::getApplication('administrator');

		// Login the "admincron" user to the Joomla backend
		$credentials = array();
		$credentials['username'] = 'admincron';
		$credentials['password'] = '[password]';
		$app->login($credentials);

		$app->execute();

		// Optional: Re-enable any firewall plugin that was disabled at the beginning of the script
		rename('../plugins/system/firewallplugin', '../plugins/system/firewallplugin');

		// Halt further execution of the script
		die();

	}

?>

Once you add the above defines.php file, you will be able to load any administrator view on your Joomla website without explicitly logging in. You will also be able to load any administrator view from the cron, by simply adding the following task to your cron job:

/usr/bin/wget "http://www.[yourjoomlawebsite].com/administrator/index.php?option=com_[yourcomponent]&view=[yourview]&hash=averystronghash"

Aren’t there any security implications for doing the above?

Not as long as you are using a very strong hash. Otherwise, you will risk allowing anyone who knows the link (in the cron task above) to login as super user to your Joomla website.

We hope you found this post informative and useful. If you have any questions about it, or if you need help with the implementation, then please contact us. We are always excited for working with new clients and our fees are very affordable!

A new client called us this morning and told us that the was seeing the following error on his website when he was visiting it:

An error has occurred. 500 JHtmlBehavior::polyfill not found

So, we visited his website in order to check what is going on, and, to our surprise, we didn’t see the error. We deleted the Joomla cache of the website and we asked the client to clear his browser cache and check the site again, but he still saw the problem. We starting suspecting that it was a propagation issue and that the client was seeing a different version of the site, but, after asking the client to ping his website, we realized that he was seeing the same website as us.

So, we dug into the Joomla code searching for the polyfill error – and we quickly discovered something very interesting in the file behavior.php which is located under the libraries/cms/html folder. It was these 3 lines:

// Include core and polyfill for browsers lower than IE 9.
static::core();
static::polyfill('event', 'lt IE 9');

Aha! It seems that the polyfill library was only being loaded if the person was using an IE browser that is less than Internet Explorer 9 (the client was using IE 8), but, why was it generating the error when Joomla tried to load it?

Further digging revealed that the polyfill library in Joomla 3.7.2 and in earlier versions does not support PHP 7 – it only supports PHP 5.5 and PHP 5.6, and since the client was using PHP 7, he was seeing an error.

So, what was the solution to the problem?

The solution to the problem was quite easy, all the client had to do was to use a different browser (he went with Google Chrome), and the problem was solved!

But what if the client didn’t have the option to switch to another browser?

Well, in that scenario, the client had to comment out the following line from the aforementioned behavior.php:

static::polyfill('event', 'lt IE 9');

Although we have to say, that commenting the above line may have other implications (we haven’t tested it to confirm) – so, as usual, proceed at your own risk.

We hope that you found our post helpful. If you are still seeing “JHtmlBehavior::polyfill not found” error on your website even after switching to another browser, then please contact us. We will always find a solution, our rates are super affordable, and our work is super clean!

At itoctopus, one of the things we are fascinated by is security – we are always researching new ways to improve the security of our managed websites. Our Joomla honeypot experiment, for example, was a huge success and we implemented it for our major clients. It also helped us better understand attack patterns, which, in turn, helped us develop the appropriate counter measures.

A common pattern in attacks on Joomla websites, as we noticed from our Joomla Honeypot Experiment, is long URLs. When a URL is abnormally long, then it’s almost a certainty that it’s an attack, and not a legitimate visit. An obvious counter measure would be to block long URLs from accessing the website. But how? And, how about false positives resulting from this aggressive security measure?

Well, the first thing that needs to be done is to check the Apache access logs, and generate a list of all the URLs sorted by their length. Here’s how (we are assuming that you are using WHM/CentOS as your hosting platform):

• ssh to the server as root.

• Change to the /usr/local/apache/domlogs by issuing the following command:

  cd /usr/local/apache/domlogs

• Run the below code (substitute yourjoomlawebsite.com with your domain):

  awk '{print $7 }' /usr/local/apache/domlogs/yourjoomlawebsite.com | sort -nr | uniq -c | awk '{ print length($2) " " $2;  }' | sort -nr -k1 > urls-by-length.txt

  Note: The above code may take some time on high traffic websites, so please be patient.

• Open the file urls-by-length.txt, and you will see a list of all the URLs accessed, reversely sorted by length. The length of each URL will be printed next to it.

• Take note of the length of the longest legitimate URL that you have on your website (you will find it at the top of the list, not necessarily in the first row, as the first row may contain an attack URL). You will need it afterwards.

Now that you have the length of the longest legitimate URL on your website, you can create a simple condition in the defines.php file to block URLs that exceed the maximum legitimate length. Here’s how:

• Open or create the file defines.php under the root directory of your Joomla website.

• Add the following code to it:

  <?php
  $maximumAllowedCharacters = [number you got from running the awk command above + some padding];
  // Replace 'http' with 'https' if your website runs in SSL mode
  $currentURL = 'http://'.$_SERVER['HTTP_HOST'].$_SERVER['REQUEST_URI'];
  if (strlen($currentURL) > $maximumAllowedCharacters){
  	header('HTTP/1.0 403 Forbidden');
  	die('Unauthorized');
  }?>

• That’s it! Now abnormally long URLs will be blocked from accessing your website.

Note that this can also be done at the ModSecurity level (if you have it installed on your server), and it is even more efficient to do it in ModSecurity, because multiple requests from the same IP can be blocked at the server level. Of course, ModSecurity is a huge realm, and developing a rule that enforces the above simple condition is not what anyone would consider a walk in the park.

But what about false positives?

There is a possibility that you will have false positives when implementing the above on your website, and that’s why it’s always a good idea to monitor your logs (especially 403 errors) in order to address any false positive.

Of course, there is much room for improvement in the defines.php code above. So if you are interested in enhancing it so that it works seamlessly on your website, then please contact us. We are experts in Joomla security, we are hard workers, and our fees are not scary!

We are getting calls from Joomla administrators telling us that they are seeing a bizarre warning in the backend of their Joomla sites after updating their sites to 3.7.x. Here is a sample warning:

Your PHP version, 5.6.30, is only receiving security fixes at this time from the PHP project. This means your PHP version will soon no longer be supported. We recommend planning to upgrade to a newer PHP version before it reaches end of support on 2018-12-31. Joomla will be faster and more secure if you upgrade to a newer PHP version (PHP 7.x is recommended). Please contact your host for upgrade instructions.

Most of the Joomla administrators contacting us are using the words bizarre or weird because they just don’t understand why they are seeing a warning for something that will happen at the end of 2018 (today is May 30^th, 2017). Naturally, they ask us for an explanation, and, as always, we oblige:

Joomla versions 3.7.0 and higher have a quickicon plugin that gets the version of the PHP instance running on the server that the Joomla website is running on and then compares it to a hardcoded database of PHP versions, with their Security Fixes Only and End of Life (or EOL) dates. If the current date (e.g. today) is higher the Security Fixes Only date, but lower then the EOL date for the PHP instance, then this means that the PHP version is longer in Active Support, and is in the Security Fixes Only phase. When this happens, Joomla displays something like the above warning.

If the current date is even higher than the Security Fixes Only date for the PHP instance, then this means that the PHP version used has reached its End of Life, and Joomla will display something like the below warning:

We have detected that your server is using PHP 5.5.38 which is obsolete and no longer receives official security updates by its developers. The Joomla! Project recommends upgrading your site to PHP 5.6 or later which will receive security updates at least until 2018-12-31. Please ask your host to make PHP 5.6 or a later version the default version for your site. If your host is already PHP 5.6 ready please enable PHP 5.6 on your site’s root and ‘administrator’ directories – typically you can do this yourself through a tool in your hosting control panel, but it’s best to ask your host if you are unsure.

As you can see in the second scenario, the Joomla website will display a warning asking the administrator to update to a PHP version that will also display a warning – this is because the algorithm works as follows:

• If the Joomla website is using a PHP version that is still supported, then do not display any message.

• If the Joomla website is using a PHP version that is only supported through security patches, then display the first message.

• If the Joomla website is using a PHP version that is no longer supported, then search for the first PHP version that is supported (even with security patches), and display the second message.

Obviously, the algorithm of the last step should be modified to return the PHP version that is still in Active Support.

In any case, most, if not all Joomla administrators feel uncomfortable when they see any of the above messages, and many of them do not have the time to update the PHP version on their server. Luckily, there is a way to get rid of these annoying messages. Here’s how:

• Login to the backend of your Joomla website.

• Go to on Extensions -> Plugins.

• Search for the plugin titled Quick Icon – PHP Version Check and disable it by clicking on the green checkmark next to it.

• That’s it!

As you can see, it is easy to get rid of the warning, but Joomla’s advice is sound, and you should update to a supported PHP version whenever is possible for the sake of both the security and the stability of your website.

Now if you’re wondering which PHP versions are the ones that do not display these messages, then they are, at the time of writing this post, all the PHP 7.x versions (including PHP 7.0).

We hope that you found this post useful. If you want to update your PHP version to the latest one but you are afraid that this may break something on your website, then please contact us. We’ll do the work for you, we won’t charge you much, and you will sleep better at night!

RSForm Pro is one of the most powerful extensions out there. You can create any type of form with it, and associate any action you can think of when the form is submitted, whether through plugins or through embedded PHP code. This makes RSForm ideal for integrating Joomla with payment gateways (such as Authorize, PayPal, and Stripe) and marketing tools (such as HubSpot, Marketo, Salesforce, and the likes).

But, hiccups are bound to happen from time to time, and we experienced a small hiccup with our last RSForm Pro integration. Here’s what happened…

A client of ours commissioned us to create a “gift card” page on their website, where people can purchase gift cards that are used towards subscriptions (the subscription system used was Akeeba Subscriptions – the gift cards are actually coupons in Akeeba Subscriptions). We used RSForm Pro for that, and we added 2 payment methods: PayPal and Stripe. The gift card process went as follows:

• The client visits the page containing the form, which asks the client about his name, his email, the type of gift card that he wants to purchase ($50 gift card or $100 gift card), and his payment method (credit card or PayPal).

• Based on the client’s choice of payment method, the system will either display a popup asking for the client’s credit card details or will redirect the client to PayPal.

• When the payment is processed, the Stripe plugin and the PayPal plugin will trigger the rsfp_afterConfirmPayment event, which is implemented in the RSForm Payment Success plugin (which we developed). The RSForm Payment Success plugin will generate the coupon code on the Akeeba Subscriptions system, and will send the coupon code to the client.

• The client is then redirected to a page displaying the coupon code (essentially a page containing the same information that was sent to the client in the email).

There was, however, two small problems in the process above: the payment status was not changed to “Accepted” (from “Pending”) and the rsfp_afterConfirmPayment event was not being triggered when the payment was completed through PayPal, which was odd, considering that, unlike the Stripe plugin, this plugin was not a 3^rd party plugin.

Naturally, the first thing that we checked was whether the notify_url was correct, and it was. Luckily, while checking that, we noticed that the PayPal RSForm plugin was logging everything to a file named rsformpro_paypal_log.php in the logs folder, and so we immediately checked that file, which contained the following (very helpful) information:

2017-05-23 23:28:26 : IPN received from PayPal
2017-05-23 23:28:26 : Connecting to https://www.paypal.com/cgi-bin/webscr to verify if PayPal response is valid.
2017-05-23 23:28:26 : PayPal reported a valid transaction.
2017-05-23 23:28:26 : Check the order's amount paid: 50.00 - 50.00. Payment amount is correct.
2017-05-23 23:28:26 : Validation failed -> The email address is not correct - [email-address]

As you can see, the last line is saying that the email address is incorrect, but it was. So, we searched for the code that was throwing this error, and it was the following code in the file rsfppaypal.php which is located under the plugins/system/rsfppaypal folder:

if (RSFormProHelper::getConfig('paypal.email') !== $validation_fields['receiver_email'])
{
	$array['error']  = true;
	$array['reason'] = sprintf('The email address is not correct - %s', $validation_fields['receiver_email']);

	return $array;
}

At first glance, nothing seemed to be wrong with the above code, but, on closer examination, we discovered a subtle bug: what if the email set in the configuration is not exactly the same as the email sent back from PayPal, what if, for example, the email set in the RSForm configuration is something like myemail@myjoomlatestwebsite.com and the PayPal email is something like MyEmail@myjoomlatestwebsite.com – in that case, the condition (in the first line) will return true, and an error will be returned by the PayPal plugin (which will halt the completion of the PayPal transaction from RSForm’s end). So, we changed the above code to the following…

if (strtolower(RSFormProHelper::getConfig('paypal.email')) !== strtolower($validation_fields['receiver_email']))
{
	$array['error']  = true;
	$array['reason'] = sprintf('The email address is not correct - %s', $validation_fields['receiver_email']);

	return $array;
}

…and then we tested the form again, and this time, it worked. The payment status was changed to “Accepted” and the rsfp_afterConfirmPayment event was finally triggered on successful PayPal payments. Phew!

Now if you, our dear reader, have problems with PayPal transactions still set to “Pending” (instead of “Accepted”) in your RSForm Pro form submissions even though these transactions were successful, then it might be that the email in the configuration settings of RSForm Pro does not exactly match that in PayPal. You can either ensure that both emails are exactly the same (e.g. they have the same casing), or modify the RSForm Pro PayPal as described above. If you need help with the implementation, or if that doesn’t solve your problem, then please contact us. Our prices are super affordable, our work is super clean, and our turnover is super quick!

A client with a huge Arabic news website called us on April 26^th (the day Joomla 3.7 was released) and told us that he’s not able to save articles after updating to Joomla 3.7. He told us that article saving was timing out, and urged us to take a look.

As usual, we obliged. We started our investigation by checking whether the issue is caused by one of the 5 reasons we listed in an earlier post. Unfortunately, it was none of them.

We then went to testing the website, and we noticed something odd: the problem was only happening when there is a lot of Arabic content in the article, because when we reduced the article’s content to a few Arabic sentences, the saving process went smoothly. When the article’s content was left as it was, the saving process was timing out. This gave us an idea:

• We reduced the PHP maximum execution time to 30 seconds by creating a local .user.ini file at the root directory of the Joomla website and adding the following to it:

  max_execution_time = 30;

• We changed Error Reporting to Maximum under System -> Global Configuration -> Server.

• We tried saving the article (using the original, long Arabic content).

• We saw the following error after 30 seconds:

  Fatal error: Maximum execution time of 30 seconds exceeded in /home/[cpanel-user]/public_html/libraries/vendor/joomla/string/src/phputf8/mbstring/core.php on line 41

Aha! We had a lead! We didn’t know what the problem was exactly but we had a lead. So, we opened the file core.php which is located under the libraries/vendor/joomla/string/src/phputf8/mbstring and we checked the problematic line, and it was this line…

return mb_strpos($str, $search, $offset);

which was located in the following function:

function utf8_strpos($str, $search, $offset = FALSE){
    if ( $offset === FALSE ) {
        return mb_strpos($str, $search);
    } else {
        return mb_strpos($str, $search, $offset);
    }
}

Hmmm! We felt, for some reason, that there was an infinite loop going on, and so we added a couple of lines to write the following variables $str, $search, and $offset to a file (we did it just before the problematic line), and, then we tried to add the article. Unsurprisingly, the file was several megabytes large after our test (mostly the same content being repeated over and over), so there was definitely an infinite loop going on.

Now, if you look at the above function, you will notice that it’s not recursive and doesn’t have any kind of loop, so, it must be the function calling it that is running into an infinite loop. Luckily, PHP has a very sweet method to know which function is the caller function, which is this line:

echo('Calling function: '.debug_backtrace()[1]['function']);

So, we used the above line to know which function was the caller function, and it was the function strpos which is located under the libraries/vendor/joomla/string/src folder. The following is the implementation of the strpos function:

public static function strpos($str, $search, $offset = false)
{
	if ($offset === false)
	{
		return utf8_strpos($str, $search);
	}

	return utf8_strpos($str, $search, $offset);
}

As you can see, the strpos function is not recursive, which means that the problem is caused by whichever function is calling it (or whichever function is calling the function calling it, or whichever function is calling the function that is calling it, OK – you get the idea!), ultimately, our investigation pointed us to the cleanTags function which is defined in the input.php file, which is located in the libraries/joomla/filter folder. So we thought, let’s compare the input.php file from the Joomla 3.7 website to the input.php file from the Joomla 3.6.5, and so we did, and, to our non-surprise, there were substantial differences between the two files.

Our next step (yes, you’ve guessed it!) was to replace the input.php file on the Joomla 3.7 website with one from a Joomla 3.6.5 fresh install. So we did that, and we tested the content adding (in Arabic) afterwards, and it worked, even when the content was very large! Hooray!

But why was the problem happening on Joomla 3.7?

Unfortunately, we can only say that the problem is caused by the cleanTags function (and possibly other functions called by cleanTags) in the aforementioned input.php file. We can’t give more details because we weren’t commissioned by the client to investigate the issue further (we can’t expect the client to pay for debugging Joomla errors).

We hope that you found this post useful. If you are running into a timeout when adding Arabic content (or any other UTF-8 content) to your Joomla 3.7 website, then try replacing the input.php file we mentioned above with the one from a Joomla 3.6.5 website. If that doesn’t work, then please contact us. We are always here to help, we will always find a solution, and our fees are extremely affordable!

Joomla 3.7 is a weird iteration of the famous CMS – it seems that quite a few important things were half baked or not completely tested. For example, the new routing functionality that was meant to be completely revamped in Joomla 3.7 was supposedly scratched from that version, to be later introduced in a future version (Joomla 3.8?). We are saying supposedly because there were some substantial changes to the routing functionality in Joomla 3.7, which left Joomla’s routing in a twilight state between Joomla 3.6.5 and Joomla 3.8. In fact, we were surprised when we saw the code because the lead developer in Joomla clearly stated in a blog post on the official Joomla blog that the new routing code will not be shipped with Joomla 3.7. He even stated (and we’re quoting his exact words) that the code itself also had some issues, some parts are pretty much unreadable and even with debugging not easy to understand. Yet it seems that some of that unreadable code made its way to Joomla 3.7.

In any case, the new routing functionality broke the URLs on some very important websites which were relying on the sef_advanced_link configuration variable in order to remove the article ID from the URL. This is because the sef_advanced_link configuration variable now belongs to the legacy router, which will only be used when the sef_advanced configuration variable is set to 1.

So, in order to remove the article ID from the URL in Joomla 3.7, you will need to do the following:

• Open phpMyAdmin and select the database powering your Joomla website.

• Click on the #__extensions table.

• Open the row where the name field is set com_content.

• Add the following to the params field (just before the curly bracket):

  ,"sef_advanced":"1","sef_advanced_link":"1"

  Explanation: The first parameter ensures that the legacy router is loaded, and the second parameter ensures that the “advanced SEF” is used (e.g. the “article ID” is removed from the SEF URL).

• Save the row and then login to your Joomla website and clear your Joomla cache.

• The issue should be solved. You shouldn’t see any article IDs in the URLs anymore.

Of course, if you’re a fan of core modifications (which is something that we do not recommend), then you can do the following instead of the above:

• Open the router.php file that is located under the components/com_content folder.

• Change the following:

  if ($params->get('sef_advanced', 0))
  {
  	$this->attachRule(new JComponentRouterRulesStandard($this));
  	$this->attachRule(new JComponentRouterRulesNomenu($this));
  }
  else
  {
  	JLoader::register('ContentRouterRulesLegacy', __DIR__ . '/helpers/legacyrouter.php');
  	$this->attachRule(new ContentRouterRulesLegacy($this));
  }

  to:

  JLoader::register('ContentRouterRulesLegacy', __DIR__ . '/helpers/legacyrouter.php');
  $this->attachRule(new ContentRouterRulesLegacy($this));

• Open the file legacyrouter.php which is located under the components/com_content/helpers folder, and change both instances of the following line:

  $advanced = $params->get('sef_advanced_link', 0);

  to:

  $advanced = 1;

• Login to the backend of your Joomla website and clear the cache by going to System -> Clear Cache and then clicking on the Delete All button.

• That’s it! The article IDs should disappear from your URLs.

We hope that you found this post useful. If you need help with the implementation, then please contact us. We are always there for you, our fees are super affordable, our work is super professional, and we always have a solution!

At itoctopus, we are paranoid about the security of our managed clients’ websites – as such, we always research revolutionary ways to better protect these websites against potential exploits.

Last week, we conducted a honeypot project on one of the largest websites that we manage. For those of you who don’t know what a honeypot project is, it is, in essence the capture and the analysis of the malicious traffic that a website receives in order to better protect it. Typically, a honeypot project consists of a bait to lure attackers (the same way honey lures bears), but, in the case of a very large and a very high traffic website, an artificial bait is not necessary since the importance of a website is in itself a bait.

Before going into the details, we will need to explain the security setup of the website:

• The website’s backend was protected with an .htaccess password: This method (described here) literally blocks all types of attacks on the backend. This means that we can completely forget about the attacks on the backend and focus on the attacks targeting the frontend.

• The “index.php” file was the only file allowed to be directly executed by the Apache web server: This ensures that any attack on the website must be funneled through the index.php file first (as direct access to other PHP files on the Joomla website is blocked). This is super important since, as you will find out below, we are doing all the work in the defines.php file which is loaded by the index.php file.

• ModSecurity was enabled on the server: This means that many known attacks are blocked before even reaching the Joomla instance. This will allow us to focus on new types of attacks.

Now that we have explained the existing security setup of the website, we can start explaining our little honepot project experiment. Are you ready? Let’s go!

Since Joomla loads the defines.php file before anything else (even before executing any serious code), we decided to add the code that will capture all the data in the requests sent to the website in the defines.php file. So, we created a defines.php file in the root directory of the website and we added the following PHP code to it:

<?php
	$file = 'project-honeypot.php';
	$projectHoneypotData = file_get_contents($file);
	$projectHoneypotData .= 'Date: '.date("Y-m-d H:i:s")."\n";
	$projectHoneypotData .= 'IP: '.$_SERVER['REMOTE_ADDR']."\n";
	$projectHoneypotData .= '--SERVER Data--'."\n";
	$projectHoneypotData .= print_r($_SERVER, true);
	if (!empty($_GET)){
		$projectHoneypotData .= '--GET Data--'."\n";
		$projectHoneypotData .= print_r($_GET, true);
	}
	if (!empty($_POST)){
		$projectHoneypotData .= '--POST Data--'."\n";
		$projectHoneypotData .= print_r($_POST, true);
	}
	if (!empty($_FILES)){
		$projectHoneypotData .= '--FILES Data--'."\n";
		$projectHoneypotData .= print_r($_FILES, true);
	}
?>

We saved the defines.php file and we uploaded it back, and then we left the website until the next day.

The next morning we checked the project-honeypot.php file, and we were amazed by the amount of garbage that hit the website: malicious referrers/user agents (that were not caught by ModSecurity), weird GET parameters, base64 POST values (some of these POST requests weren’t even relevant, since they were specially crafted for WordPress websites), and many, many attempts to upload hacked files.

The amount of data in that file was overwhelming. We had 2 options from there: either we needed to blacklist all the bad requests that we found or we needed to whitelist the good requests. We decided to go with the latter option, simply because whitelisting is a much more efficient process than blacklisting, since blacklisting means that we will need to continually monitor the website for future bad requests.

So we checked all the clean requests on the website and we whitelisted them all and blacklisted everything else (essentially, any request that was not on the whitelist was 403’d). For example, a valid POST request was one where $_POST[‘option’] = ‘com_search’ (this is sent when someone is searching for something on the website), so we whitelisted it.

Still, after whitelisting what should be whitelisted, we recorded all the whitelisted requests to the project-honeypot.php file. After 24 hours, we checked that file again, and we noticed that the absolute majority of the whitelisted requests were OK – however, there were a few bad requests among them. So, we had to blacklist those bad requests (at a granular level, any security work must include whitelisting and blacklisting – whitelisting alone is not sufficient). We repeated this process for a few days, until we were sure that almost all malicious requests were blocked from the server.

At the same time, we monitored the Apache logs for any legitimate 403s (e.g. false positives that were blocked by our whitelisting) and we did find a couple, which we immediately whitelisted.

Now, the million dollar question is, how efficient was all this?

We can safely say that the work was very efficient – we were much more comfortable with the security of the website, and we had no doubts in our minds that the website was much more secure than before. The whole experiment was so successful that we were very confident to recommend this process to our other managed clients.

Was there a performance hit caused by the security work above?

Not at all – in fact – there was a performance gain! Yes – you heard that right – there was a performance gain! And that’s because all those blocked requests were no longer loading the Joomla environment (this reminds us of a previous post of ours where we explained how not to load the Joomla environment for non existing files).

We hope that you found our post instructive and useful, and we hope that you will benefit from it to supercharge the security of your Joomla website. If you need help in doing that, then look no further. Just contact us and let us do the work for you quickly, cleanly, and affordably!

Do you have time for a simple experiment? If yes, then try the following:

• Download Joomla.

• Install it on a local or a remote server.

• Once the website is working, rename the plugins folder to plugins_old.

• Visit the website, and you will notice that the website still works.

• Now rename the modules folder to modules_old, and you will notice that although all the modules disappeared, the website still works.

The above demonstrates the power and resilience of a Joomla instance: Joomla websites can work without the plugins (and the modules) folder(s). But, why is this worth mentioning?

It is worth mentioning because this discovery can help you in quickly debugging problems on a Joomla website without taking it offline. For example, if your website is suffering from performance issues all of a sudden, then you can rename your plugins folder to plugins_old and see if the issue persists. If it does, then the problem is not caused by one of your plugins, if it doesn’t, then the problem is definitely caused by one of your plugins. The same can be done with the modules folder.

Will the website “fully” work?

Of course, although the frontend of the Joomla website will work, it will miss all the functionalities provided by the plugins including frontend login and SEF URLs. So, you shouldn’t really do this unless you are debugging a website.

Can this be done on all Joomla websites?

No – some Joomla websites use components that explicitly require files from the plugins folders. Such websites will crash if the plugins folder is renamed.

Will the backend still work?

If you delete/rename the plugins folder, then you won’t be able to login to the backend, because you will need the authentication plugin to be able to login.

We hope that you found this post educational – if you have been reading it with the purpose of minimizing the filesystem of your Joomla website, then please note that such an endeavor is not straightforward and does not simply consist of renaming folders. Just contact us and we’ll devise a clean and sustainable solution for your needs for a very affordable price.

A common issue on Joomla websites with frontend editing is that the behavior of the “Save” button on the frontend is different from the behavior of the “Save” button in the backend. The “Save” button in the backend saves the item and redirects back to the item’s page while the “Save” button on the frontend saves the item and redirects back to the listing page, which is the exact behavior of the “Save & Close” button in the backend.

This issue is particularly annoying when frontend authors/editors are writing long articles and are saving these articles frequently (they will get redirected back to the listing page, where they will need to find their article and then edit it again).

Luckily for you, our dear reader, we have a solution to this issue. With a simple trick, we were able to mimic the behavior of the backend “Save” and the “Save & Button” in a Joomla frontend. Here’s how we did it for a client of ours a few days ago:

• We copied the edit.php file which is located under the components/com_content/views/form/tmpl folder to the templates/[ourclienttemplate]/html/com_content/form folder.

• We then opened the edit.php file that we just created, and just before the following line…

  JFactory::getDocument()->addScriptDeclaration("

  …we added the following lines:

  $jinput = JFactory::getApplication()->input;
  $currentArticleID = $this->form->getData()->get('id');
  $editURL = JUri::root().'index.php?option=com_content&task=article.edit&a_id=' .$currentArticleID.'&return='.$jinput->get('return');

• In the same edit.php file, and just before the following line…

  if (task == 'article.cancel' || document.formvalidator.isValid(document.getElementById('adminForm')))

  …we added the following lines:

  if (task == 'article.savestay'){
      var inputReturnURL = document.getElementById('returnurl');
      inputReturnURL .value = "";
      task = 'article.save';
  }

• Also in the file edit.php, we replaced the following line:

  echo JText::_('JSAVE')

  with:

  echo ('Save & Close')

• Finally, while still in the edit.php file, and immediately after…

  <div class="btn-toolbar">

  …we added the following lines:

  <div class="btn-group">
  	<button type="button" class="btn btn-primary" onclick="Joomla.submitbutton('article.savestay')">
  		<span class="icon-ok"></span><?php echo JText::_('JSAVE') ?>
  	</button>
  </div>

• We saved the article and we uploaded it back.

• We tested frontend editing and everything worked as it should! The “Save” button redirected back to the the same article, the “Save & Close button (as well as the “Cancel” button) redirected back to the listing page.

There is one caveat in the above trick: it only works for existing articles. If you are working on a new article from the frontend, then you must hide the “Save” button (if $currentArticleID is empty, then this means that we are creating a new article, which means that we must hide the “Save” button).

We hope that you found this post useful. If you need help with the implementation, then you can always contact us: we are always available, we are always ready, and we will not charge you much!

Let’s do a simple experiment together. Go to your Joomla website, type in something like: http://www.[yourjoomlawebsite].com/test.jpg (assuming you do not have a test.jpg file in the root directory of your Joomla website).

You will notice that Joomla displays a 404 error. What’s wrong with that, you may be wondering? Well, what is wrong is that the whole Joomla environment is being loaded when a simple “404 – Not Found” error would have been sufficient. Of course, on small websites, this is hardly an issue, but, on large websites with high traffic, it is.

Now the question is, why is Joomla handling all the 404s, instead of just handling the 404s related to articles not found? Well, it is because of the following code in the default Joomla .htaccess file:

# If the requested path and file is not /index.php and the request
# has not already been internally rewritten to the index.php script
RewriteCond %{REQUEST_URI} !^/index\.php
# and the requested path and file doesn't directly match a physical file
RewriteCond %{REQUEST_FILENAME} !-f
# and the requested path and file doesn't directly match a physical folder
RewriteCond %{REQUEST_FILENAME} !-d
# internally rewrite the request to the index.php script
RewriteRule .* index.php [L]

If you have some experience reading and understanding .htaccess files, you will know that the above tells Joomla to process requests to non-existing files and directories, which means that any hit on the domain with a file or a folder that doesn’t exist will eventually reach the index.php file, and ultimately load the Joomla environment. As mentioned in the article we linked to above, this can cause performance and stability issues on the server, especially on sites with lots of content and very high traffic.

So, how can one solve this problem?

Obviously the solution to this problem is to tell the Joomla environment to only execute non-file requests (we cannot block it from executing directories, because a non-existing directory can be a valid Joomla link). This can be done in the .htaccess file, by adding the following code just after the RewriteEngine On line:

# if the file doesn't exist
RewriteCond %{REQUEST_FILENAME} !-f
# and the file extension is one of the below
RewriteRule ^(.+\.(?:css|gif|jpe?g|js|pdf|php|png|xml))$ 404.php [L,NC]

Before adding the above code to your .htaccess, you will need to create the file 404.php in the root directory of your Joomla website. This file should only contain the following couple of lines (a mini-sized file ensures that the server performs minimal work and that traffic spent on non existing files is minimal):

<?php header("HTTP/1.0 404 Not Found");
die('404 - not found');?>

Once you do the above, you will notice that non-existing files will no longer be processed by the Joomla engine – which is a good thing – as this will lower the server load. If you have a large website, then the performance gain by adding the few lines of code above will be noticeable.

We hope that you found this post useful. If you need help with the implementation, then please contact us. We are always there for you, our fees are super affordable, and our work is super clean!

Each day, more and more clients are calling us to address the slow article saving problem that they are experiencing on their Joomla websites. We’re not 100% sure why this is becoming an increasingly common Joomla problem, but we think that it might be that more large sites are using Joomla (though we have no actual statistics to substantiate our claims – we just feel it!). It might also be that we are getting more clients because of our high quality work, we don’t know for sure… But what we do know is that the emergence problem is not caused by a sudden change in the Joomla core (although, in most cases, it is caused by the Joomla core).

Over the past few months, we have worked on a countless number of websites having this problem, and so we have developed a list of all the issues that may result in long article saving time (or, in many cases, timeouts). Here they are:

1. A big assets table: The assets table is one of the most critical tables on a Joomla website – if not the most critical. A small inaccuracy in that table may crash your website. Additionally, the more data you have in that table, the slower the article saving process becomes (mainly because of the lft and the rgt fields – which we are not going to discuss in this post). A good idea is to trim that table on a daily basis in a cron job – just make sure that you don’t have any special permissions on your articles before doing that. Follow our guide here in order to cleanup the assets table on your Joomla website.

2. A lot of tags: We never liked tags – and we have to be honest why we don’t like them: it is because we think they are mostly used for webspam. Now, we have yet another reason to hate them, and it is because article saving takes a long time on Joomla websites that have many tags. If you have more than a few hundred tags on your website, then you will have a noticeable delay when you save Joomla articles. Luckily for you, the solution to this problem consists of simple core changes rather than deleting your tags.

3. Smart Search: Whoever called Joomla’s Smart Search “smart” must really have lax standards when it comes to smartness – since Smart Search is anything but smart. In fact, Smart Search is so bad that there were actual plans to remove it from the Joomla core (unfortunately, these plans didn’t materialize). In a nutshell, the problem with Smart Search is the following: it works well when it’s not needed, and it doesn’t work well when it’s needed. You see, Smart Search was developed to cater for the heavy search requirements on large Joomla websites since the normal search plugin is inadequate for large websites, however, the problem is that Smart Search doesn’t take into consideration one of the most important factors on large Joomla websites, and it is there are many articles created/updated every day. The thing is, whenever an article is saved, the Smart Search plugin runs, and splits the article into many fragments that are saved in many tables. The fragment-saving process becomes increasingly heavy with time ultimately slowing down the whole website to a crawl. The solution to this problem is simple, and it is using Sphinx instead (and, of course, disabling the Smart Search plugins).

4. The ordering algorithm of Joomla articles: Let us explain how the ordering of Joomla articles works in layman terms. Let’s say you have 9 bricks on top of each other, and you want to add, and label, an additional brick. If you want to follow the Joomla ordering algorithm for doing things, you will do the following:

• You will ask 9 persons to help you in your endeavor.

• You will ask each one of these persons to lift one brick, while ensuring that all the bricks are still on top of each other.

• Ultimately, you will have some empty space at the bottom of the bricks, and so you add your brick there.

• You label the bricks from 1 to 10, starting with the brick at the very bottom (the one that you just added).

Of course, a crazy idea would be to just add the tenth brick on top of the other 9 bricks instead of the bottom, and labeling the 10th brick “Number 10”, but where’s the fun in that? Additionally, you won’t get to meet these 9 nice people if you do it the old, unexciting way. Joomla’s ordering works in the same way, when a new article is inserted, all the articles in the same category are shifted (though not by real people) by 1. Obviously, this reordering process is very expensive on large Joomla websites. The solution to this problem is a core modification, which is explained, in details, in the article that we have just linked to.

5. A bad system/content plugin: So far, all the issues that we have discussed in this post are core issues, but the problem is not always caused by the core. Some system/content plugins, for example, try to grab data from another website using curl, a process that may take a very long time to execute and may cause the article saving process to hang. Last week, we worked on a Joomla website that had one of these bad plugins; we disabled all the system/content plugins one by one until we found it. When we opened the plugin, we noticed that it was setting the PHP maximum execution time to a very high number and was hiding all the errors. As seasoned PHP (and, of course, Joomla) developers, we know that both of these practices are very bad practices, but we also know that bad Joomla developers are everywhere.

Of course, there are other reasons that may cause the article saving process to take a very long time, two that we can think of are a hacked website and a ModSecurity rule. But they are not very common.

So, how to know which one from the above list is causing your problem? Well, you add an article to your Joomla website, and you monitor the performance of your Joomla website (or the load on your server, if you have access to that information), and if you see a degrading performance across the board (or a spike in the load), then your problem is likely caused by one (or more) of the first 4 reasons (in this case, you follow a process of elimination to know which one of the 4 is your problem). If you don’t, then your problem is likely caused by a bad plugin. Of course, if you need help speeding up the article saving process on your Joomla website, then you can always contact us. We are always there for you, our work is super clean, and our fees are super affordable.

Earlier this week, we published a post on the importance of switching your Joomla website from HTTP to HTTPS, mainly because of a new setting in Chrome and Firefox labeling pages with sensitive forms (such as login forms) as insecure if they are not running in HTTPS mode.

Most Joomla administrators think that switching from HTTP to HTTPS simply consists of buying an SSL certificate, installing it on the server, and then forcing SSL on the Joomla website by setting the Force HTTPS field (under System -> Global Configuration -> Server) to Entire Site. Unfortunately, in most cases, the process is not that simple. You see, when you switch from HTTP to HTTPS there are many considerations that you must take into account. In fact, there are 7 important questions that you must ask yourself when you make the switch:

1- What will happen to all the social interactions?

Social interactions, such as likes, shares, etc… took place on the HTTP version of your website. This means that, by default, you will lose all these interactions when you switch to HTTPS. However, there is a way to keep these interactions, and it is by setting the data-url attribute to the HTTP version of links for the pages that were created before the switch, and to the HTTPS version of the links for all the other pages. Typically, this change is done in one plugin.

2- Is the website “301 redirecting” all the HTTP links to their HTTPS equivalents?

You must ensure that your website is redirecting (using 301 redirects) all your previous (non-secure) links to your new (secure) links. If it doesn’t, then you should address that immediately, or else you will have a duplicate content issue and your search engine rankings will drop.

3- Do all the extensions on my Joomla website fully support the switch?

Some poorly written templates/modules/plugins/components have some HTTP links that are hardcoded. For example, they request images, CSS files, and JS files from HTTP sources. This, of course, will cause a mixed content issue on your pages. The quickest way to catch the offending extensions is to thoroughly test every functionality on your website once you switch, and note, in the browser console, which images/libraries are being loaded over HTTP, and then fix the associated extensions. Ideally, you should do this on a test environment before switching.

This task is actually much more complex and important than you think it is, and we have numerous examples on how overseeing this task has caused major unhappiness.

4- Is my Joomla website displaying fatal errors on some pages because of the switch?

This is really similar to the previous point – but with a twist. Some badly written extensions may crash (or, at best, may not work properly) when you switch from HTTP to HTTPS. What makes this even more challenging is that some of these extensions may only fail under certain conditions. In order to address problems arising from such extensions proactively, you will need to closely monitor your PHP error log for the following few weeks after the move, and immediately address any errors caused by the switch.

5- Is my “$live_site” set to the HTTP version of the website?

We never recommend setting the $live_site value in the configuration.php file, and we are not very clear on why people like to set it. In fact, we have nothing good to say about this particular variable, which caused hardship for many Joomla administrators out there. However, if you feel you are compelled to set it, then make sure it points to the HTTPS version of your domain.

6- Have I disabled all SSL redirect plugins on my website?

On more than one occasion, we have fixed a problem where an SSL redirect plugin caused an infinite loop on the website as soon as the Force HTTPS setting was set to Entire Site. Make sure you disable (or, better yet, uninstall) any SSL redirect plugin that you have on your website before making the switch (they are pointless anyway and the only thing that they can do is create problems).

7- Have I checked my .htaccess file for any weird redirects?

A couple of days ago, we had a weird case of an infinite loop on a Joomla website. The infinite loop was everywhere, on both the backend and the frontend, so we disabled all the non-core system plugins on the Joomla website using phpMyAdmin, but the problem remained. It turned out eventually that the client had a redirect, in his .htaccess file, from HTTPS to HTTP, and there was another redirect in Joomla from HTTPS to HTTP. Naturally, deleting the .htaccess redirect fixed the problem.

Switching your Joomla website from HTTP to HTTPS is seldom an easy task – but worry not – we are always ready to help. Just contact us and we’ll ensure a very smooth transition of your website for very little money.

Note: The solution presented in this post is a core change. Proceed carefully and always backup your website before modifying core files.

A few days ago, we received an email from someone managing a very large Joomla website (it was a Brazilian news website). She said that the saving of articles was taking an abnormal amount of time. When she first told us that, we were relieved; finally, something easy to work on! After all, it could be either an inflated assets table, or the an abundance of Joomla tags.

We started working on the task immediately, and, unfortunately, we quickly discovered that our excitement for easy work was a bit premature: it was neither the assets table nor the tags. In fact, the client regularly cleaned the assets table and the tags table was maintained at a manageable level. We were still hopeful though, so we started disabling plugins one by one – hoping that this whole issue might be caused by one of the plugins, but that wasn’t the case: we disabled all the plugins, but still we had the same problem. We were finally left with one option: we had to get our hands dirty to get to the bottom of this issue…

So, we enabled MySQL’s slow query logging and added an article to the Joomla website, and we were confident, no, make that positive, that we’ll find a very complex query in the slow query log (after all, the article saving process literally took 5 minutes), but there was nothing there, even though the long-query-time was set to 1 second (e.g. all queries taking more than 1 second were supposed to be logged there).

We were becoming more and more depressed: a task that we thought would bring joy to our hearts was becoming more and more frustrating. But then, something unexpected happened. While trying to save another article, we looked at the browser status bar (which is located at the bottom of the screen), and we noticed that the “Sending Request” part was taking a long time, and then the “Waiting” part was being executed instantly. So, this whole thing all of a sudden seemed like a DNS problem. We asked the client if she had any idea what might be causing this, and she told us that she was using Cloudflare, so we asked her to bypass Cloudflare to see if that solves the problem. She did, and then we tried to add an article, and this time, the whole thing was executed instantly. Hooray, we found the culprit, it was Cloudflare, or was it? You see, despite us feeling happy, we had this strange feeling that we didn’t nail it. After all, why would Cloudflare have this problem only with adding articles? Could it be a firewall rule being triggered on Cloudflare‘s end when someone adds an article? And, if that’s the case, then why weren’t other websites affected? Why just this one? Lieutenant Columbo would have said: “Something’s bothering me…”

So, we did another test to be sure, and this time, we had the problem again: it took about 4 minutes for an article to be saved. Our hunch was right, we didn’t find the solution to the problem… In a desperate move, we decided to debug the article saving process, so we opened the article.php file located under the administrator/components/com_content/models/ folder, and we added a die before the following line…

if (parent::save($data))

…and the die was executed instantly. We then moved the die to after the above line, and it took ages (like minutes), to take effect. So, the problem was really in the save line. Obviously, the next thing to do was to track which queries were executed at the database level by that save line. In order to do that, we debugged the problem at the database level: we printed all the queries before the save method, and then we printed all the queries after the save method (but before everything else), and it turned out that the cause was the following query:

UPDATE `#__content` SET ordering = 106 WHERE `id` = '48129'

In fact, the above query was sent to the database a few thousand times, but with a different ordering and id values. We thought that the client was lucky that saving articles was only taking 4 minutes for the whole thing (each iteration was taking about 50 milliseconds).

It didn’t take us long to find where that query was executed: it was in the table.php file located under the libraries/joomla/table folder. Specifically, the whole thing was taking place in the reorder function of that file. The obvious question at this point would be, what does this function do?

Well, the reorder function is responsible for (you guessed it) ordering articles in Joomla. For new articles, it ensures that the ordering field of the new article is set to 1, and that the ordering of all the other articles in the same category is shifted by 1. Naturally, when you have thousands of articles in each category, then the shifting process will take time.

If you’re thinking that the reorder function is poorly optimized, then you’re right: it is poorly optimized, and the Joomla core development team knows about it. In fact, Joomla versions 3.6.0, 3.6.1, and 3.6.2 addressed this problem, but, by addressing the problem they created an even bigger problem with current websites because the ordering of the new articles in the backend looked messed up (new articles appeared last), and so many Joomla developers voiced deep criticism about this abrupt change, including us. Eventually, the development team had no choice but to revert back the whole thing to the way it originally worked.

In retrospect, we believe that the ordering change in Joomla versions 3.6.0, 3.6.1, and 3.6.2 was done in good faith, but it was incomplete. While these versions fixed the performance problem by no longer shifting the ordering of existing articles by 1 when a new article was inserted, and rather just setting the ordering field of the newly inserted article to the highest ordering + 1, they didn’t address the backend views and the existing data. These versions should have modified the existing views and the existing ordering data to take the new ordering strategy into consideration. Obviously, the second part of the previous sentence (modifying the data) is very dangerous and complex, hence the decision to revert back rather than really addressing the problem.

So, how did we solve the problem?

Luckily, the client wasn’t using the Article Order ordering on any of her Category Blog menu items; she was using the Most Recent First ordering across the board. So, we were able to skip the whole ordering process by adding the following line to the very beginning of the reorder function:

return true;

The saving process worked very smoothly once we added the above line. In fact, it took our client less than a second to save articles on her website after implementing the above patch.

But, what about the ordering of featured articles on the homepage?

The client wasn’t using a menu item of type Featured Articles on the homepage, however, we understand that most Joomla websites do. So, if you want to implement the above solution on your Joomla website and your homepage consists of a Featured Articles menu item, then you will run into issues (the ordering of new articles will be messed up, and the reordering of old articles will no longer work properly). In this case, you will need to add the following lines (instead of the above line) to the beginning of the reorder function:

if ($this->_tbl != '#__content_frontpage')
	return true;

The above will ensure that the #__content_frontpage table will still be reordered by the reorder function. Keep in mind, however, that if your #__content_frontpage table is very large (e.g. has thousands of rows), then adding new articles to your homepage will take a lot of time. It’s always a good idea to ensure that this table contains a thousand entries or less (note that if you have pagination on your homepage, then this pagination will only show a thousand articles if you trim that table to a thousand rows – be very careful). We execute the following query in the cron of the Joomla sites that we fully manage to trim the data in the #__content_frontpage table:

DELETE FROM #__content_frontpage WHERE `ordering` > 1000

But why was the problem briefly solved when the client turned off Cloudflare?

The problem wasn’t solved. It was just a coincidence. When the client turned off Cloudflare, we added an article to a category that had very few articles, and so the reordering process was super quick.

We hope you didn’t find this very long post intimidating. After all, the solution really consists of adding one line (or a couple of lines) to just one file. The rest is purely detective work (or a wild-goose chase, depending on how you see things) and a discussion of the cause of the problem. Having said that, you should do the necessary due diligence before implementing the solution, mainly by ensuring that you are not using the Article Order ordering anywhere on your website. If you do, or if you are not sure, or if you just want help with the implementation, then please contact us. We will always find a solution to your ordering problem, we will ensure that your website remains stable after the implementation, and you won’t have to relinquish your hereditary rights on that beautiful south pacific island in favor of your evil half-sister to pay us our fees!

Back in 2013, we have debunked the myth that switching from HTTP to HTTPS will make a Joomla website more secure, and we have explicitly recommended against using it globally. Obviously, times are different now, and while we still haven’t changed our mind that such as switch will not make a Joomla website more secure, we can no longer recommending against it. Why?

Well, because of the following reasons:

• As of August 2014, Google considers the usage of HTTPS across the board as a minor positive ranking signal. In other words, using HTTPS on your website (the whole of your website) will give its search engine rankings a minor lift.

• As of January 2017, Google Chrome, the world’s most used browser (in fact, at the time of writing this article, Google Chrome has more market share than all of the other browsers combined), has started displaying a “Not secure” message in the address bar when people visit a page that has a form transmitting sensitive data. So, for example, if every page on your Joomla website has a login module, then Chrome users will see that “Not secure” message on every page of your website. Not good.

• At around the same time (January 2017), FireFox, the world’s third browser by market share, started displaying a lock with a diagonal red line to label a website matching the same criteria defined by Google Chrome as non-secure.

Unless you are sure that all of your site visitors use Bing as their search engine and Internet Explorer/Microsoft Edge as their browser (Microsoft hasn’t followed suit yet), then you must do something about it, or else your rankings will suffer considerably. Why, you may ask? Well, because when some (if not most) people see that “Not secure” message on their browser when visiting one of your pages, then they will flee your website. This will substantially increase your bounce rate, which will severely affect your search engine rankings. Yes, you’re right, the labeling used by Chrome and FireFox is inaccurate (only the data transmission is not secure, and not the whole website), but only the technical people will understand that.

So, our dear reader, if you are experiencing a higher-than-usual bounce rate on your Joomla website and you have a login form slapped on all of your pages, then the likely cause is that your website is not running in secure (HTTPS) mode, when it should. If you need help with making the switch from HTTP to HTTPS, then you are in no better place: just contact us and we’ll handle this for you quickly, efficiently, and affordably.

A client called us this morning and told us that he was unable to create new modules and save existing modules. So, we logged in to his website and checked the problem, and it was like he said: we opened a module, we modified it, and then we saved it, but it wasn’t saving the new information. We weren’t also able to create new modules. What was weird though, is that we were able to set the Published status of the module from the module listing page (e.g. by clicking on the green published check mark).

At first, we thought the issue had to do with caching, but we quickly ruled out the problem when we noticed that modifications from the module listing page were taking effect immediately. In fact, the client didn’t have any caching on his Joomla website whatsoever.

We then thought the issue had something to do with a JavaScript conflict, so we checked the console in Google Chrome, but we didn’t see any errors.

We were desperate – and so we started blaming solid extensions for the problem, like JCE, for example. So we switched from JCE to the default Joomla editor, but that also didn’t fix the issue.

We were just about to do some serious debugging on the website when we noticed that the user had hundreds of groups in the Joomla ACL. Could it be like the problem we had a couple of weeks ago, where Joomla’s configuration settings were not saving, we thought?

So, we created a .user.ini file with the following content:

max_input_vars = 8000

We then placed the file in the root folder of the Joomla website, and we tried again, and this time it worked! Hooray! It was a super simple fix to a very annoying problem!

If you have the same problem on your Joomla website, then you can fix it by setting the max_input_vars to a very high number in the .user.ini file. If that didn’t work for you, or if you need help with the implementation, then please contact us. We always find a solution to any Joomla issue, our fees are super competitive, and we are always overjoyed to have good clients like yourself!

Back in 2011 (yes – that’s 6 years ago!) we published an article titled: “My Joomla Changes Are Not Showing!“. In short, the article stated that when Joomla changes in the backend are not visible on the frontend, then it’s a caching issue.

Of course, Joomla has changed substantially between then and now, but typically, when Joomla changes are not showing immediately, then it’s almost a certainty that the cause is caching. It might be browser caching, Joomla caching, or server (or proxy) caching – but it is, almost always, caching. We’re saying almost because on very rare occasions, the cause is not caching. Let us tell you a little story to make our point…

Yesterday evening a new client called us, and told us that she was not seeing her changes, despite clearing the cache. So, we logged in to the backend of her Joomla website, made a small change, and then checked the website, and our change did show. Naturally, we told her that her website didn’t have a problem in reflecting new changes, and we asked her to clear her browser cache. And so she did, but, even after clearing her browser cache, she didn’t see our little change. It was puzzling. We asked her to use another browser, and another machine, and she did, but the problem was still there…

Eventually, she mentioned to us that the whole problem started when her host moved her website to another server early in the morning. And so it hit us: we asked her to ping her website from her end and we pinged her website from our end, and, unsurprisingly, we both got different IPs. It seems that on her end, the website hasn’t resolved yet, but on our end, it did, and that’s why we were seeing the changes and she weren’t. Mystery solved!

Now if you, our dear reader, are not seeing your Joomla changes immediately and if you are sure that it is not a caching issue, then it might be caused by a recent move to another server, and just restarting your Internet router (and your machine) may resolve the problem. If you need help investigating the problem, then please contact us. We are always ready to help, we are always happy to help, and our prices are super affordable.

One of the most annoying problems on Joomla websites is that saving articles can timeout (and display a 500 – Internal Server Error message) or can take an insane amount of time. This problem is not uncommon on large websites, and it typically gets worse every day. But why does this problem exist on a high profile CMS such as Joomla?

Well, there are five reasons that can cause a timeout when saving Joomla articles (or can cause the process to take a lot of time, like many seconds or even minutes):

1. An overcrowded assets table: When you save an article on a Joomla website, a row (sometimes even more than one row) gets inserted/updated in the assets table. The problem is, the more rows you have in that table, the more costly the operation is. A “good” assets table is a table that only has a few hundred entries, more than that and this table can be the source of major trouble on your website. A quick (and dirty) fix to the “assets” problem is to clean the assets table as described here. You should also consider running a nightly cron that will clean your assets table (that’s what we do on large Joomla websites that we support).

2. Smart Search: The so-called smart search is the unsung hero (or is it villain?) of Joomla performance nightmares. Smart search was created to replace classic search plugins on Joomla websites, and that was because the latter created performance issues on large websites. Unfortunately, not only smart search did not solve those performance issues, it added even more performance issues. You see, when you save an article with the Smart Search – Content plugin turned on, many entries (rows) will be created in the finder tables (the number of entries is proportional to the length of the article), which is a very costly process when it comes to the database. If you have millions of rows in those finder tables, then likely the problem lies in the smart search plugins that you’re using. To make sure, just disable all the smart search plugins that you have and try to insert an article. If the problem is gone, then the cause is definitely smart search. The solution to this problem is to use Sphinx instead.

3. A system plugin: Some non-core system plugins are triggered when you save a Joomla article. These plugins are typically harmless, and they merely run only to support a feature on your website. Some of these plugins, however, perform some heavy database operations, or request a remote file (but timing out on that file because of permission reasons), thus slowing down the saving process. A good strategy to solving this problem is to disable all your non-core system plugins one by one in order to locate the rogue plugin.

4. Shared hosting: If your website resides on the server of a mainstream shared host, then be prepared to encounter performance issues everywhere, not just when saving articles. This is because mainstream shared hosts try to cram as many websites as they can on the same server, so they’ll barely give each website enough power to run. If you are serious about your Joomla website (or any type of website), then you should go with at least a VPS. Note: avoid all these “cloud hosting” solutions – cloud hosting is technically shared hosting, but with a different, more attractive name to lure the masses.

5. A hacked website: There are many variations of hacks on Joomla websites. Some of these hacks affect specific functionalities on the Joomla website, such as when saving new articles. Run a scan on your website (a super quick method that has very high accuracy is to run these couple of shell commands to find hacked files on your Joomla website), and ensure that you have the latest version of Joomla installed, and also ensure that all your extensions are up to date.

We hope that our post helped you find the cause (and the solution) to your article saving problems. If it didn’t, then don’t despair, just contact us and we will find the solution to the problem quickly, efficiently, and affordably.

Note: The above list is not fully exhaustive, there are other reasons (mainly having to do with the firewall or the server’s environment) that may cause article saving problems on a Joomla website.

Back in 2014, we presented a solution to a common Joomla problem then, where Joomla displays the mobile version of the template on the desktop (or vice versa). We stated that the problem was caused by the Joomla cache, which caches the mobile version of a page and displays it for both desktop and mobile devices if a visitor visits the website with his mobile device first (e.g. before the cache is created or after the previous cache has expired).

Admittedly, our solution was a core hack, which was the best solution then since the problem lied in the core. About 14 months after our post (on March 21^st, 2016), Joomla 3.5.0 integrated our changes – almost exactly as they are, in its core. The question that you probably have on your mind right now is: “where?”. The answer is Platform Specific Caching.

Login to your Joomla website, and then click on System -> Global Configuration, and under Cache Settings, you can see Platform Specific Caching, which is set to “No” by default.

How does Platform Specific Caching works?

Platform Specific Caching works the same way our core hack works: it differentiates between the desktop cached version and the non-desktop cached version of the page by appending a string to the cache file name. The only difference is that instead of prefixing the cache file with mobile-, it prefixes it with M-. When someone from a mobile device visits the site, it will then create/serve a cache file prefixed with M-. If someone visits the site from a desktop, then it will create/serve a non-prefixed cache file.

Why is Platform Specific Caching set to “No” by default?

Because of 2 reasons: 1) not all sites have a mobile template, and 2) not all sites have a mobile template that is different from the desktop template (this is the case of websites with a responsive template, which generally don’t need to use Platform Specific Caching).

Are there any disadvantages of using Platform Specific Caching?

The only disadvantage is the fact that number of cached files will double – but, in this day and age of SSD drives with large capacity, this disadvantage is negligible.

Do websites with responsive templates need to use Platform Specific Caching?

Generally no – unless there are extensions that generate different HTML for desktop and mobile devices. In this case, it must to be used. Note that Platform Specific Caching will not cause any issue on the website (even if it’s not needed), so, if you are unsure, then you can just safely set it to “Yes”.

If your website is displaying mobile layouts on the desktop, then you should enable Platform Specific Caching. If doing so doesn’t solve your problem, then please contact us. We are always happy to serve, we are experts in Joomla, and our fees are very affordable!

Note: In this post, by “core file” we mean a Joomla file that cannot be overridden at the template level.

We have a somehow hawkish attitude when it comes to optimizing Joomla: we are not afraid of using core modifications (e.g. core hacks) to optimize/secure Joomla websites. For that, we are often criticized by some members of the Joomla community; these people state that it is always better to use overrides instead of core hacks. We don’t disagree, but like in everything else in life, a balance must be struck between what’s ideal and what’s realistic.

Ideally, we should have the ability to override any file on a Joomla website. Realistically, this is practically impossible, and we have an experiment that backs our statement. A few months ago, we were assigned to a project that consisted of optimizing a large Joomla website. We decided to use an extension called Joomla Override (the extension is actually referred to in Joomla’s official documentation) in order to do the optimization. At the end of the day, we discovered that many important files cannot be overridden with that extension (and any extension, for that matter), mainly because these files are loaded before anything else. By the end of the optimization mess work, we ended up with 4 overridden files, and 6 Joomla files with core modifications. Naturally, we decided that the whole thing was counterproductive, and we switched the file overrides to core modifications. A substantial part of this (failed) experiment was documented here.

But what if itoctopus developers don’t know how to properly override core files?

If you are an avid read of our blog, then you should have an idea of our level of expertise in Joomla. We know what we are talking about: there are many files that cannot be overridden on a Joomla website, and a quick example is session related files, as initial session activities must take place before loading any Joomla extension.

The only way to have the ability to override all Joomla files is for Joomla to implement the mechanism to do so. For example, Joomla can have a folder called userdefined in its main directory, and, before loading any Joomla file, it will check if a file with the same path exists under the userdefined directory. For example, if someone wants to override the includes/framework.php file, then he should create a file called framework.php under userdefined/includes. Adding such ability to Joomla is not an easy task, as all the requires / includes functions must be replaced with jrequires / jincludes functions. The latter functions will be wrappers (that must be defined in the index.php file) and will first attempt to require/include a file from the userdefined folder, and, if the file doesn’t exist, then they must require it/include it from its normal location.

Here is an example implementation of the jrequire function:

function jrequire($path, $once = false){
	if (file_exists('userdefined/'.$path)){
		if ($once)
			require_once('userdefined/'.$path);
		else
			require('userdefined/'.$path);
	}
	else{
		if ($once)
			require_once($path);
		else
			require($path);
	}
}

The above function could be written more elegantly, but you get the idea. This will allow Joomla developers to override any Joomla (PHP) file, with the exception of the index.php file in a very clean way that will sustain the test of time.

Unfortunately, at the moment jrequire and jinclude do not exist, so we have to work with what we have, and what we have (we know, that’s the third what we have in the same sentence) is core hacks, which are extremely necessary on large Joomla websites (mainly for performance reasons). Clearly, core hacks are not ideal because they get wiped out with future updates, but, if you document the changed files, then you should be OK.

Now, if you, our dear reader, would like to modify a core file that cannot be modified at the template level and you need help, then please contact us. We would love to do that for you, you will love our work, and you will love our prices!

As of version 1.6.0 (which was released 6 years ago in January of 2011), Joomla checks for the presence of a defines.php file at the main directory of the website. If it finds it, then it includes it.

The defines.php file is not part of the Joomla core, it doesn’t even come packaged with Joomla. It is, in fact, a user defined file, and not many people in the world know about it. Unfortunately, most of those who know about it associate with a horrible experience, because it was used to exploit their website (someone took advantage of their Joomla website and uploaded a malicious defines.php to their website).

For us, we think that the defines.php is an excellent tool for security. Yes – we know that the reaction to this statement would be similar to that of Professor Slughorn when Tom Riddle (aka Lord Voldemort) asked him about the Horcruxes, but let us explain…

The defines.php file is the first file to be loaded by Joomla. In fact, the code to load the defines.php is in the top of the main index.php, and not much code is executed before it (save for a few lines of code for checking if the version of PHP is OK and for getting the current time and the current memory usage). This means that it is impossible, from an application perspective, to hack the Joomla website before the inclusion of the defines.php file. This in turn means that, assuming that the website is clean, one can add additional security in the defines.php file mainly by monitoring POST requests (or any type of requests) to the website. Here’s how:

• You create a file called defines.php at the main directory of your Joomla website.

• In the define.php, you add the following PHP code:

  $strPostData = file_get_contents('php://input');

  The above code will grab all the data submitted using POST.

• You check the above data for any suspicious pattern using the PHP function stripos, and, in the case of some malicious content, you just block further processing of the page using the die function.

As you can imagine, you can do a lot of powerful things when it comes to security with the defines.php file. There is one caveat, however, is that the code in the defines.php will only intercept POST requests made to the main index.php file. So, if you have an additional PHP file that is directly accessible by the outside world, then the defines.php code will not be executed for that file. If you want to ensure that all traffic goes through your defines.php filter, then you should ensure that the index.php file is the only PHP file directly accessible by the outside world.

So, was additional security the original purpose behind the defines.php file? If yes, why wasn’t this additional security added to the Joomla core?

No. Additional security was not the original purpose. The defines.php concept was created by Joomla developers to store user defined global constants. For example, if some of your extensions use the now obsolete DS constant, then you can add the following to your defines.php file:

define('DS', '/');

The above line will ensure that any DS constants are properly replaced by the forward slash (/).

Do you need to place code in the defines.php file inside PHP tags?

Definitely! The defines.php must be treated like any other PHP file. You must have the opening and closing PHP tags.

We hope you found this post useful, and we hope that you take advantage of the defines.php file in order to strengthen the security of your Joomla website. If you need help doing so, then please contact us. We are always there for you, our prices are super affordable, and we are experts in Joomla security.

A curious thing happened to us this morning while trying to update a supposedly small Joomla 2.5.28 website to 3.5.1. No – we didn’t suddenly develop xray vision, it was something even curiouser than that!

Let us begin by explaining what we were doing… We were, as stated before, trying to update a small Joomla website from 2.5.28 to 3.5.1 by following the official guide (we only do that for very small websites – since it expedites the work – otherwise, we perform the migration manually). As you may probably know, the second step of a migration from 2.5.28 to 3.5.1 (the first step is always the backup step), would be to change the Update Server from Long Term Support (recommended) to Short Term Support in the Options of the Joomla Update extension, but, when we tried to do that on the client’s site, we saw something unexpected: we saw the Joomla configuration settings page loaded in the popup after clicking on the Save button! Yes – you heard (or read) that right: the popup loaded the configuration settings page! Huh?

That was very weird, we thought. We thought it was a glitch, and so we did it again, and again, and again. Still, we had the same problem, but we did notice something: we noticed that the popup had a large number of fields inside it, mainly because there was a huge ACL database on that particular website. Could it be, we thought?

We then went to the configuration settings page, and made a small modification (we decreased the session lifetime from 60 to 59), and then tried to save it. Unsurprisingly, our little change wasn’t saved! We knew immediately what was happening: the number of the fields on that page (and the Joomla Update popup) was exceeding the maximum number of inputs allowed by the PHP instance on the development server (we were doing the update on a development server).

To verify our theory, we checked the value of max_input_vars (by creating a small PHP page with phpinfo(); in it), and it was set to the default value of 1000. Checking the HTML code of the Joomla configuration settings page revealed that the page had the following fields:

• 2933 select fields
• 583 input fields
• 4 textareas

The total number of fields was 3520, which was more than triple the PHP limit. Clearly, that was the problem!

So, how did we solve the problem?

We solved the problem by creating a .user.ini file in the main directory of the Joomla website (e.g. at the same level of the index.php file) with the following content:

max_input_vars = 8000

As you can see, we increased the maximum number of input variables to 8,000, which is more than double the current need. We then tried to modify a setting in the configuration settings, and this time it worked!

Will the above solution always work?

The above solution should work flawlessly on most recent servers (especially those using WHM). However, on some older servers (or servers with weird environments), it may not work. If it doesn’t, then try renaming the .user.ini file to php.ini file. If it still doesn’t work, then create a file called defines.php in the main directory of your Joomla website, and then add the following code to it:

<?php ini_set('max_input_vars', '8000'); ?>

If the above still doesn’t work, then you will need to modify the max_input_vars value in the master php.ini (e.g. the server’s php.ini file) and then restart Apache. If you are on a shared hosting, then most likely you don’t have access to that file, which means that you will need to beg ask your hosting company if they can do it for you.

How come the client didn’t notice the problem?

The client didn’t notice the problem because the production website resided on a different server with an already altered max_input_vars. Obviously, whoever was working previously on that Joomla website noticed the problem before and addressed it on the production server.

Are there any side effects for having a large max_input_vars value?

Yes – DoS (Denial of Service) attacks – especially ones using hash collisions (aka HashDOS attacks) – will be more dangerous on servers where the max_input_vars directive is set to a high value. This problem, of course, is more relevant to websites with high exposure and many enemies (as these are the websites that typically get hit with more than the FDA recommended intake of DoS attacks).

But why so many fields in the configuration settings page?

Well, there are 2 entities that must be blamed here: the Joomla administrator (that’s you), and Joomla itself. First, if a Joomla website has more than 100 groups and/or access levels, then there must be something conceptually wrong with the website. In other words, the groups/access levels are being used for the wrong purpose. Second, Joomla must not display all the ACL data on one page – not only this is a problem at the server level (where the max_input_vars can be exceeded), but it is also a problem at the client level (the browser may crash because of all these fields). Additionally, Joomla must have a mechanism to check for the max_input_vars and warn the administrator if a certain page is exceeding the effective limit.

We hope you enjoyed reading this post as much as we have enjoyed writing it. We have tried our best to make it as uncomplicated as we can. We reckon, however, that non-developers may find it a bit over their head. If this is the case for you, then please contact us. We are always happy to serve, our fees are affordable, our work is top notch, and we always strive for the cleanest solution possible.

Ever since the elevated permissions (or elevated privileges) exploit in Joomla versions less than 3.6.5 was made public, we are seeing weird things happening to the #__users table on hacked Joomla websites. These weird things can be any (or all) of the following:

• The usernames of all the users are changed to admin.

• All super users are deleted.

• New super users are created.

In this post, we will address the second problem – when all super users are deleted. We will explain how to create a super user from scratch using phpMyAdmin:

• Login to phpMyAdmin.

• Select the database that is used for your Joomla website. You can know which database is used on your Joomla website by checking the $db value in the configuration.php file.

• Click on the SQL tab, and then run the following queries (replace #__ with your database prefix, which is the value of the $dbprefix variable that can be found in your configuration.php file):

  INSERT INTO `#__users` (`id`, `name`, `username`, `email`, `password`, `block`, `sendEmail`, `registerDate`, `lastvisitDate`, `activation`, `params`, `lastResetTime`, `resetCount`, `otpKey`, `otep`, `requireReset`) VALUES (1000000, 'tempsuperuser', 'tempsuperuser', 'youremail@yourjoomlawebsite.com', MD5('temppassword'), '0', '0', '0000-00-00 00:00:00', '0000-00-00 00:00:00', '', '', '0000-00-00 00:00:00', '0', '', '', '0');

  INSERT INTO `#__user_usergroup_map` (`user_id`, `group_id`) VALUES ('1000000', '8');

• That’s it! You should now be able to login to your Joomla website as super user using the username tempsuperuser and the password temppassword. Try it!

Does the above guide guarantees that the super user won’t be deleted in the future?

No – it just recreates a super user from scratch. If you see weird activity in your #__users table, then most likely your website is hacked/vulnerable and it is very possible that you will see the same issue in the future. You will need to update your Joomla website to the latest version, ensure that your extensions are all up-to-date, and follow our important Joomla tips here.

If the above guide is a bit over your head, or if you need help in securing your website, then please contact us. We are always here to help, we work 24/7/365, our fees are super affordable, and we are the friendliest Joomla developers on this planet (and many other uninhabited planets)!

A new client emailed us this morning and told us that the titles of modules assigned to a certain position are not showing. He told us he was using the latest version of Joomla, and that he ensured that “Show Title” was set to “Show” in the module’s settings. He told us that he’s confident that it was not a caching problem, since he was not using any type of caching and all of his other changes were showing immediately… “What could it be?”, he desperately asked us…

Luckily, we have seen this problem many times before… The problem is neither caused by caching nor by the module itself – the problem is caused by the template. Let us explain…

When you want to create a new module position on your website, you add something like the following code to your template’s index.php:

<jdoc:include type="modules" name="module-position" type="xhtml" />

The above ensures that any enabled module assigned to module-position will be displayed on the pages it is assigned to. Additionally, the above ensures that the styling of the module is set to xhtml, which means (among other things), that the module’s title will be displayed.

However, if you add something like the below to your template’s index.php…

<jdoc:include type="modules" name="module-position"/>

…then the default styling will be used for the modules assigned to module-position, and that default styling is none (note that Joomla’s official documentation erroneously states that the default styling is table – this is wrong – the default styling is none). The none default styling means that the title will not be displayed, and no preset styling will be applied to the module.

As you may have probably guessed when reading this post, our client used something like the latter line in his template, and that’s why the module title was not showing. Adding type=”xhtml” to the module declaration in the template fixed the problem…

If you have the same problem on your Joomla website, then make sure you add type=”xhtml” in the module’s declaration in your template (or make sure you replace type=”none” with type=”xhtml”). Once you do that, clear your Joomla cache and see if it works. If it doesn’t, then please contact us. We will help you fix the problem in no time and for a very affordable fee.

A client of us told us that he didn’t like the default CAPTCHA (by the way, in case you really want to know, CAPTCHA stands for Completely Automated Public Turing Test to Tell Computers and Humans Apart) field that comes with RSForm. He said that the letters looked very small, and that it affected his conversions.

He asked us to switch all his forms to use reCAPTCHA instead, and so, we downloaded and installed the reCAPTCHA RSForm plugin on the Joomla website, we enabled the plugin, and then we went through each and every one of his forms, and switched all the CAPTCHAs to reCAPTCHAs.

While having the strange feeling that we missed something, we went to one of the forms to see if it’s working on not, but instead, we saw the following message on the very top of the page…

To use reCAPTCHA you must get an API key from http://www.google.com/recaptcha/whyrecaptcha

…and, in the place where the CAPTCHA should have been displayed, we were seeing the following error:

Input error: k: Format of site key was invalid

Aha, so our instincts were right, we did forget something: we forgot to generate a reCAPTCHA API key, and so we went to the above URL (while logged in to our Google account), and followed the instructions to generate a Site Key and a Secret Key. Once we did that, we went back to the Joomla website, we enabled the Captcha – ReCaptcha core system plugin, and we added our Site Key and our Secret Key in its settings.

We then went to an RSForm form on the website, but still, we saw the same error… That didn’t make sense, we thought… How come we are still seeing the same error when we just added the API keys to the core reCAPTCHA plugin. Logically, the next step was to find out how the above error was triggered, and so we searched the Joomla files for the following line: To use reCAPTCHA you must get an API key.

The search revealed that the error was thrown by the function rsform_recaptcha_check_answer in the file relib.php (which was located under the plugins/system/rsfprecaptcha folder), and it was thrown because $privkey (which is reCAPTCHA‘s Secret Key) was empty. So we checked how this variable was passed to the function, and it turned out that the plugin was grabbing this value from the settings of the RSForm extension, which was obvious in following line (the line was located in the rsfprecaptcha.php file, which was in turn located in the plugins/system/rsfprecaptcha folder):

$privatekey = RSFormProHelper::getConfig('recaptcha.private.key');

Aha! So RSForm had its own reCAPTCHA settings, and so it wasn’t using the settings of the core reCAPTCHA plugin. So we logged in to the backend, and then we clicked on Components -> RS Form Pro -> Configuration, and we found a tab called reCAPTCHA. We entered our Site Key in the Public Key field and our Secret Key in the Private Key field, we saved the settings, and then we visited the form and this time it worked! Hooray!

Now if you, our dear reader, are seeing the above error on your website, then make sure that you enter reCAPTCHA‘s Site Key and Secret Key in RSForm‘s reCAPTCHA settings. If, after doing that, you are still seeing the same error, then please contact us. We will solve the problem for you quickly, affordably, and professionally!

While processing what seemed to be a very simple request from one of our clients this morning, we encountered a very weird issue. The simple request was that the client wanted a simple registration form on his website, and the weird issue was that, for an unknown reason, we were redirected to the login form everytime we tried to visit the registration URL.

Here’s what we did:

• We created a new menu item of type Registration Form.

• We set the alias of that menu item to be register.

• We visited http://www.[ourclientjoomlawebsite].com/register, but we were redirected automatically to http://www.[ourclientjoomlawebsite].com/register?view=login, which is the login page.

We checked everything – we checked whether the client had sh404SEF installed, and he didn’t. We checked the .htaccess file for weird redirects, but the client was using a very standard .htaccess file.

After a lot of investigation, we discovered that the root cause of the problem was that user registration was disabled on the website. We did the following to solve the problem:

• While in the backend of the Joomla website, we clicked on Users -> Manage (which displayed a listing of all the Joomla users).

• We clicked on Options on the top right.

• We changed the value of Allow User Registration from No to Yes.

• We saved the settings and then we tried again.

• This time it worked!

Joomla was doing the right thing by redirecting the registration page to the login page, but, in our opinion, Joomla must have warned us when we created the menu item of type Registration Form that Allow User Registration was disabled. Had it done so, it would have saved us a lot of time!

In any case, if you, our dear reader, are having the same problem on your Joomla website, where the registration page is redirecting to the login page, then make sure that Allow User Registration is enabled on your website. If it is, then try using another browser (some browsers temporarily save redirects) and see if the problem is fixed. If it still isn’t fixed, then please contact us. Our fees are super affordable, our work is super professional, and we are the friendliest developers on planet Earth.

We had a weird case late last week: a new client contacted us and told us that all the links on his website were pointing to the homepage. He told us that he was using NGINX, and he told us that he thought that NGINX is the cause of the issue.

So, we checked the website – it wasn’t that all the links were pointing to the homepage, it was that all the links were displaying the homepage. For example, http://www.ourclientjoomlawebsite.com/ and http://www.ourclientjoomlawebsite/link-1 and http://www.ourclientjoomlawebsite.cm/link-2 were all displaying the contents of the homepage. The links were not being redirected, the links were just displaying the contents of the homepage.

The first thing that we did was to empty the $live_site variable in the site’s configuration.php file. So, we changed:

public $live_site = 'http://www.ourclientjoomlawebsite.com';

to:

public $live_site = '';

Unfortunately, that didn’t work, but, deep down inside, we didn’t expect it to: it was a long shot…

The second logical thing to do was checking the nginx.conf, which is the NGINX configuration file. Here’s what we did:

• We ssh’d to the server as root.

• We opened the file nginx.conf which is located under the /etc/nginx folder.

• We checked if the location / statement existed in the code (which is needed for Joomla to work correctly under NGINX), and it didn’t, so we added the following code immediately before the last line (the last closing curly bracket):

  server {
    location / {
       expires 1d;
       try_files $uri $uri/ /index.php?$args;
    }
  }

• We saved the file, and then restarted the NGINX webserver by issuing the following command in the shell:

  /etc/init.d/nginx restart

  Note: If after restarting nginx you see the following error: Restarting nginx (via systemctl): nginx.serviceJob for nginx.service failed because the control process exited with error code. See “systemctl status nginx.service” and “journalctl -xe” for details., then the problem might be that you didn’t put the location code block inside a server code block.

• We tested the website, but still, the same problem…

Odd, we thought… Mind you, disabling SEF did solve the problem, but, naturally, that wasn’t a real solution and it wasn’t an option for the client…

After spending a few hours beating around the bushes, we started pondering about exploring cheap suicide tactics, like holding our breath for too long. Luckily, it didn’t come to this, as we saw something curious: when we visited the administrator section of the website, we noticed that no CSS style was applied to the login page, which was weird. Checking the HTML code, we noticed that the following file was requested:

<link href="/templates/isis/css/template.css?de6fd0cafae3e6687ab8a1abd290a957" rel="stylesheet" />

But, from our experience, the above line should be something like:

<link href="/administrator/templates/isis/css/template.css?de6fd0cafae3e6687ab8a1abd290a957" rel="stylesheet" />

So, why is administrator missing from the link? Is it related to the original problem? It must be related to the original problem – because it’s a substantial issue with the URL, which is exactly what the original problem is.

The missing “administrator” problem restored our hopes, because not only we were increasingly confident that it was directly connected to the original problem, we were also confident that we were able to fix it, as the problem was easily traceable through code.

Once we investigated this issue further, we discovered something interesting, the JURI::Base() function was always returning empty. Huh?

So, we checked the implementation of the JURI::Base() function in the file uri.php file which is located under the /libraries/joomla/uri/ folder, and we discovered something even more interesting, $_SERVER[‘PHP_SELF’] was empty! In fact, it was always empty!

Joomla relies on $_SERVER[‘PHP_SELF’] heavily in its handling of SEF URLs and in its backend, which means that the original problem is really caused by $_SERVER[‘PHP_SELF’]. We found the root cause of the problem! Hooray! The next step was to actually solve the problem…

It was obvious that the problem was caused by the NGINX web server, and so we made an extensive research about the issue, and we discovered that, in order to address the problem of the empty $_SERVER[‘PHP_SELF’], a certain PHP global variable, called PATH_TRANSLATED, must be explicitly defined in the fastcgi_params file that is used by the nginx.conf file. The fastcgi_params is typically found in the /etc/nginx folder (at the same level of the nginx.conf folder).

Another thing that needed to be done was to change the value of cgi.fix_pathinfo in the php.ini from 0 to 1.

Here’s how we did all the above:

• We added the following code to the nginx.conf file (instead of the code added near the beginning of this post):

  server {
    location / {
  	expires 1d;
  	try_files $uri $uri/ /index.php?$args;
  	fastcgi_split_path_info ^(.+?\.php)(/.*)$;
      if (!-f $document_root$fastcgi_script_name) {
  		return 404;
      }
  	fastcgi_param HTTP_PROXY "";
  	fastcgi_pass 127.0.0.1:9000;
  	fastcgi_index index.php;
  	include fastcgi_params;
    }
  }

• We opened the file fastcgi_params (again, which is located under the /etc/nginx ) folder and we added the following line (after the PATH_INFO line):

  fastcgi_param  PATH_TRANSLATED    $document_root$fastcgi_script_name;

• We restarted NGINX by issuing the following command:

  /etc/init.d/nginx restart

• We opened the php.ini file (please check phpinfo(); to know the exact location of your working php.ini file) and we changed the following line:

  cgi.fix_pathinfo=0

  to:

  cgi.fix_pathinfo=1

• We restarted php-fpm by issuing the following command in the shell:

  /etc/init.d/php7.0-fpm restart

• We checked the website and the problems, all the problems, were solved! The URLs were working and the administrator section was working! Double hooray!

We are not sure whether this problem applies to all NGINX servers, or to just NGINX servers using PHP 7. In either case, the official Joomla documentation about NGINX should be updated, as it no longer reflects a guaranteed working solution.

If you’re running NGINX and your website is displaying the homepage for all the URLs, then please follow our guide above. It should work for you as it worked for us. In the unlikely case where it doesn’t work for you, then please contact us. Our service is fast, our work is professional, we are reliable, and our fees are super affordable.

We know, it’s been a couple of weeks since we last wrote a post on this blog – we’re not getting lazy, it’s just that we had quite a few large projects in December. Still, we feel guilty, and so we decided to write a post right now, and if you want to know what right now is, then it is 8:52 PM EST on December 31^st, 2016. Yes, it’s new year’s eve, and we’re writing a blog post!

OK – enough with this introduction. Let’s start with the juicy stuff…

As a Joomla administrator, you may have seen one of the following errors on your Joomla website (or maybe you have seen both of them):

Fatal error: Allowed memory size of x bytes exhausted (tried to allocate x bytes)

Fatal error: Out of memory (allocated x) (tried to allocate x bytes)

To the untrained eye, the above errors look more or less the same: they look like a memory problem, and this is correct, they are memory issues. But, they are completely different memory issues. Unfortunately, most Joomla administrators think that the above errors are the same, and they think that the solution presented in this post will fix both of them. This isn’t the case…

In fact, increasing the memory_limit in the php.ini or the .user.ini file may address the first problem, but it certainly will not address the second problem. You see, the first problem means that the Joomla website has exceeded the allowed memory limit for the PHP instance, while the second problem means that the Joomla website has used up all the available memory. Let us give you an example…

Say your default php.ini has a memory limit of 32 MB, and say you have 2 GB of RAM on your server… If a page on your Joomla website exceeds 32 MB, then you will see the Allowed memory… fatal error. However, if the memory limit in your php.ini file is set to a very high number, like 4096 MB (or 4 GB, which should never be the case), and you have 2 GB of RAM on your server, then if a page on your Joomla website needs more than 2 GB, then you will see the Out of memory fatal error.

Now, if you read the above paragraph very carefully, then your question would be, why would anyone, in a real life scenario, see the Out of memory fatal error? Good question!

In fact, in a real life scenario, the Out of memory fatal error is never thrown because all of the server’s memory was exhausted, it is thrown because a hard limit set at the Apache level is exceeded, and that hard limit is set using the RLimitMEM directive in Apache. For example, having the following in your httpd.conf file…

RLimitMEM 33554432

…means that there is a hard limit of 32 MB on the memory usage on your Joomla website, which means that any page on your Joomla website needing more than 32 MB will throw the Out of memory fatal error (assuming the PHP memory_limit is higher than 32 MB, otherwise, the page will throw an Allowed memory fatal error).

So, how to address an “Out of memory” fatal error?

Obviously, addressing such an error can be done by either commenting out the whole RLimitMEM line in the httpd.conf file, or by increasing the value of RLimitMEM (also in httpd.conf) to a higher value, and then restarting Apache. Note that if you’re using WHM, you can change the value of RLimitMEM from within WHM, here’s how:

• Login to the WHM console as root.

• In the search box on the top left, search for Apache Configuration.

• Click on Apache Configuration.

• Scroll down a bit and click on Memory Usage Restrictions.

• In the Configure Apache RLimits page, you can either disable RLimitMEM or you can just set it to a higher value. Note that you will need to restart Apache for the settings to take effect.

If you’re seeing the problem and you are on shared hosting, then you should contact your hosting company, and they may increase it for you (but don’t hold your breath, as they most likely won’t).

If the above doesn’t fix the problem, then please contact us. We are always ready to help you (even on new year’s eve), our fees are super affordable, and our work is top notch!

On a final note, we would like to wish you a Happy New Year 2017, may it be full of happiness, health, and prosperity for you and your family!

Let us tell you a little story…

A huge Joomla website that we fully manage uses Disqus for commenting. The website receives an insane amount of traffic and it is in the Alexa top 10K websites in the US. About 4 months ago, we noticed that the Disqus commenting section had irrelevant ads, but we thought that the client authorized these ads, and so we dismissed this as a non-issue.

Two months ago, the client contacted us and complained about these ads, and so we told them that we were under the impression that these ads were authorized from their end, the client responded that this wasn’t the case, and so we delved deeper into the mysterious world of Disqus.

After investigating the issue, we discovered that Disqus had a platform for displaying ads called “Reveal” which was made very aggressive beginning of the year (beginning of 2016). “Reveal” pays money to publishers, and it was paying the Disqus account owner (who was a developer who worked on the website back in 2011, before we started managing it) about $3,000/month by ACH (ACH stands for Automated Clearing House, another term for direct deposit). Naturally, we disabled “Reveal” in Disqus in order to stop this madness.

We then urged our client to do one of the following:

• Switch to another Disqus account that they own in order to prevent this in the future, and then ask Disqus to copy the data from one account to another. This option was ideal, but we had serious doubts on whether Disqus will accommodate this request; why should they? It is a free service after all and no one should expect support for anything that was given away for free.

• Talk to the previous developer and ask him to surrender his admin rights on the account. We also had doubts about this – a developer who sneaks in his information to make a quick buck out of a very reputable client is not a developer that will give away such things easily (or cheaply).

• Use a Joomla extension such as JComments for commenting platform and then copy the data from Disqus to JComments. This suggestion was the weakest, in our opinion, but it was the only one that was guaranteed to work. Disqus has a solid API, and that API can be used to build the migration tool.

After discussing the issue internally for a few days, the client told us that they will go with the second option, and so we closed the issue on our end. However, we did have this little gut feeling that we will hear from the client about this particular issue in the near future.

And so we did: a couple of weeks ago, the client called us and told us that they were seeing those ugly ads again, and that they needed them removed. So, we tried to login to Disqus, and this time, it didn’t work for us. We asked the client whether they changed their username and/or password on Disqus, but they said they didn’t. We understood what happened immediately: the previous developer discovered that he didn’t receive that $3K in the previous month, so he re-enabled “Reveal” (the Disqus advertising platform, in case you just skipped to this paragraph), and he changed the client’s Disqus password. That was nasty but it was expected.

So, we presented the client with the above 3 options again, and they told us that they tried contacting the previous developer, but he didn’t reply. So, we asked the client to see if they can contact Disqus about this (we also told them to expect very little), and we told them that meanwhile, we will migrate their Disqus comments to JComments.

Immediately after we finished the Disqus migration script to JComments, the client contacted us and told us that, to their (and our) surprise, Disqus replied to them about this and they were willing to help. Here is the email that Disqus sent to our client:

Hi,

If your situation demands more than being set as the Primary Moderator for the forum and having access to the Moderation Panel, please provide us with a detailed description of the issue you are facing.

Woohoo! Disqus did beat our expectations (and those of our client), and they only requested from us to create a specific page for them in order to prove ownership of the website. So, we quickly created a page called “Hi Disqus!” which contained the following text:

“This is a message for Disqus. Thank you for giving us back control over our Disqus account.”

We also created a Disqus account for our client (using an email address belonging to our client’s domain), and we forwarded the account information to our client, telling them to ask Disqus to make the account the master account.

In less than 24 hours, Disqus did just that! They were unbelievably helpful and they were not as we thought they were. We were surprised that they even responded, let alone granting our client full control over his Disqus comments.

We were thrilled that this whole thing ended in the best way possible, and so was our client.

Now if you, our dear reader, lost control over your Disqus account (or didn’t have full control in the first place), then we suggest that you contact Disqus (they will reply to you). If you need help with the process, then please contact us. We have done this before, our fees are super affordable, and we will do our best to turn that nightmare of yours into a positive experience.

Frequently, we get calls from potential clients asking us to fix problems on their Joomla websites, only to find out afterwards that we are unable to do any work for them because of the nature of their websites. Clearly, this leads to disappointment on both ends: the clients need our work and they can’t have it, and we need the clients’ money but we can’t have it.

In order to minimize the disappointment on both ends, we are publishing the list of websites that we cannot work on. Without further ado, here’s the list:

• Websites with adult material, including nudity websites.

• Websites selling alcohol and/or promoting the use of alcohol.

• Websites related in any way to gambling/betting. This includes websites primarily discussing gambling/betting tips and tricks, gambling/betting websites that do not use real money for transactions, and websites primarily selling products that are used for gambling/betting.

• Websites selling narcotics and/or promoting the use of narcotics. This includes websites selling and/or promoting cannabis for medical or non-medical purposes.

• Websites with extreme political content (e.g. political websites that incite hate).

• Websites used for scamming visitors.

• Websites knowingly and intentionally selling counterfeit products.

• Websites glorifying the suffering of any living thing (living thing may be an oxymoron, but you get the idea).

• Websites selling (and/or promoting) magic services and/or products.

• Websites with obscene content.

• Websites intentionally promoting fake and/or misleading content.

• Websites belonging to organizations and/or countries that are on the US or the Canadian embargo list.

The list above reflects our solid values that we believe will promote a better and cleaner Internet (and world) for everyone (we hope that all other developers follow suit, but we know that it’s almost impossible, after all, money is insanely seductive to the absolute majority of developers out there).

If you are in doubt whether your website falls into our blacklist or not, then please contact us, and we’ll give you a definitive and straight answer.

Note: Before you use our script, please backup your website (filesystem + database). While we have tested our script many times it’s always better to be on the safe side and backup your Joomla website first.

Important note: The script is provided “as-is”, we can’t claim responsibility for it. We can’t even claim that it will work for you – in fact, running it may even turn on and off the lights in your house incessantly until a 33 year old man who speaks 7 languages (with Gibberish as his first language) says the word “itoctopus” correctly! You have been warned!

We don’t think highly of tags, especially Joomla tags. In previous posts on this blog, we mentioned why they were bad from an SEO perspective and from a technical perspective. Nevertheless, many of our clients use them, and so we cannot ignore the fact that they are there.

In fact, quite a few Joomla administrators were using tags even before Joomla integrated a tag system into its core. Some of them used extensions for tagging, others used meta keywords in articles and simulated a tagging system on their Joomla website.

Naturally, almost all of these Joomla administrators wanted to switch to Joomla’s own tagging system ever since it was introduced, but it wasn’t an easy task: the switch was technically a (costly) migration from one system to another. So, the absolute majority of them kept on using what they were using, with a dream that one day, someone, somewhere, will create a tool that will migrate their tags to Joomla’s tagging system.

Well guess what? For those using meta keywords for tagging the someone is itoctopus and the somewhere is the charming city of Montreal: we have created a script that will smoothly migrate the meta keywords of Joomla articles to tags. Yes, we’re not kidding, you can download the script from here!

All you need to do is to extract the zip file, and copy the unzipped PHP file called migrate-meta-keywords-to-joomla-tags.php to the root directory of your Joomla website (e.g. at the same level of your index.php file), and then point your browser to http://www.[yourjoomlawebsite.com/migrate-meta-keywords-to-joomla-tags.php. Once you do that, the script will automatically generate tags out of your meta keywords and assign them to the appropriate articles. That’s it, you don’t have to do anything else! Convenient and simple, isn’t it?

So, how does the script work?

The script grabs all the articles (that have meta keywords) from your #__content table, and then it loops through each entry, generating tags using a modified createTagsFromField function (this function exists in the tags.php file which is located under the libraries/cms/helper folder), and tagging articles using the tagItem method of the JHelperTags tags class.

Will the script create redundant data if it’s run more than once?

No – it won’t. If you run the scripts multiple times, then the existing tags won’t be recreated, and the articles will be untagged and retagged. In other words, you won’t end up with duplicate data if you run the script multiple times.

How long will the script take to execute?

It depends on the number of articles with meta keywords that you have on your website. We found that it takes around 1 minute to process 400 articles on a medium power server. Note that the more tags you have in the #__tags table, the slower the script will be (e.g. the second batch of 400 articles will take a bit more than a minute to process, and the third batch will take more than the second batch, etc…)

Will the script timeout if there are too many articles with meta keywords?

In most cases, it shouldn’t. We have ensured that there are no limits on the amount of time that the script can take. Nevertheless, some server settings (especially on cheap hosts) disallow any alteration of script timeout limit. If this is the case for you, then you will need to use the LIMIT MySQL clause in the main query to ensure that you process the articles in batches (we have explained how in the script).

Will too many tags cause the Joomla website to be slow?

Definitely. Especially on the backend when you are trying to create/edit articles. One of the posts we have linked to at the very beginning of this post explains this issue in details and how to overcome it.

Can I use this script anywhere for free?

Yes – it’s free for both personal and commercial projects. No need to alert us if you want to use the script. But, if you need help using it, then you can always contact us, but please note that our super affordable fees apply.

Note: Any reference to ‘#__’ in this post must be replaced with the database alias of your website, which is defined in the configuration.php file.

This noon, a client called us and told us that whenever he was trying to login to his Joomla website, he was seeing an empty page. He was quick to point out that by empty page he meant a page with virtually no menus and no modules, and not a blank page (which is the sign of a fatal error). He said he was only seeing the Joomla logo, the notification messages module, and the copyright at the bottom of the page. In short, the client was seeing something exactly like the below (we changed the name of the client’s website to Joomla):

Luckily, we saw this error before, and we remember it had to do with either a corrupt #__assets table or a corrupt #__viewlevels table. We weren’t completely sure (unfortunately, we haven’t documented the solution on our blog).

Since the #__assets table is much larger and much more complex than the #__viewlevels table, we decided to start by checking whether the #__viewlevels table is corrupt or not. So we logged in to phpMyAdmin and we compared the table to the one of a fresh Joomla install, and we noticed the following couple of differences:

• There was one additional row in the first table (e.g. that of the client).

• The rules were not identical for the corresponding rows.

We immediately concluded that the problem was a corruption in the #__viewlevels table, and so we replaced the data of in the first table with that of the fresh Joomla install the following way:

• We truncated the table by issuing the following SQL command in phpMyAdmin:

  TRUNCATE TABLE `#__viewlevels`;

• We added the clean data by issuing the following query:

  INSERT INTO `#__viewlevels` (`id`, `title`, `ordering`, `rules`) VALUES
  (1, 'Public', 0, '[1]'),
  (2, 'Registered', 2, '[6,2,8]'),
  (3, 'Special', 3, '[6,3,8]'),
  (5, 'Guest', 1, '[9]'),
  (6, 'Super Users', 4, '[8]');

• That’s it!

We then tried to login to the website, and this time it worked. Hooray!

So, what caused this problem?

We think that either the client or someone working for the client mistakenly caused the problem by modifying the Joomla ACL (Access Control List). This is a common problem and we really think that Joomla must implement measures to automatically mitigate this problem (one shouldn’t be allowed to intentionally or unintentionally corrupt the Joomla ACL while working from within Joomla).

If you are seeing an empty backend after logging in to your Joomla website, then most likely your #__viewlevels table is corrupt. Follow the above instructions to fix it and you should be all set (make sure you backup your database first). If you think the above is a bit over your head, or if you’re just a bit afraid of doing it yourself, then please contact us. We are always ready, we are always happy to help, and our fees are super duper affordable.

There is a confusing “feature” in Joomla, and it is that the CMS, unintentionally, can have multiple links pointing to the same article. For example, if you go to the following URLs http://www.[your-joomla-website]/category-1/12-test-article.html and http://www.[your-joomla-website]/category-2/12-test-article.html then both URLs will work despite the fact that they point to the same article, and despite the fact that Test Article belongs to Category 1, and not Category 2.

And why is that a problem?

It is a problem, and a big problem, for that matter, because it will negatively affect rankings in search engines, especially Google rankings (Google will think that the website administrator is trying to spam its database with multiple listings for the same article). Imagine if you have 100 categories and 500 articles on your website. A worst case scenario will mean that you will end up with 50,000 different URLs instead of 500 – which will certainly dilute the strength of each and every URL. From an SEO perspective, this is a nightmare…

So, what causes this issue?

The issue is caused by the parse function which is located in the router.php file (which, in turn, is located under the components/com_content folder), which only takes into consideration the last part of the link for content items when displaying them. So, not only you can have a different category alias in front of the article alias, you can literally have anything! So even the following will work: http://www.[your-joomla-website]/abcdefg1234567/12-test-article.html (try it!), when abcdefg1234567 is not even a category alias (by category alias, we mean a menu item of type Category Blog with an alias that is the same as the alias of the category it is pointing to). What’s even worse is that the home menu item ID (which is the default one) will apply to these weird links, and so all the homepage modules will be assigned to them…

So, how can this problem be solved?

The best solution to this problem is through the SEF Advanced Link, which is a secret feature in Joomla (its mystery is only exceeded by its power, just like the Continuum Transfunctioner). All you need to do is to set it to 1, and we explained how to do that here. Now, if you’re wondering, the reason why doing this will fix the problem is because when this is set, the article ID will no longer appear in the URL, and so Joomla will have no option but to check all the parts of a URL in order to know which article it should display, as the article alias by itself is not sufficient to correctly locate an article (in Joomla, article aliases do not have to be unique).

The downside of the fix above is that it will alter the URL structure of the Joomla website (it will remove the ID of the article from the URL), which can result in a problem bigger than the original problem especially if you have too many articles on your website. So, what you will need to do is develop as simple script (and maybe place it in the defines.php file under the root directory of your website) which will check if the current URL has an ID in front of the alias, and, in case it finds one, then it will issue a 301 REDIRECT to redirect to the URL without that ID.

We have tried out best to make this post as easy as possible, but we reckon that it’s still somehow complex to implement, especially with the last part (redirecting from the old links to the new links). So, if you need help with the implementation, then please contact us. Our work is professional and efficient, our rates are affordable, and we are the most reliable Joomla developers on planet Earth!

A new client called us a couple of hours ago and told us that he had a problem on the VirtueMart store on his Joomla website. He said that early in the morning a client called him and asked him about the status of his one week old order which he paid for through PayPal. Our client was perplexed, because he doesn’t have PayPal as payment method (he sells e-cigarettes and accessories on his online store, which means that he can’t transact through PayPal [PayPal doesn’t allow merchants selling e-cigarettes to transact through their system]). So, our client searched through the VirtueMart orders, and found an order matching that of his client (our client doesn’t use VirtueMart to see the orders, as he uses an application called ShipWorks to do that [which only lists orders that transact successfully through Authorize.net], that’s why he didn’t see that order before). To make a not-so-long story concise, the website made the sale, but someone else took the money.

We immediately investigated the issue, and we noticed the following:

• The website was using the latest version of Joomla.

• The website was using the latest version of VirtueMart.

• The PayPal VirtueMart plugin was installed on the website, and it was configured to accept money to someone with a French email address (the email ended with .fr). The fraudulent orders totaled a tad over $200.

• There was a weird super user with an even weirder registration date. That user never logged in to the website (according to Joomla anyway).

In order to address the problem, we did the following:

• We uninstalled the PayPal plugin (we took note of the fraudulent email account first and forwarded it to the client).

• We removed the weird user from Joomla.

• We changed the password for the following: the Joomla super user, the cPanel account, the FTP account, and the MySQL database powering the Joomla website.

• We scanned the website for viruses/backdoors (there were none).

• We installed our own firewall on the Joomla website.

So, what caused this problem?

We don’t know (we weren’t commissioned to do so), but we have the following theories in our mind:

• The client did not update the Joomla website to 3.6.4 immediately, so his website suffered from the elevated permissions exploit for a while before updating it, and during that time, someone registered the weird Joomla super user, and used it to install and configure the PayPal plugin. This is a weak theory, however, because the weird Joomla account was never used.

• Someone knew the super user password, and used it to install the PayPal plugin and configure it. This is a plausible theory because the super user password was super easy.

• An attacker used a hidden exploit in VirtueMart and installed that plugin. This is improbably but not impossible, and if it is true, then many e-commerce shops are at risk.

We can’t be really sure of the root cause of this problem (we were not commissioned to investigate further), but we’re leaning towards the simple theory, which is the second theory, where someone knew the super user password and used it to install and configure the VM PayPal plugin.

If you’re running a VirtueMart powered online store, then we suggest you verify your payment settings; the prospects of someone changing the payment information on your website to his own are really terrifying. If you found something fishy on your website and you need help, then please contact us. We will clean your website, we will protect it, and we won’t charge you much.

Note: This post is extremely advanced. Only try to implement the below if you’re a solid developer and after backing up the Joomla website (database + filesystem).

We had a weird case this morning. A client called us and told us that her staff were able to edit certain articles only once every day – in other words, they edit the article the first time, they save it, and then they can’t do edit it again until the next day.

That was very weird, we thought. We haven’t seen something remotely similar in a decade of Joomla consulting. So, we visited the website, and we edited one of those cursed articles, and then we saved it and closed it. To our surprise, the article was no longer editable (the article title was no longer a link).

We knew this whole thing had something to do with the #__assets table, but we just didn’t know where to look. However, after examining the issue further we noticed that the all of these articles belonged to the same category, which was the Uncategorized category.

So We looked up the entry of the Uncategorized category in the #__assets table, and we couldn’t find one, we were, however able to find an entry in the #__assets table matching the value of the asset_id field of that category, and that entry was that of a module (its name was com_modules.module.334). Clearly, that was wrong…

After some testing, we discovered what was happening, when an article belonging to the Uncategorized category was edited and saved, a wrong entry for that article was created in the #__assets table. Why is it a wrong entry? Well, because the parent of that entry is not the right parent (it is a module instead of the Uncategorized category). Obviously, fixing the problem consisted of recreating an entry in the #__assets table for the Uncategorized category. Here’s how we did it:

• We created a new category, we called it “Test Category”. We made sure that it had the same parent category as that of the Uncategorized category (it actually had no parent).

• We went to the #__assets table, and then we searched for the entry corresponding to “Test Category”.

• We changed the value of the name field for that entry in the #__assets table to be com_content.category.33 (where 33 is the ID of the Uncategorized category in the #__categories table), we then changed the title field in the #__assets table to be Uncategorized.

• We changed the value of the asset_id field of the problematic (Uncategorized) category to that of the ID of the row that we have just changed in the #__assets table.

• We deleted the Test Category from the #__categories table.

• We ran the following query on the #__assets table…

  DELETE FROM #__assets WHERE name = 'com_content.article.nnn';

  where nnn is the article ID that we were not able to edit. Doing this removed the bad asset information associated with that article.

• That’s it. The problem was fixed.

So, what caused the problem in the first place?

We don’t know what caused this data corruption in the #__assets table, but we think it was one of the following:

• A previous migration that wasn’t 100% clean.

• A poor extension that corrupted the #__assets table.

• An incorrect manual edit to the #__assets table.

Why were they able to edit articles the next day?

Well, because the website had a cron job to optimize the #__assets table at midnight of each day. The optimization consisted of pruning all the articles entries in the #__assets table as per our guide here. Once the cron ran, all the bad references in the #__assets table were removed, and so editing articles belonging to the Uncategorized category was possible again.

If you can’t edit articles on your website, then check your #__assets table, and implement the guide above. If you have problems with the implementation, or if you’re too scared to do it, then please contact us. We will do the work for you swiftly, professionally, and for very little money.

Note: You must replace #__ in the post below with your database alias which is the value of the variable $dbprefix that is defined in the configuration.php file of your Joomla website.

Another note: Always backup your Joomla database before making any modifications to it.

A not so uncommon fatal error on Joomla websites is the following error:

Application Instantiation Error: Table ‘db.#__session’ doesn’t exist

What the above error essentially means that Joomla is unable to read/write to the critical session table. This can be caused by one of the following (the below are listed in order of occurrences):

1. The session table is corrupted: The most common reason of the Table ‘db.#__session’ doesn’t exist error is corruption in the #__session table, this is especially the case when the table is using the MyISAM database storage engine (which is infamous for causing table corruption). If this is the case, then you should repair the table, which can be done by issuing the following query in phpMyAdmin:

   REPAIR TABLE `#__session`

   The above query should be sufficient to repair the table. If it doesn’t work, then you may need to reconstruct the table from scratch, which is described in the following section.

   Note: We recommend switching the database engine of the #__session table to InnoDB or to MEMORY as MyISAM is a very flimsy database engine.

2. The session table was deleted: One some occasions, the cause of the problem is a deleted #__session table. In this case, you will need to reconstruct the table. Reconstructing the table also works if the table is corrupted but cannot be repaired using the REPAIR syntax. Reconstruction of the #__session table is a safe process because the information in the #__session table is meant to be volatile and can be safely deleted at anytime.

   To reconstruct the table, you will need to run the following queries in phpMyAdmin (the following is compatible with Joomla 3.x):

   DROP TABLE `#__session`;
   CREATE TABLE IF NOT EXISTS `#__session` (
     `session_id` varchar(191) NOT NULL DEFAULT '',
     `client_id` tinyint(3) unsigned NOT NULL DEFAULT 0,
     `guest` tinyint(4) unsigned DEFAULT 1,
     `time` varchar(14) DEFAULT '',
     `data` mediumtext,
     `userid` int(11) DEFAULT 0,
     `username` varchar(150) DEFAULT '',
     PRIMARY KEY (`session_id`),
     KEY `userid` (`userid`),
     KEY `time` (`time`)
   ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 DEFAULT COLLATE=utf8mb4_unicode_ci;

   The above code should drop your #__session table and then reconstruct it.

3. The database user doesn’t have full read/write control over the session table: A not so uncommon cause of the problem is a user which doesn’t have full privileges over the #__session table. This is likely to happen when the Joomla database is managed at a granular level, and not through a mainstream system (like WHM). Fixing the problem is usually done by granting full read/write permissions on the #__session table (this is typically done by a system administrator).

4. The database is empty: Last week, we had the Table ‘db.#__session’ doesn’t exist problem on a website, and it turned out that the database was empty. It didn’t have a single table, not one. The database did exist and the connection parameters were correct, but it was empty (if the database didn’t exist, or if we had the wrong database credentials in the configuration.php file, then we would have seen the Application Instantiation Error fatal error instead). We fixed this problem by reverting to an old backup. The backup didn’t contain the most up-to-date data, but it was much better than nothing.

5. The configuration file is pointing to a different database: On some instances, we have seen a configuration.php file where the database parameters are pointing to an existing database, but not to the Joomla database. This is not common but it can happen, and it is usually caused by a human error: for example, the system administrator copied the content of the Joomla database to another database, emptied the original database (and used it for a different purpose), but did not update the parameters in the configuration.php file to point to the new database. Fixing the problem simply consists of updating the database parameters in the configuration.php file to point to the actual database.

We hope that you found our post useful and that it helped you solve your problem. If it didn’t, then you can always contact us. We are always happy to serve, our prices are super cheap, and we really love to have you as a client and a friend!

At itoctopus, we love doing things in different ways, this helps us broaden our horizon and it also allows us to provide several alternative solutions for our readers, who may prefer one method over another.

In a previous post, we have discussed, in details, 2 methods for hiding the Joomla version. The first one consisted of modifying a core file (which we don’t usually recommend), and the second one consisted of just modifying the template file.

In this post, we will explain a third method to do that, and it is through the .htaccess file. Yessiree Bob, you heard that right, the only change that you will need to make in order to remove the Joomla meta generator tag is through the .htaccess file.

So, how is it done?

Well, there is this little Apache module called mod_substitute (note: Apache modules are not the same as Joomla modules – Apache modules, if you really want to know more about them, work more or less in the same way as Joomla plugins), which (unsurprisingly) allows the substitution of one pattern with another in the served HTML page. So, to replace the Joomla meta generator tag on the website, all we need to do is to add the following lines to the beginning of the .htaccess file:

<IfModule mod_substitute.c>
	AddOutputFilterByType SUBSTITUTE text/html
	Substitute "s|<meta name=\"generator\" content=\"Joomla! - Open Source Content Management\" />||i"
</IfModule>

That’s it!

What if it doesn’t work?

If it doesn’t work, then most likely you don’t have the mod_substitute Apache module installed and/or enabled on your server. This can be quickly verified by using one of the following methods:

• Issue the following command in the Linux shell:

  httpd -M | grep 'substitute'

  If you get an empty result, then you don’t have mod_substitute installed.

• Remove the first line and the last line from the .htaccess code above (only if you are running on a development server), and your website should display a 500 error. Additionally, you will see the following entry in your Apache error log (/usr/local/apache/logs/error_log in a WHM/cPanel instance): Unknown filter provider SUBSTITUTE.

Once you confirm that you don’t have mod_substitute installed, then you can talk to your system administrator about it and he should be able to sort it out for you.

Are there any other uses of mod_substitute?

There are. In fact, it is mostly used to append some JavaScript tracking code to the end of the served page. This is very practical for a system administrator who’s asked to add some tracking code to a Joomla website, but who doesn’t know how to work with it (with the Joomla website).

Are there any disadvantages of using mod_substitute?

Aside from a slight-to-negligible performance hit, there are no technical disadvantages for using mod_substitute. There are, however, some reservations. For example, when you use mod_substitute, you are transferring some of the logic on your website to the .htaccess file, which is not good. You will also make the process of maintaining your website confusing. If you want our opinion, then while we think that mod_substitute is a wow feature (there are only a few people out there who know that you can replace the HTML on a page using .htaccess), we think it should be used only when it’s really needed, and not because of its mystique.

If you think that this whole thing is a bit over your head and you need help with the implementation, then we’re here to help! Just contact us and we’ll implement the above for you swiftly, professionally, and for very little money!

Yesterday noon, a new client contacted us and told us that he was having problem when using HTTPS on his Joomla website (the website in question was using the latest Joomla version, which is 3.6.4). He said many images did not appear, and it seemed as if no CSS was applied to the website whatsoever. So, we loaded his website in our browser (we were using FireFox), and then we checked the console log of the browser, which showed the following errors:

Blocked loading mixed active content “http://www.[our-client-joomla-website].com/templates/[our-client-joomla-template]/css/bootstrap.css”
Blocked loading mixed active content “http://www.[our-client-joomla-website].com/templates/[our-client-joomla-template]/css/default.css”
Blocked loading mixed active content “http://www.[our-client-joomla-website].com/templates/[our-client-joomla-template]/css/template.css”
Blocked loading mixed active content “http://www.[our-client-joomla-website].com/templates/[our-client-joomla-template]/css/touch.gallery.css”
Blocked loading mixed active content “http://www.[our-client-joomla-website].com/templates/[our-client-joomla-template]/css/responsive.css”

Looking at the above errors (there were many other similar errors), we immediately knew what the problem was: the website was trying to load HTTP content when in HTTPS mode. The browser, in order to protect the end user, blocked all non HTTPS content.

What was interesting about this whole thing was that the problem disappeared as soon as we switched off Joomla’s SEF plugin. In other words, the problem was technically caused by Joomla’s SEF plugin (the System – SEF plugin).

Naturally, the next step was to examine the HTML code, and so we looked at the HTML code very thoroughly and very carefully, and we were surprised when we saw that all the CSS/JS/image references were relative, and not absolute, meaning we shouldn’t have a problem with them, but we also noticed another thing, it was this line in the head tag:

<base href="http://www.[our-client-joomla-website].com/" />

Aha! So the base href was set to HTTP, and that’s why all the relative links were also set to HTTP. So, our task was to change the http in the above line to https. In other words, we needed to change the above line to:

<base href="https://www.[our-client-joomla-website].com/" />

So, how did we fix it?

Fixing the problem was very easy. All we needed to do was to open the index.php file in the templates/[our-client-joomla-template] folder and add the following code immediately after <?php defined( ‘_JEXEC’ ) or die( ‘Restricted access’ );?>:

<?php $document = JFactory::getDocument();
$document->base = str_replace('http://', 'https', $document->base);?>

That was it! When we refreshed the page after adding the above code, all the Blocked loading mixed active content errors were gone!

But why did the Joomla SEF plugin cause this problem?

While the enabling the System – SEF plugin did result in the problem, we don’t think it was the root cause. We think that the root cause lied elsewhere (no – it wasn’t the $live_site in the configuration.php file). Unfortunately, we were not commissioned to find out what the root cause was, as the client was happy with the fix that we provided (which was pretty solid anyway).

If you’re having the same problem on your Joomla website, then try our solution above. If it didn’t work for you, or if you’re having problems with the implementation, then please contact us. Our work is quick, our results are professional, and you won’t have to pay an arm and a leg (at least not both) to afford us.

While trying to update the Joomla website of a regular client of ours, we were faced with the following error:

“The most recent request was denied because it contained an invalid security token. Please refresh the page and try again.”

If you are an avid reader of our blog, then you will know that the above error is one of the most dreaded Joomla errors out there. While we have devised a method to get rid of the invalid token problem, we admit that our solution is really about hiding the actual issue, rather than resolving it. Additionally, it was the first time that we have seen this error when trying to update a Joomla website.

Further examination of the update process, we noticed that Akeeba intercepts the update process to make a full backup of the website before the actual update. So we thought, what if the problem had to do with Akeeba?

Knowing that Joomla and Akeeba work closely together, we were confident that if the problem was caused by Akeeba, then it must have been resolved in the latest Akeeba version. So, we updated Akeeba from 4.7.4 to 5.2.4, which is the latest version at the current time, and we tried to update the website again, and this time, it worked. There was no Invalid security token error or any other error for that matter – the update worked super smoothly.

But why did the problem occur with the previous version of Akeeba?

We don’t know – but most likely it was a compatibility issue with Joomla’s updated session management engine. Luckily, the Akeeba people were proactive and fixed the problem immediately.

So, if you’re seeing the “Invalid security token” error on your Joomla website when you’re trying to update it, and if you have Akeeba backup installed, then make sure you update Akeeba backup to the latest version as that might solve your problem. If it doesn’t, then please contact us. We will fix the problem for you quickly, affordably, and professionally.

We think very highly of JCE and we have huge respect for its developers. We recommend it to our clients and we always install it on the Joomla websites that we fully manage. With the exception of very few quirks, our experience (and that of our clients) is very smooth with JCE.

The very few quirks that JCE has, however, are very irritating, even more irritating than having mayonnaise in a burger, and you know we don’t like mayonnaise, don’t you? (yes – this line is inspired by Jimmy the Tulip.) On the bright side, however, all the quirks that we have experienced with JCE were addressed by modifying its settings (such as this one), which technically doesn’t make them quirks.

But, this morning, a very important client of ours emailed us and told us that they were not able to save a specific Joomla article. They said that the browser was hanging for that specific article and, as such, they weren’t able to make any modifications to the article.

We immediately checked the Joomla article, and we noticed that it had many images, and so we thought that the browser was crashing because of this. So, we went to phpMyAdmin, and we removed all references to the images from the article, and then we tried loading the article from the Joomla backend. Still, the same problem: the editor crashed.

We then thought, what if there are settings in JCE that are causing this crash? What if, for example, JCE was set to validate the HTML and the HTML validation is crashing? So, and while in the Joomla backend, we went to Components -> JCE Editor -> Global Configuration, and, to our joy and rejoice, we noticed that HTML validation was turned on! So we set the Validate HTML field under Cleanup & Output to No. We tried the page again but, unfortunately, it kept crashing. We then set the Compress Javascript, the Compress CSS, and the Compress with Gzip fields under Compression Options to “No”, but that didn’t help either.

At this point, we knew that the problem was with the JavaScript code of the JCE editor, and this made us somehow desperate. Why? Because only a few programmers on planet Earth would love to work on the JavaScript code of the JCE editor, and we’re not among those programmers.

But then we thought, why is the problem happening only on this page and not on other pages? Maybe there is something “special” on this page that is causing the JavaScript to crash. So we started the process of elimination, in other words, we removed, bit by bit, HTML code from the article in phpMyAdmin until the problem was no more, and we finally were able to find the root cause of the problem, it was this line:

<a href='http://www.[ourclientjoomlawebsite].com/page.html'>Click here to continue &nbsp; &gt;&gt; </a>

But what is “special” in the above line? Well, it was this &nbsp;&gt;&gt; bit, for some reason, JCE editor did not love this combination in the anchor tag. Removing the above did fix the problem.

We will report this issue to the JCE development team on Twitter. Meanwhile, if the JCE editor on the backend of your Joomla website is crashing for a specific article, then make sure you 1) update your JCE editor to the latest version, and 2) try our process of elimination above. If none of the above works, then please contact us. We are always happy and ready to help, we love having new clients, and we our fees are super affordable.

Note: This is an advanced post which mentions an unstable extension at the time of writing (November 2016). We suggest you avoid overriding the Joomla core if you’re not comfortable with your programming skills. If you need help with overriding the Joomla core, then you can always contact us!

Until very recently, we used to overwrite the core mainly to resolve performance issues. But, last week, we have discovered a not-so-new extension, an extension called Joomla Override, which allows developers to override the Joomla core. So, we decided to give this extension a chance… But, before delving into the details of our experience, let us explain the difference between overwrite and override.

In case you don’t know already, the basic difference between overwriting and overriding is that in the former, you open a Joomla core file, you make the changes, and then you upload it back. In the latter, you copy the Joomla core file (that you want to override) to a different location and you make the changes there. Obviously, overriding has a huge advantage over overwriting, and it’s that the changes don’t get wiped out with a Joomla update. Another major advantage is that all the overrides are in one location – instead of having them spread over the many folders of the Joomla website.

So, how did our experience go with using the Joomla Override plugin?

In all fairness, the experience didn’t go very well, because the moment we tried to override the Joomla feed of the homepage, we saw the following error on the homepage of the Joomla website:

Fatal error: Class ‘JFeedItem’ not found in templates/[our-client-joomla-template]/code/com_content/views/featured/view.feed.php on line 66

As mentioned above, the error was on the homepage of the Joomla website, and not on the feed page, which we thought was odd. Why would overriding the feed file affect the homepage? So we thought, it might be that we also need to override the view.html.php file (and not just the view.feed.php file), and so we copied the view.html.php file from the components/com_content/views/featured folder to the templates/[our-client-joomla-template]/code/com_content/views/featured folder.

This time, when we refreshed the page, we saw the following (different) error:

Fatal error: Cannot redeclare class ContentViewFeatured in templates/[our-client-joomla-template]/code/com_content/views/featured/view.html.php on line 0

After a long and tedious research of the above problem, we discovered that it was caused by 2 issues:

• Joomla has the exact same class name in both the view.html.php and the view.feed.php files (for both the category and the featured views in the com_content component). We consider this to be a Joomla bug as Joomla shouldn’t have the same class name in 2 different files.

• The Joomla Override plugin doesn’t care about the current format, and loads both the view.feed.php and the view.html.php despite the fact that they are almost never needed at the same time.

So, how did we fix the problem?

We fixed the problem the following way:

• We opened the file component.php located under plugins/system/joomlaoverride/helper

• Just before the following line:

  if ($params->get('extendDefault',0))

  We added the following code:

  $arrRealFileName = explode('/', $filePath);
  $realFileName = $arrRealFileName[count($arrRealFileName) -1];
  $format = JFactory::getApplication()->input->get('format','html');
  if (stripos ($realFileName , $format) === FALSE)
  	continue;

• We saved the file and then uploaded it back. We then refreshed the homepage, and the problem was fixed!

Did things go smoothly afterwards?

Well, not really. We discovered that there were some important core files that we were not able to override. For example, we wanted to override the file cms.php which is located under the libraries/cms/application/ folder, mainly so that we can disable session activity for visitors , but we weren’t able to. Even if we were able to, the override was useless, simply because session activities take place before any plugin is loaded. So, we had to overwrite the cms.php file instead of overriding it.

In total, we had to overwrite about 4 files for a large Joomla website, which wasn’t that bad, but wasn’t that good either. In addition, we were faced by several problems when we used this extension (including the one in this post), all of them were hard to fix. Because of that, we can’t recommend using the Joomla Override plugin, in fact, we recommend against using it, until it is stable enough.

If you want to override/overwrite some core files on your Joomla website, then please contact us. We will amaze you with our professionalism, cleanliness, and our super affordable fees!

Sometimes, large companies have some text that they want to display somewhere on the backend of their website for their staff. The text might consist of editing instructions, a legal disclaimer, or a general announcement. Now, on the frontend, it is very easy to display such a text by using a Custom HTML module (nowadays, it is called Custom module, without the HTML), but how about the backend?

Well, luckily, the backend also has a Custom module that can be used the same exact way as a site Custom module. Here’s how you can use it…

• Login to the backend of your Joomla website.

• Click on Extensions -> Modules.

• On the top left (just below the green New button), choose Administrator from the dropdown that displays Site.

• Click on the green New button on the top left.

• Choose Custom from the Select a Module Type page.

That’s it! Now the only thing that you need to worry about is the position of the module – which should be very easy to figure out once you click on the Position dropdown on the right. Note that you must know which administrator template you are using in order to choose the correct module position. If you don’t know which template you’re using, then just click on Extensions -> Templates, and then, in the dropdown that has Site in it, choose Administrator. The administrator template that you’re using has a yellow (or is it amber?) star next to it.

Is the administrator Custom module the exact same as the site Custom module?

While both modules are very similar, they are not the same: the administrator Custom module and the site Custom module use different files (they do not share the same files), and the files have some slightly different code.

Can the module’s layout be overridden?

Well of course. All you need to do is to create to copy the default.php file from under the administrator/modules/mod_custom/tmpl folder to the administrator/templates/[your-joomla-administrator-template]/html/mod_custom folder (you will need to create the mod_custom folder if it doesn’t exist).

We hope that you found our little guide above helpful in creating Custom HTML modules in the backend of your Joomla website. If you need help with the implementation, then please contact us. We’ll do the work for you swiftly, professionally, and for very little money!

A client of ours emailed us yesterday and told us that clients weren’t able to checkout on his VirtueMart store. He told us that they were seeing the following error:

Unknown SSL protocol error in connection to www.eprocessingnetwork.Com:443

We did a quick research on the issue and it looked like it was caused by an update to the SSL certificate on the destination (e.g. on the eProcessingNetwork website), and so we tried adding the following code in the authorizenet.php file (just immediately after curl_setopt($curl_request, CURLOPT_RETURNTRANSFER, 1);), which is located under the plugins/vmpayment/authorizenet folder of the Joomla website:

curl_setopt($curl_request, CURLOPT_SSLVERSION, 3);

We tried the transaction again and we had the same problem. And so we flipped the 3 in the above line to 2:

curl_setopt($curl_request, CURLOPT_SSLVERSION, 2);

Still, we had the same error.

And then we thought, what if the OpenSSL library on the source server (e.g. on the server hosting the Joomla website) was outdated. So, we logged in to the WHM account, and we saw a big yellowing warning on the top stating the following:

Red Hat® will deprecate all CentOS 5 systems on March 31, 2017. cPanel, Inc. and Red Hat will no longer provide your operating systems with updates or security fixes. You must migrate your server to a CentOS 7 server with the Service Migration Tool to ensure that your server remains up to date. You must contact your hosting provider for a destination server.

Aha! So the server was using a very old version of CentOS (which is CentOS 5), hence the outdated OpenSSL library. That meant that the only solution to the problem was to update the OS on the server, and so we initiated the update process with the hosting company, and they took care of the rest.

We started testing the website immediately when the update to CentOS 7 was done, and, unsurprisingly, payment processing worked again! The problem was fixed!

We hope that you found our little post helpful and that it helped you fix the checkout problem on your Joomla website. If it didn’t, then please contact us. We’ll fix the problem for you quickly, professionally, and for very little money!

This morning, we thought we had a little time to do something fun, and so we created a command to generate a list of all user agent strings (or signatures) on a very high traffic Joomla website that we maintain. “Why is that?”, we hear you ask… Well, because 1) we were curious about which user agent signature is the most common on the Internet, and 2) we had some security concerns because of the 500 HTTP Errors that we found a few days ago on that particular website and which were caused by a fake user agent signature.

So, what is the most common user agent string as of October 2016?

Well, if you really are interested, then the most common user agent string (as of October 2016) is (drum rolls, please):

Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko

In case you’re wondering which browser the above user agent string belongs to, then it’s FireFox (we love FireFox, although we have to say it’s becoming more memory hungry with each iteration).

And what are the top 500 user agent strings?

A list containing the top 500 user agents strings can be downloaded here. The original raw list contained 53530 unique user agent strings (that’s unique, in case you didn’t notice the underline), and we generated it by issuing the following command in the Linux shell (note that the below will just return the top 500 user agent strings, if you care about the whole list, then you should delete this part | sed -n 1,500p from the below string):

awk -F\" '{print $6}' [ourclientjoomlawebsite].com-Oct-2016 | sort | uniq -c | sort -k1nr | awk '!($1="")' | sed -n 1,500p > user-agents.txt

We know that you’re curious about the above set of commands, so let us dissect them for you:

• The first command…

  awk -F\" '{print $6}' [ourclientjoomlawebsite].com-Oct-2016

  …grabs the 6^th column from the deflated Apache log file. The 6^th column is typically the user agent string.

• The second command…

  | sort

  …uses the list returned by the first command as input (which is a raw list of all the user agent strings), and sorts it by user agent (this is necessary for the subsequent command).

• The third command…

  | uniq -c

  …creates a unique list of all the user agent strings (out of the result of the second command), where each line consists of a user agent string, preceded with the number of times that that user agent string occurred in the logs.

• The fourth command…

  | sort -k1nr

  …sorts the list returned by the previous command by number of occurrences of user agent strings descending.

• The fifth command…

  | awk '!($1="")'

  …removes the first column (which contains the number of occurrences of each user agent string).

• The sixth command…

  | sed -n 1,500p

  …returns the first 500 lines (e.g. the top 500 user agent strings, ordered by number of occurrences, descending).

• The seventh command…

  > user-agents.txt

  …redirects the output to the user-agents.txt file.

As you can see, it’s not that hard once it is explained! Nevertheless, one has to admit it’s a bit hard, however, it clearly highlights the power of shell commands in Linux (if you’re a PHP developer, then imagine how much code would be needed to do all the above in PHP).

If you decide to run the above set of commands, then please keep in mind that it will take about 10-15 minutes to process a 10 GB log file even when using a relatively powerful server with an SSD drive.

A couple of notes about the generated file…

The Apache log that we have used to generate the list is that of a very professional website, which main visitors work for professional companies and governmental agencies. Additionally, the website in question has the absolute majority of the known spam bots blocked (which explains the absence of many mainstream spam bots in the list).

We hope that you found this post fun and useful. If you need help generating an even more interesting set of information from the Apache logs of your Joomla website, then please contact us. We are particularly fond of Linux (and awk), we love our job, we love our clients, and we won’t charge you much!

Let’s say you want, for one reason or another (hopefully a good reason), to know the version of a Joomla website that you don’t own. You don’t have FTP/sFTP access to the site’s filesystem, you don’t have access to the backend, and you don’t know the owner. So, what do you do?

Well, for the absolute majority of Joomla websites, there is at least one file that you can check that will tell you the exact version the Joomla website is running.

For Joomla websites >= 1.6.0

Joomla websites, ever since version 1.6.0, have a very easy method that reveals their exact version (which may or may not be a good thing), all you need to do is to access the following URL:

http://www.[thejoomlawebsite].com/administrator/manifests/files/joomla.xml

The above URL will display an XML file containing the site’s version in the version XML element.

Yes – it’s scarily easy, huh! It also applies to all versions of Joomla from 1.6.0 until 3.6.3 (including, of course, all the versions in the 2.5.x line), which is excellent!

For Joomla websites >= 1.5.0 and <= 1.5.26

Joomla websites in the 1.5.x line do not have the joomla.xml file, but there is another link that one can check, which is this one:

http://www.[thejoomlawebsite].com/language/en-GB/en-GB.xml

You can clearly see, in the XML version element, the version of the Joomla website in question. However, the problem here is that this method only reveals the major version, and not the minor version. The minor version is typically guessed by checking some core JavaScript files under the media/system/js folder.

For Joomla websites < 1.5.0

Before you say anything, yes, there are still some Joomla websites on the Internet using the 1.0.x versions (in fact, we worked on one back on Sunday). In order to know if a Joomla website is version 1.0.x, then all one needs to do is to check for the existence of a custom.xml file by visiting the following URL:

http://[thejoomlawebsite].com/modules/custom.xml

Similarly to the Joomla 1.5.x line, the Joomla 1.0.x line does not have a straightforward method to accurately determine the minor version of the Joomla website (for example, if it’s using version 1.0.0 or version 1.0.15). However, JavaScript files in the includes/js folder typically reveal information (such as file revisions) that will help determine the minor version of the Joomla website.

In short…

• If the link http://[thejoomlawebsite].com/modules/custom.xml works: This is a Joomla 1.0.x website.

• If the link http://www.[thejoomlawebsite].com/administrator/manifests/files/joomla.xml works: This is a Joomla website with version >= 1.6.0, and the exact version number is stored in the version XML element of the file.

• If none of the above links works, then the Joomla website is a 1.5.x website.

Again, the above assumes that you don’t have filesystem access to the website. If you do have access to the filesystem, then you can check the file version.php which tells you the exact version of the Joomla website. Unfortunately, the version.php file is not present in a standard location. Luckily, however, you have itoctopus on your side to let you know of its exact location for each Joomla version:

• For Joomla 1.0.x websites: The file version.php is located under the includes folder.

• For Joomla 1.5.x and Joomla 1.6.x: The file version.php is located under the libraries/joomla folder.

• For the short lived Joomla 1.7.x: The file version.php is (oddly) located under the includes folder (which is similar to 1.0.x websites).

• For Joomla 2.5.x and Joomla 3.x: The file version.php is located under the libraries/cms/version folder.

We hope that you found this post useful. If you didn’t, or if you have just discovered (after following the above guide) that you are running a very old version of Joomla and you have decided that you want to update it, then please contact us. We are always here for you, we do the job right, and our fees are super duper affordable!

One of the biggest mistakes that system administrators make when analyzing the Apache server logs, is that they ignore many of the “500” HTTP errors (also known as Internal Server Errors). They think that these fatal errors – if they are not happening on legitimate pages, then they are not worth investigating. Little do they know that there is usually much more beneath the surface. Let us give you an example…

This morning, as part of a routine security check on a Joomla website, we ran the following grep command to get a list of all the pages that generate 500 errors:

grep ' 500 ' ourclientjoomlawebsite.com-Oct-2016 > 500-errors.txt

(Note: The file ourclientjoomlawebsite.com-Oct-2016 is the Apache access log file for the website ourclientjoomlawebsite.com).

When we examined the 500-errors.txt file, we saw the following fatal error, many times:

[ip] - - [16/Oct/2016:14:14:09 -0400] "HEAD /component/mailto/?tmpl=component&template=template-name&link=http://www.ourclientjoomlawebsite.com/category-1/page-1.html HTTP/1.1" 500 - "-" "Mozilla/5.0 (compatible; um-LN/1.0; mailto: randomemail@[anotherdomain.com])"

Now, if you just copy and paste the above link to a browser (of course, you will have to append the domain name in front of it), then you will notice that the link works (it will show the “Send this Article to a Friend” form), but there’s more to it…

The user agent, which is the root of a substantial amount of evil in the online world, has an email in it. Obviously, such a user agent is fake, and has been specifically tampered with to crash the Joomla website, and it did do its job!

The thing is, not a single request on a Joomla website (or any other website, for that matter) should crash it. A solid website should handle any request gracefully, and should never crash, because a crash is a huge positive sign for a blind SQL injection (which may lead the way for an actual exploit).

Naturally, we fixed the problem by 1) addressing the issue in the mailto extension, and 2) by cleaning up the $_SERVER[‘HTTP_USER_AGENT’]. But, we thought, what would have happened if we didn’t examine the server logs this morning? Would it have been possible for the attacker to exploit the website? The latter thought sent chills through our spines, because the website in question was for a very important client of ours.

So, as a system administrator on a Joomla website, it is up to you to examine the server logs for “500” errors, and it is up to you to immediately report any of these errors to your developer(s). If you don’t have any developers to handle these errors, or if you need help performing the log analysis on your website, then please contact us. We are always happy to serve and our fees are super affordable!

Occasionally, Joomla administrators face the infamous allowed memory size error which forces them to increase the memory_limit value in the global php.ini or in a local .user.ini file by adding the following line:

memory_limit=256M

The above code will increase the memory limit to 256 megabytes, which is more than ample for any Joomla website out there. Now, the thing is, the absolute majority of the times, Joomla administrators increase the memory limit to allow an occasional usage of a certain Joomla functionality, such as updating the Joomla website or installing an extension. Once the update is done or the extension is installed, then there is no need for the high PHP memory limit anymore.

Unfortunately, most Joomla administrators do not revert back the changes they did to the PHP memory limit – mainly because they think it’s better this way (it did fix the problem – didn’t it?). But it’s not – in fact, a high memory limit makes a DoS (Denial of Service) attack easier and much more dangerous. Let us explain…

The PHP memory limit defines how much memory each PHP page is allowed to consume – so if, for instance, you have a memory limit of 256 megabytes, then, assuming a worst case scenario, merely 8 simultaneous page views can take up 2 GB of your server’s RAM. So, if an attacker knows of a page on your website that consumes a lot of memory, then the attacker can simply release some bots on that page and quickly bring your server to its knees by exhausting all of its (the server’s) memory.

But what is an ideal PHP memory limit?

Ideally, a page on a Joomla website should not consume more than 32 MB of memory (and that’s very generous). If you find that on some pages (with the exception of the core update page) your Joomla website needs more memory than that, then you might be a victim of a bad Joomla programmer. In that case, you will need some Joomla experts, such as (ahem) us, to address this issue for you. Just contact us and and we’ll fix the problem for you in no time and for a very reasonable price!

We just got a call from a new client, telling us that she was perplexed about the fact that the number of banner impressions on her Joomla website is unrealistic. She said that her Joomla website is getting about 10K visitors/month according to Google Analytics, but the number of impressions counted on her Joomla website are almost double that number.

Our immediate answer was: “yes – this is normal”, to which she replied: “How?”, and we replied: “Google Analytics only counts the net traffic to the website, so, it excludes all bots and spam traffic. Joomla doesn’t have the mechanism in place to discount that traffic, and so the traffic figures (from Joomla’s perspective) are highly inflated – hence the huge discrepancy.”

“Bummer!”, she said, “what are we going to do? We need to sell advertising to potential clients and we don’t have the correct number of impressions. Is there anything we can do now?”, she asked. We told her that for now she can communicate the numbers to the advertisers along with a disclaimer that these numbers may be highly inflated, but for the future, that she must use Google DoubleClick to manage her ads. The nice thing about Google DoubleClick is that it displays the exact number of impressions, and the exact number of clicks, and it automatically excludes all non-desirable traffic from the actual traffic numbers. The other nice thing about it is that it’s free (unless the website serves more than 90 million impressions/month – in which case publishers must pay money for Google).

So, does the above mean that the Joomla Banners extension is useless?

Yes – it does – we can’t think of a single reason which makes the Joomla Banners extension a practical ad management tool in today’s world wide web (in fact, two years ago, we have stated that this extension must be removed from the Joomla core). Maybe it was useful back in the early 2000s (back in the Mambo era), but, in 2016, there is not one reason for such an outdated extension to exist. Unless Joomla plans to integrate a webspam engine in its core, then the Joomla banners extension should no longer exist in the core.

If you are currently using the Joomla Banners extension and you just discovered that its statistics are way off and you want to implement Google DoubleClick, then we’re here to help. All you need to do is to contact us and we’ll take care of the whole thing swiftly, cleanly, and affordably!

The last time we had a case of SQL injection on a Joomla website was a long time ago – and the affected Joomla website was a Joomla 1.5.10 website (which is highly exploitable – even by Joomla 1.5.26 standards). Since then (we are now in October of 2016), we have not seen a single case of SQL injection on a Joomla website.

Does that mean that Joomla is more secure now?

Well, Joomla is much more secure now than anytime in the past, but that is not the only cause leading to the significant drop of SQL injection attacks on Joomla websites. In fact, there are other, more important, causes for this drop:

• Security tools on servers becoming mainstream: A few years ago, ModSecurity was installed on a very small percentage of servers. Nowadays, almost all servers come with ModSecurity installed and enabled by default (hence the increase of quirks on Joomla websites caused by ModSecurity). ModSecurity is excellent in blocking patterns that may lead to a SQL Injection attack.

• A more rigid process for accepting Joomla extensions: Joomla extensions were notorious for being insecure. But, as of a couple of years ago (when the JED was revamped), the once lax process of accepting and testing Joomla extensions evolved into a more serious and rigid process, quickly rejecting extensions that do not meet the Joomla security standards. Additionally, the process of immediately suspending vulnerable extensions (and publishing them on the the VEL [the Joomla Vulnerable Extensions List]) has forced developers to ensure that all user input is filtered properly in their extensions.

• A lack of attackers’ interest in SQL injection: Attackers follow the trends, and SQL injection is no longer the trend. In fact, the current trend is to initially upload a few files and then overwrite some core files, ultimately controlling the website and use it to attack other websites and/or use it to download malware to innocent visitors’ computers. Attackers are simply no longer interested in altering the database in any way (for unknown reasons), even though they could (if the attacker is able to upload one file – just one file – to a website, then he can potentially read the main configuration file and then gain easy access to the database).

We think that the last point is the most important one of the lot – it is the mysterious absence of interest in modifying the database (by the attackers) that has contributed the most in the huge drop of SQL injection attacks. One would think that the attackers have a syndicate that decides which types of attacks are allowed and are not allowed – and it seems that some time ago, the syndicate has decided that SQL injection attacks are no longer permitted.

Whatever the real reason behind this is, we hope that SQL injection attacks remain very low on Joomla websites, because database hacks are the worst type of hacks on any website.

So to answer the question of this post – is SQL Injection in Joomla still a concern? – then the answer is yes – it is – simply because vulnerable websites can still be easily SQL injected – but it’s just that the attackers have elected to stop doing it, and it may be only a matter of time before they revert back to their old habit.

Now if you, our dear reader, are one of those unfortunate Joomla administrators whose website experienced the worst type of hacks, then fear not, all you need to do is to contact us. We’ll cleanup your website and we’ll secure it for you quickly, efficiently, and affordably!

Some time ago, while performing daily maintenance work for one of our large clients, we noticed that some of the external links that they had had some weird hash in them. What was interesting is that all these external links that had this problem were domains owned by the client. A little digging into the HTML code unveiled that the problem was caused by HubSpot.

We communicated this issue to HubSpot, and they confirmed to us that this is because the client elected to track the outgoing traffic from their main website to sister websites, and that was the only way for HubSpot to do the cross-domain tracking. At that time (that was months ago), we told HubSpot that we were concerned that these weird hashes may cause content duplication issues, as they may get indexed by Google. They replied that this shouldn’t have a problem whatsoever, since all the pages on the website had canonical URLs, which means that these weird hashes will not result in many variations of the same pages, which in turn means that Google will not index those variations.

Fast forward to a couple of weeks ago (yes – we prepared this post a couple of weeks ago – but we didn’t have the time to publish it until today), where it was a completely different story. The moment we logged in to the Google Search Console of that particular website, we discovered that thousands upon thousands of pages such as http://www.[ourclientjoomlawebsite].com/page-1.html?__hstc=[long-hash]&__hssc=[medium-hash]&__hsfp=[small-hash] (where long hash, medium hash, and small hash were long, medium, and short random strings) were 404 pages. Huh?

After analyzing the issue while in super panic mode, we discovered the sequence of issues that were causing this mess:

• The Google bot visits a page on the website.

• The visited page contains a link to a non-existent page on a sister website.

• The link has the HubSpot tracking code appended to it.

• Google tries to visit the link and thus results in a 404 error for that link.

• Since the link is a 404 link, then the whole canonical concept does not apply to it, which means that Google ends up visiting many variations of that same 404 link, and thus resulting in many 404 errors. This is a huge issue because it highly inflates the 404 count of the website (which has negative SEO impact.

Of course, the root of the problem was the 404 link, but the problem was amplified by HubSpot‘s tracking code. We brought up this issue with the client, and we presented them with a couple of options:

1. We fix the 404 links – which means that we will risk having the same series of events in the case of new 404 links.

2. We disable tracking on auxiliary (sister) websites.

The client went with the second option, mainly because they were completely disturbed by all these weird hashes they were seeing on their website. As usual, we happily obliged! Here’s how:

• We logged in to the HubSpot account of the client.

• We clicked on Reports -> Reports Settings.

• Under Domains, we unchecked the Enable checkbox under Automatic cross-domain linking.

• We saved the form and then we checked the website, and hooray, all these annoying hashes were gone!

We think that HubSpot‘s strategy for tracking outgoing traffic is somehow flawed: the whole concept of adding weird hashes does not fly well on serious websites. Additionally, as we have demonstrated above, this strategy can cause SEO issues.

If you see weird hashes in some of your URLs after adding HubSpot to your website, then you should uncheck the Automatic cross-domain linking feature in HubSpot. If you need help in doing that, then just contact us and we will gladly do it for you. Our fees are affordable, our Joomla expertise is second to none, and we will strive to be your friends (but not in a cable guy kind of way).

Login loops are probably the most complex problems to fix on a Joomla website. Here’s the scenario: you go to the login page of the backend of your Joomla website, you enter your username and password, you click on the blue Log In button, but, to your surprise, you are redirected back to the login page with no error.

The thing is, we have encountered this problem many times before, and still, each time a client tells us that he has this issue on their website, we swallow about 8-10 aspirins, and we brace ourselves for a very, very long night… Unfortunately, yesterday (a Sunday), we were unlucky enough to work on this issue…

Yesterday started as very beautiful day, very sunny and with a very blue sky (which is not very common in Montreal), but it got cloudy pretty fast when a new client contacted us with the most terrible news: he wasn’t able to login to the backend of his Joomla website, and every time he tries to login he is redirected back to the homepage.

The reason why the news were that terrible is because we have yet to see two of these login loop problems caused by the same root issue. Each of these login problems is unique, as well as its solution. Still, were started the work with a healthy amount of optimism. But, as we later learned, optimism is a fickle friend, who jumps a sinking ship at the nearest opportunity!

We started the work by trying to find whether the cause of the problem is one that we have encountered before:

• Is the tmp physical path (the $tmp_path variable in the configuration.php file) correct and writable by the web server?

• Is the log physical path (the $log_path variable in the configuration.php file) correct and writable by the web server?

• Is the cookie domain (the $cookie_domain variable in the configuration.php file) empty?

• Is the plg_user_joomla plugin enabled? Is the plg_authentication_joomla plugin enabled? (Note that we used phpMyAdmin to check for the value of these two plugins in the #__extensions table).

• Is the session handler (the $session_handler variable in the configuration.php file) set to database?

• Are all types of cache disabled?

As you may have probably expected, all the above were OK – if they were not OK, then what is the point of this post? So we decided to take a more aggressive approach to solve the problem…

We started our aggressive approach by overwriting the Joomla website with a fresh copy of the same version of Joomla using our super quick famous technique for cleaning up Joomla websites, but that didn’t work!

So the next step in our aggressive approach was to check the controller.php file (located under the administrator/components/com_login folder) which is the entry file responsible for logging in the user (there are other, deeper files that effectively handle the login, but this is the entry file). A thorough debugging of the file revealed that the problem lied in this line:

JSession::checkToken('request') or jexit(JText::_('JINVALID_TOKEN'));

If you’re a close follower of our blog, then you may remember that we have published several articles on Joomla’s invalid token error. In fact, in one of our blogs, we concisely explained how to get rid of the invalid token error once and for good. But, we didn’t want to do apply this fix for that client because 1) he was using the latest version of Joomla, and 2) there shouldn’t be a reason for him to see that error on his website in the first place, and 3) this is a last resort solution.

So, obviously, the issue is caused by a problem with the session storage, but we have already checked everything – literally everything having to do with the session, so why do we have a problem with session storage? At this point, we were only able to think of one culprit: a Joomla update that left some reminiscent files from a previous version… So we compared the files of the Joomla website with those of a fresh Joomla website (having the same version), and we noticed that there were some extra files/folders in our client’s websites (reminiscent files from a previous Joomla version). We deleted the extra files and folders and we tested this problem again, but still, it didn’t work.

In order to break the series of unfortunate events (yes, we are alluding to the movie here), we called our moms so that they can give us much needed morale, and they did. We then relaxed for a bit and then started attacking the problem again! We thought, hey, there is this one file that we haven’t checked yet and that we credit with a not-so-small percentage of all the evil in the world (let alone Joomla): it is the .htaccess file. So, we opened the .htaccess file, and, in its very beginning, we saw the following 2 lines:

Header set Set-Cookie "VC-NoCache=1; max-age=900; path=/"
Header set VC-NoCache "1"

Huh? What is that, we thought? Is that something that has to do with caching? Let’s just comment out these 2 lines and see if that takes care of the problem, and it did! We were able to login without being affected by the vicious and ruthless login loop. We were happy but we were tired – it was a frustrating and a scary experience!

If you, our dear reader, are on the brink of deleting your whole Joomla website (and business) because of this error, then please check our previous posts about this issue (which are linked to in the beginning of this article), and also make sure that you don’t have something in your .htaccess file causing this problem. If you are not able to solve this problem, then just contact us and we’ll take care of it. But, before doing that, please make sure you check our fees and please don’t forget to send us a blindfold and a cigarette – we may need them!

A Joomla designer called us this morning and told us that he was seeing the following error when trying to load the Joomla website of his client:

500 – Unable to load renderer class

He told us that this whole thing happened after switching to a new template, and that reverting back to the old template did fix the problem for him (but he really needed the new template to work). This little piece of information was super useful for us, as it narrowed down our search area from all of the Joomla website, to a specific template.

So, we opened the main index.php of the problematic template, and it didn’t take us long to figure out where the problem lied. It was this line:

<jdoc:include type="logo" name="modules" style="none" />

The thing is, logo is not a jdoc type – it is a jdoc name. So, we changed the above line to this one…

<jdoc:include type="modules" name="logo" style="none" />

… and that fixed the problem!

But what are the allowed jdoc types?

In case you’re wondering, the allowed jdoc types are the following:

• component
• head
• message
• module
• modules

If one of your jdoc types is not not in the above list, then you are likely to see the “500 – Unable to load renderer class” error on your Joomla website.

We hope that you found this very shot post helpful (we’re not fond of short posts at itoctopus) and we hope that it solved your problem. If it didn’t, then please contact us. We are always glad to help, our work is super professional, and we don’t charge much.

Last Monday, we received an email from a company (which is a client of ours) stating that any email sent from their Joomla website to an email to their domain isn’t being received. For example, assuming that their Joomla website is ourclientjoomlawebsite.com, and the Joomla website tries to send an email to admin@ourclientjoomlawebsite.com (the email, in the case of our client, was a notification email from an RSForm form), then that email doesn’t get delivered.

When they first told us about the problem, we insisted that the problem was caused by missing SPF settings, and so we told them to add the IP of their server to the SPF record in their DNS (they were using Office 365 for their mail), but they didn’t do that. The next day (Tuesday), the IT Manager working for the client called us and told us that the email wasn’t even caught in a spam filter by the mail server – so, most likely – the email wasn’t even sent by the server which was hosting the Joomla website, so it’s definitely not an SPF issue. He then mentioned that they receive notifications from their Joomla website to Gmail and Yahoo accounts, but they don’t receive them when they are sent to company emails. We thought, that was strange, but we insisted that they add the IP of the server to their SPF records, just to see if that solves the problem. Finally, they did – but, to our regret (and shame, and humiliation, and embarrassment, and disgrace, and…), that didn’t fix the problem.

The IT Manager then emailed us the negative results, urging us to see if the issue is caused by RSForm, the Joomla website, or the server. So we did (today), and the first thing that we examined was the exim logs the following way:

• We ssh’d to the server.

• We changed to the /var/log/ directory by issuing the following command:

  cd /var/log

• We searched the log for any entries pertinent to the affected email address:

  grep -R "admin@[ourclientjoomlawebsite].com" *

• We saw the following results:

  exim_mainlog:2016-09-23 15:11:20 1bn9Oy-000DDQ-73 <= no-reply@[ourclientjoomlawebsite].com U=**** P=**** S=1662 id=cfc73da22404d08163d64fbbab9316b5@www.[ourclientjoomlawebsite].com T="New submission from 'Contact Us'!" for admin@[ourclientjoomlawebsite].com exim_mainlog:2016-09-23 15:11:20 1bn9Oy-000DDQ-73 ** admin@[ourclientjoomlawebsite].com R=virtual_aliases: No Such User Here

Hmmm! Looking at the above was enough for us to explain what was going on. The server hosting the Joomla website had its own mail server, and that server thought that it was responsible for all email accounts belonging to ourclientjoomlawebsite.com (which was wrong, since Office 365 was responsible of these email accounts), and so the exim mail application was checking the internal (server) database for the existence of these accounts, and then silently failing when it’s not able to find them.

So, how did we fix the problem?

Fixing the problem was quite easy:

• We logged in to the cPanel account of the domain.

• We clicked on MX Entry under Email.

• We selected our client’s domain and we chose Remote Mail Exchanger in the Email Routing section.

• We clicked on the Change button.

• That’s it!

After doing the above, we tried sending a notification email from the Joomla website (we did that by filling out an RSForm form), and, guess what? It worked! In fact, we knew that it worked because the client sent us an email a few seconds after we submitted the form telling us that the problem was solved! We were happy and the client was happy, which is always ideal in our line of job!

How come the fix worked immediately?

Technically, the fix didn’t consist of any DNS changes which usually take time to resolve. The fix consisted of a small setting change on the server, telling it that it’s not responsible for routing emails (and that the routing of emails is handled by an external server). These kind of setting changes take effect immediately, and they don’t even require restarting the mail server.

What about our ego?

We admit it – we started this task with an inflated ego. We were confident that the problem was caused by SPF settings and we didn’t listen carefully to what the client was saying. That was a mistake but the consequences of that mistake were not. In fact, we were humbled when we found out that we were wrong (after insisting that we were right), and we were gently reminded that an inflated ego attitude is a bad thing in our business, and that we should always, always, listen very carefully to the client.

We hope that you found this post informative and that it helped you solve your problem. If it didn’t, then please make sure that your SPF settings are correct. If they are, or if you feel that this whole thing is a bit over your head, then fear not, we’re here to help! Just contact us and we’ll take care of this issue for you in no time and for a very affordable fee!

Note: This post is a rant! Proceed at your own risk!

When Joomla 3.6 was released, most of those who updated their websites started complaining: I added a new article, but I can’t find it anywhere on my website. What is the problem?

The problem is that someone (you know yourself if you’re reading this!), who suddenly decided to become a Joomla core developer (that someone has never developed anything on Joomla before), enhanced the adding of new Joomla articles by giving newly added articles the last possible ordering. In other words, let’s say that you have 100 articles, all in the same category. When you add a new article to your website, the article ordering of the new article is 101.

Now, that someone (we will not refer to him as a developer and we will explain later why), thought that he made a breakthrough in the science of Joomla: by using very little code he made the process of adding new articles much faster on large Joomla websites. How is that? Well, in Joomla versions 3.5.1 and prior, when a new article was added, it was given the ordering “0”, and all the other articles’ orderings were shifted by 1. The shifting part is an expensive process in MySQL, especially if the website has many articles. By avoiding the shifting process, the adding of new articles became immensely faster.

But, what is the problem?

Well, the problem is that this enhancement breaks the default ordering functionality in Joomla. So, if an administrator adds an article to his Joomla website and is using the Article Ordering sorting feature for ordering his Joomla articles, he won’t be able to see that article neither in the frontend nor in the backend unless he browses to the last page.

This has caused a huge issue in the Joomla community, with many people complaining that they can’t find their new articles in Joomla 3.6.+ (or that they can’t add new articles in Joomla 3.6.+), while in reality, their articles were there, but they’re just ordered last. Unfortunately, that someone who broke the ordering functionality refused to admit that it was a mistake – claiming that (and we’re quoting his exact words) “it is wrong to use the field ordering to get content sorted by a date”. Which brings us to the next question:

Why do we refuse to call that “someone” a programmer?

Well, he is not a programmer for the following reasons:

• He failed to understand that the code to shift all the articles when a new article was inserted was there for a reason. He happily erased many lines of codes and just added a few lines without even analyzing the original code. Any programmer should thoroughly examine the code he’s replacing before doing anything.

• He didn’t test his code on a real large website. How did we know that? Because if he tested his code on a real large website he must have noticed the issue. But he didn’t (notice this issue), and so we are assuming good faith here that he didn’t test his super enhancement. The irony in this whole thing is that his code was meant for large websites. Small websites don’t suffer from any performance issues because of the shifting process.

• Even after seeing the reports that his new enhancement broke many Joomla websites out there, he stuck to his code change claiming that all these Joomla administrators were using the wrong ordering, which is, in fact, a wrong statement for 2 reasons: 1) there is no such thing as wrong ordering in Joomla (those administrators were just using an existing sorting option in the backend), and 2) even if that’s the case, he shouldn’t develop something that breaks websites. In fact, he only yielded when some heavyweight contributors in the Joomla core were involved in this issue, while still claiming that reverting back was the wrong thing to do.

• He wasn’t quick in cleaning up his own mess. A programmer, a real programmer will immediately revert his changes if they negatively affect the product. The Joomla community has to wait until Joomla 3.6.3 in order to see this issue fixed (though, in all fairness, we cannot blame this delay totally on him).

The reason why we have strong feelings about this is that the actions of this “someone” caused many people using Joomla to lose some of their trust in the product. His actions were immature at best, and destructive at worst. There are people who are born to be developers, and there are people who are born to be core developers, and this person is in neither group.

In our opinion, he shouldn’t have been allowed to touch the Joomla core, and this brings us to a few existential Joomla questions: Who authorizes work on the core? Who authorizes who should work on the core? And who approves core changes? If that kind of extremely weak code was let through, then this means that the answers to these 3 questions must be all revised.

We are always trying to make Joomla better, hence the rant. If you are reading this and you are a core Joomla developer, then please understand our true motives – we are not against you, we are against bad code especially in the core. We respect the work of all of those who are involved in Joomla, and who are considered, by programming standards, to be programmers.

Now, we turn to you, our dear readers. If by any chance you are reading this post because you updated to Joomla 3.6.+ and you can’t see your new articles, then fear not, we are here to help! Just contact us and we’ll fix the problem for you in no time and for a very affordable fee. We will also make sure that our fix is update proof!

We are currently getting swamped with hacked Joomla websites with a malicious security.php file in their root directory (e.g. at the same level of the index.php file). The name of that file is extremely misleading because it implies security, while, in reality, it is the complete opposite. In fact, Joomla websites should not contain a file carrying this name anywhere (not in the root directory, not even in any subdirectory).

Ever since August 20^th, 2016 (it was when we saw the first occurrence of the security.php file), we are seeing more and more of these malicious files on Joomla websites.

So, how is this file uploaded onto the Joomla website?

In most cases, the file was uploaded using an exploit on the Joomla website, and that exploit is typically a vulnerable extension. We know that because the absolute majority of Joomla websites infected with that file are running the latest Joomla version (3.6.2 at the time of writing this article), which is known to be secure. In some cases, the file gets uploaded because of a vulnerability on the LAMP server, but again, the absolute majority of the time the culprit is an outdated extension.

So, what does this file do?

The security.php file is a main backdoor file, that is used to control the Joomla website (even when all the other exploits [vulnerable extensions, outdated Joomla website, vulnerable server] are closed) by allowing the attacker to execute remote commands comfortably. The fact that it has this trustworthy name lets it go unnoticed by many Joomla administrators, who are misled into thinking that this file is actually about security, and deleting it might compromise the security of their websites. In fact, we had several people emailing us about this file as they have never seen it before (almost all of them were convinced that it was part of Joomla 3.6.2 security).

So, what can someone do to protect his Joomla website from that file?

The best way to protect a Joomla website from the malicious security.php is to prevent the upload in the first place by closing all the possible exploits, which consists of: updating the Joomla website to the latest secure version, removing unnecessary and vulnerable extensions, updating all the extensions, and ensuring that the environment is secure (e.g. Linux, Apache, MySQL, and PHP do not have any vulnerabilities).

There is also a fallback protection method, which consists of only allowing the index.php to be access from the outside world as described here.

If you ever see a security.php under the root directory of your Joomla website (or anywhere on your Joomla website, for that matter), then you should resign to the fact that your website is hacked and you should proceed accordingly (you should clean it up and secure it). If you need help with unhacking and securing your Joomla website, then look no further: we are the Joomla security experts! Just contact us and we’ll ensure that your website has stellar security in no time and for very little money!

A new client, with a Joomla 3.6.0 website, called us yesterday evening and told us that every time he tries to update his Joomla website, he sees the following the following error:

Ajax Loading Error: Category not found

So, we went to his website, and tried to update Joomla, but, as expected, we saw the above error, in a JavaScript popup. Hmmm! Debugging these JavaScript problems is typically not easy, we thought… But we promised the client that we will find a solution for him!

So, the first thing that was on our mind was cache: we disabled all types of cache on the website (global cache and system cache), and then we cleared the cache (Joomla cache and browser cache), and then we tried to update the website again, but we still saw the above error.

We searched for the error in the Joomla code base, in hope of finding the exact location of the problem, but we couldn’t find anything. To be honest, it’s not that we didn’t find anything, we did find something, but what we found wasn’t really helpful in getting to the root of the problem. So we decided to stop beating around the bushes, and instead, we took an approach that was completely against our programming habits: debugging Ajax code (nobody likes to debug Ajax code).

So we opened the JavaScript file containing the Ajax code responsible for throwing that error, which is the update.js file (which is located under the media/com_joomlaupdate/js folder) and we checked its code. After a thorough examination, we knew that for us to know what is really going on, we needed to find which PHP file is processing the Ajax request. And so we changed the following line in the update.js file:

var message = 'AJAX Loading Error: ' + req.statusText;

to this line:

var message = 'AJAX Loading Error: ' + req.statusText + ' update URL is: ' +  joomlaupdate_ajax_url;

We then cleared our browser cache (again), and tried to update the website, and this time, the JavaScript alert window displayed the URL that the Ajax request was being sent to. It was this one: http://www.[ourclientjoomlawebsite].com/administrator/components/com_joomlaupdate/restore.php

So, we visited the URL directly, and it had the following error:

404 – Category not found

That was odd, because the file restore.php existed. So we added the following code to the very beginning of the restore.php file (just after the opening tags):

die('In restore.php file');

We then visited the above URL again, but we were surprised to see the same error again. This meant 2 things: either there is an .htaccess rule whitelisting specific PHP files, or the file permissions/ownership were wrong. So we first checked the .htaccess file and we noticed that it had the following code in its very beginning (probably copied from here):

<Files *.php>
	Deny from all
</Files>
<Files index.php>
	Allow from all
</Files>

The above code secures the Joomla website by allowing direct access to the just one file, which is the index.php file. On the flip side, it doesn’t allow direct access to the restore.php file, which is a legitimate file used for the update.

So, we updated the above code to the following to allow access to the restore.php file…

<Files *.php>
	Deny from all
</Files>
<Files index.php>
	Allow from all
</Files>
<Files restore.php>
	Allow from all
</Files>

… and then we visited the website, but still, we saw the same error. So we quickly checked the permissions/ownership on that file and we noticed that both the permissions and the ownership were incorrect: the permissions were set to 600 (note that we didn’t notice this when editing the file because we edited the file while ssh’ing as root), and the ownership was set to root. We suspect that that file was infected previously and so whoever did the cleanup decided to just make the file completely inaccessible (with the exception for the root user) after the cleanup process. So we changed the file permissions to 444 and we changed the ownership and the group ownership to the cPanel user of the website. We then visited the file, and this time, we saw the message that we added in the restore.php file. We then removed the die statement that we have added, and we tried updating the website, and, as expected, it worked smoothly! Hurray! Lush homes all around! (In case you’re wondering, that Lush homes thing is from the game Tales of Monkey Island).

So, if you, our dear reader, are seeing the Ajax Loading Error: Category not found popup when trying to update your website, then fear not: check if your .htaccess file is allowing access to only specific files and also check the ownership/permissions on the restore.php file. If everything checks and you are still having the problem, then it’s time to contact the Joomla experts! We are there for you 24/7/365, we know our Joomla, and our fees are super affordable!

A client contacted us on Friday of last week and told us that he had the following weird problem: every time he clicked on Read Messages (which is located in the You have post-installation messages box) in the backend of his Joomla website, he saw a blank page.

A blank page in Joomla, as you may already know, is a definitive sign of a PHP fatal error somewhere… So, in order to see the error, we set the Error Reporting value to Maximum in the configuration settings, and then we clicked again on the Read Messages link, and we saw the following error:

Fatal error: Class ‘FOFRenderDirs24’ not found in libraries/fof/view/view.php on line 966

Hmmm! What is that FOFRenderDirs24 class? Something smelled very fishy as we have never heard of that class before…

In any case, we checked the line 966 of the view.php file (which is located under the libraries/fof/view folder), and its job was to instantiate an object from a PHP class which name is based on appending the word FOFRender with the name of each and every file name in the libraries/fof/render folder. For example, if the libraries/fof/render folder had two files, one is called first.php, and the other is called second.php, then line 966 would instantiate two objects, one from class FOFRenderOne and the second from class FOFRenderSecond, which means that the first file must define a class called FOFRenderOne, and the second file must define a class called FOFRenderSecond (yes, we know, it’s starting to get confusing a bit – just don’t ask to repeat this paragraph).

Now, the fact that line 966 was failing when instantiating an object from class FOFRenderDirs24 meant only one thing: that there is a file called dirs24.php in the libraries/fof/render and that that file doesn’t define a class called FOFRenderDirs24. So, we checked the libraries/fof/render, and we noticed that we had six files in there, instead of 5 on a standard Joomla 3.6.2 website, and one of them was a zero byte file called dirs24.php. As you might have probably guessed, the dirs24.php was the intruder file, and so deleting that file fixed the problem.

But where did that file come from?

As we have mentioned earlier, the website was previously hacked and freshly cleaned. The cleanup work was done by an automated tool, which emptied files with purely malicious content. The thing is, with Joomla, emptying files (instead of deleting them) is a bad strategy, as the Joomla engine, in certain directories, automatically tries to define classes (and instantiate objects based on those classes) based on the filenames in those directories.

So, if your Joomla website is showing a blank page or a fatal error when you click on that Read Messages button, then you should check the libraries/fof/render for any intruder. If there isn’t any, then the problem might lie elsewhere, and you may need some Joomla experts for help. So go ahead, contact us, we will solve the problem for you, and we won’t charge you an arm and a leg (or even a toe) for it!

Very early in the morning today, a new client sent us the following email:

“Hi,

Is there a fix that does not involve reverting to an earlier version of PHP?”

When we read the email, we quickly remembered our post on the hidden menu manager issue in Joomla 1.5, which is also caused by a PHP upgrade.

So, naturally, the first thing that we did was that we enabled error reporting on the Joomla 1.5 website, and then we went to the Content -> Article Manager page where we were greeted with the following 2 errors (repeated about 20 times):

Warning: Parameter 1 to JHTMLGrid::access() expected to be a reference, value given in libraries/joomla/html/html.php on line 87
Warning: Parameter 1 to JHTMLGrid::checkedOut() expected to be a reference, value given in libraries/joomla/html/html.php on line 87

Aha! So the root cause of the problem was very similar to the root cause of the “hidden menu items issue”: it was an expected reference but value given issue. So fixing the problem was actually very easy; all we did was the following:

• We opened the file grid.php which is located under the libraries/joomla/html/html folder.

• We changed the following line:

  function checkedOut( &$row, $i, $identifier = 'id' )

  to:

  function checkedOut($row, $i, $identifier = 'id' )

• We also changed this line:

  function access( &$row, $i, $archived = NULL )

  to this one:

  function access($row, $i, $archived = NULL )

  Notice the missing ampersand (&) from the second, correct line.

• We saved the file and then refreshed the Article Manager page, and this time, all the checkboxes displayed and worked properly, and all it took was the removal of an ampersand (well, two ampersands, but you get the point)!