Skip Navigation

[Resolved] Very slow performance | slow filtering.

This is the technical support forum for Toolset - a suite of plugins for developing WordPress sites without writing PHP.

Everyone can read this forum, but only Toolset clients can post in it. Toolset support works 6 days per week, 19 hours per day.

This topic contains 21 replies, has 3 voices.

Last updated by Waqar 7 months, 3 weeks ago.

Assigned support staff: Waqar.

Author
Posts
#2150541

Refer this ticket to Waqar.

Hi Waqar,

I am referring to this ticket https://toolset.com/forums/topic/very-slow-performance/ which was closed without any solution due to your holiday, I assume.

We've unckecked the "show only filter" option but filtering keeps being very slow. It takes almost 60 seconds to load the cities (stad)!

Moreover when we try to visit the next pages on hidden link the pages keep loading very slow (also as a logged out user).

Please suggest a solution that will significantly improve the performance.

#2151093

Waqar
Supporter

Languages: English (English )

Timezone: Asia/Karachi (GMT+05:00)

Hi,

As mentioned in the other ticket, your website is not accessible at the moment.
( ref: https://toolset.com/forums/topic/pages-dont-show-for-logged-out-users/ )

Can you get in touch with your hosting support team to resolve this and let me know once the website is back online?

regards,
Waqar

#2151121

Both are back online.

#2155865

Waqar
Supporter

Languages: English (English )

Timezone: Asia/Karachi (GMT+05:00)

Thank you for waiting.

I've been performing some tests and speed analysis on your website at different times, but the overall results are not much different than the previous ticket.

I'm currently performing some tests on your website's clone on a different server and will share my findings, as soon as this testing completes.

Thank you for your patience.

#2163311

Waqar
Supporter

Languages: English (English )

Timezone: Asia/Karachi (GMT+05:00)

Thank you for waiting.

Just wanted to let you know that I had a busy forum queue last week, but, I'll be resuming investigation on this ticket today.

I'll share my findings and recommendations, accordingly.

#2164205
httpslandman.re.png

Refer this ticket to Waqar.

Hi Waqar

I understand but this issue should be solved ASAP as it hugely affects our sales, clients and business!

Our server is not responding (and even goes offline regularly) and the site speed is VERY poor due to many inefficient queries that are executed in Toolset, as mentioned several times in previous threats.

Error message:

Deze site is niet bereikbaar
hidden link werkt even niet, of de pagina kan verhuisd zijn naar een nieuw webadres.
ERR_HTTP2_PROTOCOL_ERROR

See mobile screenshot attachment.

We have added two scripts for you on the server with which you can check it yourself:

This script would not load if the server is down, or if there are problems with the hosting:
hidden link

This script shows you whether the database is up or down, along with a bunch of other stats from which you can deduce whether the database is busy or not.
hidden link

In other words, if both links above load normally (and quickly), the problem is entirely with our Toolset instance.

When I look at the stats of the database (and you can do that yourself via the link above) I see that there are an average 77 queries per second. That in itself is not an abnormal number, but interesting is the number of slow queries (is now around 7.000 (!!)).

When we log in to the database ourself, I also constantly see queries that take up to 70 seconds (!!) and slow everything down. And that is a clear problem that needs to be solved ASAP. Ther problems are not caused by the server we have set up, or the database we have set up, or our configuration, but are entirely due to the code itself.

Hope to hear back from you very soon with a solution.

#2165435

Hi Waqar

Thought you would share your findings and recommendations september 8th?

This is a **major** issue for which we want a **solution** very soon.

#2171203

When will you reply to this ticket Waqar? We are again a week further...

#2174077

Waqar
Supporter

Languages: English (English )

Timezone: Asia/Karachi (GMT+05:00)

Thank you for waiting and I really apologize for the time this ticket has taken.

The challenge here is that the website is really huge and the conventional methods to create a fully working clone/snapshot failed, despite using different approaches.

I tried the manual migrations method as well, but even that didn't work as effectively as I hoped to give a true picture of the website's performance. I'm currently working with my team member to achieve this using a different method ( Cloudways WordPress Migrator plugin ) and I'll keep you updated on the progress and findings, in a more timely manner moving forward.

Thank you for your patience.

#2174115

Waqar
Supporter

Languages: English (English )

Timezone: Asia/Karachi (GMT+05:00)

Following up on the earlier message, we'll need to split the two issues and deal with them seperately.

1. The first issue that you originally reported in this ticket revolves around the slow performance of the website, specifically when the relationship search filters are used in the front-end views.

For that, we can have that investigated, by cloning the website on a different server and suggest the improvement accordingly.

2. However, if your server is constantly getting down without any major work done on it or even any updates, then that is a totally different matter and requires investigation at the server level.

There is nothing in the Toolset plugins that can keep the server resources constantly engaged in the background, even when the website's pages are not loading or getting refreshed on the front-end.

Today, when I try to access any page from your website (including the two debugging links that you've shared), it just keeps loading and eventually, shows the error page: "This site can’t be reached with ERR_HTTP2_PROTOCOL_ERROR", after some time.

Have you checked the server's traffic statistics and noticed any extraordinary spike in traffic? It is possible that the website is under DDoS attack from bots acting as a regular traffic and unrealistically exhausting the server resources.
( ref: hidden link )

Please also check the server for any malicious code, which can be running in the background and consuming the server resources. The server's error logs and the hosting support team can help you identify exactly which code is utilizing the most resources.

If you still feel this is only happening because of the code in the Toolset plugins, you can temporarily rename the folders for the Toolset plugins in the "plugins" directory, which will deactivate them.
(but your Toolset settings and data will not be lost)

Before, we can continue the investigation around the point 1, it is important that the issue described in the point 2 is addressed and the website is accessible.

#2174261

I can confirm that this MAJOR performance issue is NOT server related, there is NOT any extraordinary spike in traffic, the website is NOT under any DDoS attack or the server does NOT have malicious code. The issue is simply due to the code in the Toolset plugins which keeps the server act SUPER slow.

#2174981

Nigel
Supporter

Languages: English (English ) Spanish (Español )

Timezone: Europe/London (GMT+01:00)

Hi Matthias

I have identified several issues with the set up on the search page responsible for making it so slow and affecting the server.

On the copy of your site installed on a test server, it was taking 1-2 minutes to load the test page, and a similar time if you selected a province to update the list of cities. It's now taking a few seconds to load the page, and no more than a few seconds to update the city list.

I'm going to attempt to make the same changes on your server now, hopefully it will remain stable long enough for me to do so, after which hopefully you won't run into such problems again.

I'll post again with more details.

#2175165

Nigel
Supporter

Languages: English (English ) Spanish (Español )

Timezone: Europe/London (GMT+01:00)

Screenshot 2021-09-21 at 15.17.46.png

I've made the same changes on your site, which appears to be working much better, although not as well as on our test server.

While I was testing on that server, I had disabled most non-essential plugins to work on the issue more easily.

Your site has a lot of active plugins, which all influence performance. With the Query Monitor plugin added for debugging, it can be seen that page load times with these other plugins active more than double, given a 3-fold increase in the number of queries run.

As general advice, I would suggest you delete the 70+ inactive plugins on your server—they are just taking up server space—and review the active plugins to see if they are all really needed.

Also, it looks like you are accumulating post revisions and autosaves, which will increase the size of your wp_posts and wp_postmeta tables, and the size of your wp_postmeta table is particularly problematic, which I'll come back to.

I suggest you edit your wp-config.php file and add a setting to limit the number of revisions stored to whatever you think you need, maybe to just 1, and at the same time perhaps extend the auto-save interval to, say, 10 minutes, e.g.

define( 'WP_POST_REVISIONS', 1 );
define('AUTOSAVE_INTERVAL', 600);

Then use a plugin (e.g. WP Sweep: https://wordpress.org/plugins/wp-sweep/) to clean out existing revisions and auto-saves.

Regarding the property search page itself, I found several things that can affect performance, one in particular.

This is nothing to do with Toolset itself, but rather reflects the underlying architecture of WordPress. Specifically, it is extremely inefficient to filter queries using custom fields (i.e. post meta).

It is very quick to retrieve a post meta value so that it can be used or displayed, but if you query the database for posts and use post meta in the search parameters, that is extremely slow.

You wouldn't notice on smaller sites, but it becomes apparent on larger sites.

Checking the database, your site has 2.6 million post meta entries, so any query that is based upon post meta will inevitably be slow.

In the case of your property search page, when you first load the page and no filters have been applied, it was still extremely slow to load the page, and that is because the View was set to order the results by a custom field, meaning that every time the query runs the post meta table is included in the query, even on the initial load.

Changing that setting so that custom fields are not used for ordering was the single biggest factor accounting for the improvement in page load times. (Currently the order is set to random, which is what you had for the secondary sort option, but that likely means that the View cannot be cached, and so I would suggest an alternative, based on standard post fields such as post date or post title.)

In the back end it is still quite slow loading the page to edit, and—again—the problem lies with the size of your wp_postmeta table and how inefficient post queries filtered by post meta are. In the screenshot you can see an example of the results shown by the Query Monitor plugin when loading the page in the back end. The key point to note is that a query run by WordPress core (nothing to do with Toolset) is taking 17s to complete (the query is just to determine which meta boxes to include on the page edit screen). That's a direct result of having 2.6 million rows in your wp_postmeta table.

The other changes were less dramatic, but still helpful.

The View block has a setting "Don't include the current page in results", which in this context is redundant, but has a modest effect on the query times, so I disabled it.

The Map block had markers from the View added twice for some reason (the markers were duplicated, but you wouldn't notice looking at the map), so I removed one of the markers.

Lastly there was a broken distance filter in the View that wasn't shown on the front end, so I removed it.

Although the site is not quite as zippy as the clone we have on our servers that doesn't have all of the other plugins active, it appears to be significantly more useable and stable than before, if you'd like to check.

#2176177

Waqar
Supporter

Languages: English (English )

Timezone: Asia/Karachi (GMT+05:00)

{ticket status updated}

#2176249

Hi Nigel,

Thank you for the update.

What non-essential plugins did you actually disable? If these plugins are currently not in use we can disable them as well in order to decrease the number of queries being performed and increase the performance. We would like the site to appear as zippy as the clone you have on our servers.

If I understand the biggest factor that currently slows down the complete site was due to the ordering of the posts following a custom field. The downsize is now that we can no longer show properties on top that have 'verkoopstatus' for example? Can you suggest a workaround for this?

I will wait for your further update on the problematic issue with the size of our wp_posts and wp_postmeta tables and how to solve it.