[Escalated to 2nd Tier] Issues with performance on page loading and pagination

[Author]

Posts

[Bob]

Hi Mohammed,
I've set the staging site to use the latest PHP available from my hosting company.
Apache and SQL are as set/supplied by the hosting company for my server instance.
I cannot change these for my server as it is not a full Virtual Host.
So the staging site versions are (now)...
PHP: version: 7.1.0 RC4
MySQL version: 5.6.33
Apache version: 2.4.16

The above versions should be satisfactory for Toolset to run with and should be typical of what would be found on a great many hosting services.

Thanks,

Bob

[Bob]

Mohammed,
I've had time to do some testing with the staging site running PHP 7.1
Definitely performance has improved substantially on filtering with dependency enabled in the View.
The performance seems good enough to deploy on my live site.
I'll do some further testing and verify performance on all three CPT visual libraries before declaring a win though.

I've fixed a minor PHP bug in my Tabs & Accordions plug-in which was unrelated to any of the reported issues.
That new version is now on the staging site.

I still experience the pagination modal spinner delay (bug #1) issue though so that still needs investigation by Toolset.

Thanks,

Bob

[Bob]

Hi Mohammed,
Just a quick update...
Unfortunately the theme used on the Slide Magician system has some issues with PHP7.
The issue was with order of evaluation of expressions.
A few curly braces added in the right places seems to have rectified the situation for now on the staging site.
I'll apply changes to the live site after further testing.

It is probably time for a bit of a site revamp and change of theme.
That's a bigger exercise which I'll need to plan out (decide on new theme etc.).

So for now let's continue with the staging site testing for the pagination spinner delay issue using the patched theme.

Thanks,

Bob

[Mohammed]

Hi Bob,

I finally replicated the issue on a copy of your website on my local machine.

I see now that you have thousands of posts and categories.

And after enabling the Views debug mode, I found that Gallery side View takes around 20 seconds to load.

So, we are on the right way now.

I will investigate further to know the actual problem cause and take the proper action.

I will reply here ASAP.

Thanks.

[Mohammed]

Hi Bob,

I've removed the Query Filters you added to the view and tested the page.

It loaded in 1.7 seconds !!!

We are closer now.

[Mohammed]

Hi pop,

I'm escalating this issue directly to the development team to investigate in it.

Please wait and I will get back to you once I get any updates.

Thanks.

[Bob]

OK.
Since I only recently made PHP7 fixes to the WordPress Theme on the staging site I do hope you only took the snapshot in the last 48 hours.
Otherwise there are some patches that may need to be applied to your install.
The patches affect editing of CPTs so maybe you did not encounter those issues but be aware they do exist and will result in editing sections for a post not rendering or behaving correctly.

Yes I do have thousands of posts and large taxonomies.
This number grows each month as I catalog deliveries of new content from vendors of animations and images.

In fact the staging site you have been testing with is a reduced set of posts/taxonomies versus the live site.
The code base on the staging site is current but the CPT dataset has not been updated since last year.

On the other hand the live site has 30.6K CPTs and 3.4K taxonomy terms as of February 2017.
The CPT count increases at ~1000 per month.
The taxonomy term increase is substantially lower and sometimes does not grow from month to month.
This happens when no new Content tags (tax terms) are added.
I may also trim/consolidate some terms from time to time and actually reduce the tax term count but have not done so recently.

I'll wait for your further update from dev team.

Thanks,

Bob

[Mohammed]

Hi Bob,

I've escalated your issue to the development team and they checked your installation and provided me the following response.

"Dependency and counters demand that you load in memory all the posts together with some metadata, just to prepare the operation of deciding which filter elements appear or not, and just to prepare the operation of including counters. To perform those two operations, in addition to keeping that huge amount data in memory, we need to iterate it at least once per term. In a form that has hundreds, if not thousands, of terms, you can imagine how many iterations we are talking about."

I'm sorry to say that nothing we can do at this point.

But, I may refer you to this article that would help you to get better performance: hidden link

Also, you may migrate to better hosting to get better RAM and Processor that may help too.

Thanks.

[Bob]

Hi Mohammed,
I figured that would be the response.
In any case the upgrade of the host to PHP 7.1 has provided sufficient performance increase to enable the production host to be switched to using dependency setting on all three Gallery views.

Regarding the guide on performance improvement my site and code is optimised quite well already.
So I'll monitor performance as the site grows to determine if I need a dedicated VPS in the future.

On the matter of the pagination bug did the dev team look at that issue?
That one is a real problem with users since when they click on pagination and do not get any feedback that the system is thinking (no modal spinner from Views) it causes users to click again and again and to think that the system has gone dead.

Please look at that bug in the snapshot you have.
Follow the instructions provided earlier in this thread to reproduce the issue.

Thanks,

Bob

[Mohammed]

Hi Bob,

I"m posting here just to notify you that there is a work in progress regarding this issue.

I will post here again once I get any updates.

Thanks.

[Bob]

Hi Mohammed.
I'll look forward to receiving your update(s).

Thanks.

[Beda]

HI Bob.

I just wanted to update you that I am going to change the status of this ticket to "Escalated to Second Tier" and i will also let you know what the plans are:

- On a View listing posts, after adding a taxoomy filter, or on a View listing terms, after adding a filter by specific terms, we display a set of checkboxes for all the terms in that taxonomy.
- On sites with many(!) terms, this might be too slow.
- We will try to avoid loading that loist by default, and load it on demand and even improve this with some lazy-loading, or select2 search.

This are plans, not promises, or ETA's.
Those might change. The current idea is to improve this in future (several releases in future) releases.

I will update you here once we have something more to show in this scenario.

Thanks a lot for the patience.

[Bob]

Thanks Beda.
I did look at possible strategies that might be implemented to improve performance some time back.
One was to break the taxonomy list into alphabetic segments and then lazy/deferred load sections of the taxonomy checkbox fields under tabs/accordions by controlling the backend rendering.
FYI I have my own Views tabs/accordion plug-in I developed with nested tabs/accordion capabilities and re-draw of the tab/accordion JS structure on View refresh.
To gain performance improvement I would need to combine the tabs/accordions with a taxonomy filter to split out the taxonomy into sections.
I did propose a taxonomy filter some time back (to Adriano/Mohammed) and actually prototyped that as an enhancement to Views.
The prototype used a simplified regex added to the Views filter to select which taxonomies to render.
It also implemented the free form search of taxonomy (using the search box) in the same prototype.
It might be time to get all that code out again and look over it.
If I recall correctly all the filtering was at the backend and depending on the query performance it may be a way to improve performance.

Regards,

Bob

[Beda]

I understand.

I will update here as soon I have more news.