Skip Navigation

[Fermé] Views/Parametric Search Memory Requirements

This is the technical support forum for Toolset - a suite of plugins for developing WordPress sites without writing PHP.

Everyone can read this forum, but only Toolset clients can post in it. Toolset support works 6 days per week, 19 hours per day.

This topic contains 4 réponses, has 2 voix.

Last updated by Beda Il y a 6 années et 6 mois.

Assigned support staff: Beda.

Auteur
Publications
#314780

Using Views 1.9, I have run into an issue that is clearly memory related. I have a large website project under development, and ran into these issues in the past that were solved with the assistance of developer Juan. However, they were pre-Parametric Search for Views, and its been a while in the development process for me to get back to the search requirements.

The website under private development currently has over 85k posts, that I expect to grow much larger. Using nested Views, I am able to query the custom post types and present them in a manner that uses a moderate amount of RAM... however, there is a massive discrepancy between what Views reports for Memory usage, and what other tools, such as DebugObjects reports. I am more inclined to believe DebugObjects. Views reports memory usage around 12 MB, DebugObjects may report up to 120MB. That particular View, at least, does have a great deal going on.

However...

When I attempt to setup a very simple View with a simple search field, I am forced to provide limitations, such as only the last 15-20 days worth of posts, that defeat the entire purpose of the search; to allow ALL posts to be searched and the results returned about 25-50 at a time with pagination. When the search is displayed, Views reports RAM usage at 9MB, yet DebugObjects shows about 95MB. Removal or increasing of limits results in the page not displaying and eventual memory errors in PHP.

1. Why is there a massive difference between what Views debug output shows as RAM usage and DebugObjects seems to MUCH more accurately show?
2. Just what is Views and the Parametric Search feature doing to use so much RAM???
3. Can I have the old search back, without the Parametric bloat?
4. Is there a "How to Optimize Parametric Searches" resource?

I'm seriously concerned about inefficiency and ultimately, code bloat... and over a year into the project, I'd rather not have to switch to DataTables...

#314939

Thank you for contacting us here in the Support Forum and for providing the Debug Informations

It would be good that you first update your Types Plugin to it's latest version. Can you do this please?

❌ Be sure to backup your database first before you proceed! ❌
⌥ You can use a plugin for this if you like.
I often use the Duplicator plugin for this purpose.
See: http://wordpress.org/plugins/duplicator/

I apologize that you are experiencing issues with the Views Plugin.

Considering the relevant amount of Data Views has to Query in this case, I must threat it a bit as a "special case", this does not in any way mean I don't take it serious.

It's just not a common situation, and actually it will be very helpful for us to troubleshoot this, and to adapt the Plugin to able to handle such Queries with the same ease as it does for a lower amount of data.

To do this, I will need a few details and specifications:

1. Please note that Theme and other Plugins can have a high impact on performance.
Please try to test also with:
A) Toolset + Custom Theme and Custom Plugins.
B) Toolset + Custom Theme BUT Custom Plugins OFF.
C) With default WP theme and other non-Toolset related plugins OFF, BUT Views ON.

2. Please be aware that newer PHP versions are essentially always faster. Same goes for newer MySQL and Apache
==> I see you use PHP 5.4.42 and MySQL 5.5.41
Would it be possible to contact your Server Admin and gently ask him if eventually a update would be possible?
(those are already quiet up to date, I would focus on PHP version in this case)

3. Also newer CPU's and more RAM can help to improve. Please provide me this Info about your Server:
-CPU data
-Software version
-Shared or VPS server?

4. Please check with your Server Admin if some OPCache Scripts are installed on your Server:
- APC hidden link //Older PHP versions <5.5
- Zend-OP-cache hidden link //newer PHP versions >5.5

5. Could I ask you to install "Query Monitor" plugin to measure performances as well?
https://wordpress.org/plugins/query-monitor/

"Query Monitor" provides generic and detailed reports:
focus on queries per component to get the actual queries ran by Views.

Try to use only Views and a minimum set of Theme and 3rd Party plugins, so we can isolate this on our Software.

Please then provide me with eventual slow queries or repetitive queries you may find during this process.
(screenshots or pasted in pastebin.com would be enough)

6. Also I need to know what the "general" data is, which you are querying:
- Does it contain lot of Images, Videos or similar?
- Is it bare text?

I also would need a Site's Snapshot so to reproduce this locally.

We usually recommend the free Plugin "Duplicator" for this porpoise.

If you already know how Duplicator works
(http://wordpress.org/plugins/duplicator/),
please skip the following steps and just send me the installer file and the zipped package you downloaded.

★ Duplicator Instructions
hidden link
Send me both files (you probably want to use DropBox, Google Drive, or similar services, as the snapshot file will be quite big)

❌ IMPORTANT ❌
Remember to create or keep an admin account for me before creating the snapshot, or I won't be able to login. You may delete the new admin account once the snapshot has been built.

I will enable your next answer as private so you can provide me the information securely.

Please could you provide me the additional Infos?

Regarding your specific Questions, I will contact our DEV and ask for those, so I can provide you exact information as soon I have details.

Thank you

#316500

Thanks for the Details

I believe you already spotted the issue here, by locating the issue mainly in the setting "Show only available options for each input"

This issue is known, specially on very large databases as yours. Our DEV is working hard on a solution, and it is planned to release a fix for it in Views 1.10

Regarding your initial Questions, I am waiting for more details, so I can provide you with accurate information.

It would be good for our DEV Team to have some sites to test Views 1.10 on it, and your site with over 80 K Posts, would be very appropriate for this.

Would you agree, and would it be possible for you, to provide me with a site's snapshot?

With this, our DEV can test the changes made on a real Site which has a large database. As you can imagine, it's not a common situation and it is always best to test the product on different scenarios, where your site with the mentioned amount of content entries would be perfect for.

I would need a sites snapshot if possible.
We usually recommend the free Plugin "Duplicator" for this porpoise.

If you already know how Duplicator works
(http://wordpress.org/plugins/duplicator/),
please skip the following steps and just send me the installer file and the zipped package you downloaded.

★ Duplicator Instructions
hidden link
Send me both files (you probably want to use DropBox, Google Drive, or similar services, as the snapshot file will be quite big)

❌ IMPORTANT ❌
Remember to create or keep an admin account for me before creating the snapshot, or I won't be able to login. You may delete the new admin account once the snapshot has been built.

I will enable your next answer as private so you can provide me the information securely.

As mentioned, I can unfortunately not provide a solution for the issue right now, as we must wait for Views 1.10 to be released, but I will keep you posted and informed on any progress made, and I can discuss with the DEV if it's possible to provide you a DEV Version once the changes are implemented.

Please could you provide me the additional Infos, and please do not hesitate to open a new thread if other issues or problems arise

Thank you

#317602

I update you here with the Infos form the 2nd Tier and DEV on this

Note that a Duplication Package isn't anymore needed.

The situation and possible solutions are as follow:

If a site has +80K posts, it is going to be slow no matter what. Parametric search does expensive queries by definition, since it needs to check and compare data on different tables. The more filters you have, the more expensive it becomes. Also, the more posts you have, the more expensive it is too.

Views 1.10 will cache the first page load of any View displayed in the site. It should help with the first page load, obviously, but we can not cache any parametric search combination of selected options. So searching itself will still be expensive. The setting to display just relevant options must check every option to see if it is going to produce results, so it is expensive by definition too.

If you want basic searching, the most lightweight one, the one that looks just like the one we had before we enhanced it, you can just select "Let me choose individual settings manually" in the Parametric Search settigs, and then make sure you disable the expensive things.
Those two things should return the search to the state we had before 1.9:
- Check "Update the View results only when clicking on the search button".
- Check "Always show all values for inputs".

Regarding the big difference in the reported consumed RAM:
Views numbers are orientative, and it is good that you are combining them with other sources.
Anyway, those two mentioned above settings, when disabled, should reduce the amount of RAM needed, since Views will not need to do lots of operation to decide whether each option should be shown and no automatic AJAX-related event should happen.

There is no generic way of optimizing parametric search, just some guidelines:
- If you have +80K posts, the database will suffer. With more than one filter, it can suffer a lot.
- Use taxonomies over custom fields when available, since the table access for that data is much faster.
- Disable features like automaic updates and options checking, to have just the plain old parametric search.

So a summary just tobe clear:
some settings and operation are expensive by definition.
You should disable them if they cause perfrmance issues on your site, since it all depends on your amount of data.
Views 1.10 will help on the first page load, since it will cache the output so later visitors do not need to perform the queries.
But as soon as you do perform a search, things are computed on-the-fly, generating the queries that might be expensive.

In the future releases we will also add notifications, if your View is querying certain amount of data, and this notification will not only include the warning that certain settings might slow down the server, but also links to dedicated DOC pages on wp-types.com on how to:
- Cleaning up your database (removing revisions, etc.)
- Disabling the 'relevant results' option in Views
- Links to resources on DB optimization

Please let me know if you have further questions regarding the issue mentioned in this Thread
and do not hesitate to open a new thread if other issues or problems arise

Thank you

#318012

Thanks for the Details

I am consulting with the Views DEV if any (theoretic) issues could arise if you use MemCache systems, but I don't think it will conflict with Views 1.10

Thank you very much for willing to share your solutions in this Forum, it will help as well other users in future!

I am glad you could solve it already with the current Views Version (partially) and I am sure the next Views vs. will improve the experience in this matter as well.

Please expect my informations here in the forum, in regard to your previous Cache related concerns.

Thank you

Le sujet ‘[Fermé] Views/Parametric Search Memory Requirements’ est fermé à de nouvelles réponses.