Skip Navigation

[Closed] return large data set with 42000 records

This support ticket is created 8 years, 5 months ago. There's a good chance that you are reading advice that it now obsolete.

This is the technical support forum for Toolset - a suite of plugins for developing WordPress sites without writing PHP.

Everyone can read this forum, but only Toolset clients can post in it. Toolset support works 6 days per week, 19 hours per day.

Sun Mon Tue Wed Thu Fri Sat
- 8:00 – 17:00 8:00 – 17:00 8:00 – 17:00 8:00 – 17:00 8:00 – 17:00 -
- - - - - - -

Supporter timezone: America/Sao_Paulo (GMT-03:00)

This topic contains 28 replies, has 3 voices.

Last updated by Adriano 8 years, 2 months ago.

Assisted by: Adriano.

Author
Posts
#352198

so, i've created a formatted table that gets generated from a view, which pulls all of the product attribute information. it also includes the sku and category info. you can see it here:

hidden link

this dataset when coming from the pre-built table returns 51,848, which is the total number of all my products. it returns in about 13 seconds now. and if i use a sql where statement to filter a category it's even faster.

so really, this is a great potential start to a solution... i just need the views to be able to pull from this pre-built table and run the parametric search off it?? can we make this happen?

i know that i will have to update the pre-built table if any of my product data changes, that's why i created a view to pull the product meta keys, then a query to drop/create the data table.

now, i think if we can just tie the views plugin to this table and return it and search it then my return time/failure issue will be resolved.

can you please take a look at the link i provided and please respond if this is at all feasible?

#352333

Hello,

I'm Adriano, Toolset support lead. I'm checking your issue, give me some time to understand everything and do some tests. I'll keep you posted.

#352367

I'm doing some tests in the dev environment but it seems fine there? hidden link

How can I reproduce the problem?

#352373

the problem was never the hidden link which is the velocity data... it's always been the other dataset which contains 42,000+ records, which is this link hidden link

the second one fails when attempting to push that many records through the types view

#352416

Sorry, I see the problem now. I will consult our Views development team to see what we can do regarding this issue. I'll keep you posted.

#352709

Is that possible to put in your FTP folder the database dump and a zipped file of your entire WP installation? It will help a lot to debug it locally.

#352779

archives are available in the root hidden link.
current wp: gcvd.wp.20151209.zip
current db: velocitydata.20151209.sql

#352996

Ok I've found it. I'll pass the files to our development team and they will check what can be done. I'll keep you posted.

#353000

Thanks! I've added some debug information in wp-config.php of your dev site but the error_log.txt isn't being created. Could you share the information of your PHP log?

define('WP_DEBUG', true);

// Enable Debug logging to the /wp-content/debug.log file

define('WP_DEBUG_LOG', true);

// Disable display of errors and warnings 

define('WP_DEBUG_DISPLAY', true);

//@ini_set('display_errors',1);

ini_set('log_errors',TRUE);
ini_set('error_reporting', E_ALL);
ini_set('error_log', dirname(__FILE__) . '/error_log.txt');
#354574

I'm not sure how to help with the logging. I don't understand php logging. Do you have full access to the ftp? Shouldn't this enable you to be able to see what's going on? Have you been able to determine anything from this?

#354888

Ok, I've forwarded your site backup to our Views expert, he will check what can be done there. Thank you for your understanding, we will keep you posted.

#364759

We will release a new version for Views this week, 1.12. We expect to see some improvement in the rendering speed of some kind of Views. Could you wait until then and give us some feedback? Thank you for your help and understanding.

#364798

Awesome. I look forward to testing that out. Thanks.

#365897

After some tests we found the errors below on your log when accessing the login page:


The client's site is showning some very nice PHP errors when trying to log into it or even visit the login page.

Notice: The called constructor method for WP_Widget is deprecated since version 4.3.0! Use
__construct()
instead. in /home/content/71/10299071/html/velocitydata.com/dev/wp-includes/functions.php on line 3457

Warning: Cannot modify header information - headers already sent by (output started at /home/content/71/10299071/html/velocitydata.com/dev/wp-includes/functions.php:3457) in /home/content/71/10299071/html/velocitydata.com/dev/wp-login.php on line 431

Warning: Cannot modify header information - headers already sent by (output started at /home/content/71/10299071/html/velocitydata.com/dev/wp-includes/functions.php:3457) in /home/content/71/10299071/html/velocitydata.com/dev/wp-login.php on line 444

Fatal error: Out of memory (allocated -1228406784) (tried to allocate 20 bytes) in /home/content/71/10299071/html/velocitydata.com/dev/wp-content/plugins/leaflet-maps-marker-pro/leaflet-core.php on line 2

So it seems that your site is having some performance problem. However, I do not think there is much we can do here. Parametric search, in a raw sense, is expensive but acceptable. When you add dependency and/or AJAX results, it becomes much more expensive. If the post type that you are querying has lost of posts, those functionalities need to get that for each one of those posts, hence hitting the database hard and requiring lots of resources. You can always disable those extra things in the View edit page.

The topic ‘[Closed] return large data set with 42000 records’ is closed to new replies.