[Resolved] Performance issues with Toolset CRED

[Author]

Posts

[smileBeda]

On a website with ca. 180k users, they made some performance tests using New Relic and they found that each time a "Create New User" Toolset Form is submitted, a huge query is run twice, resulting in a 20 seconds lead-time.

The query run twice is:
SELECT DISTINCT meta_key FROM wp_usermeta WHERE meta_key NOT LIKE 'wpcf-%' AND meta_key NOT LIKE '\\\\_%' LIMIT 512

The query comes from function getCustomFieldsList in cred-frontend-editor/library/toolset/cred/embedded/models/UserFields.php

I checked that code and found that the results of this inefficient query are supposed to be saved in the Datbase, as a transient called cred_transient_usermeta_keys_visible512, expiring weekly.
That would at least avoid the hugely inefficient query to be run each time the form is used, however, that transient is never saved.

Thus, the code runs each time the form is used.

Not sure why the transient is never saved.
When I checked I could only find the Views counterpart to this same thing which is wpv_transient_usermeta_keys_visible512

Note that I can save other transients just fine on this website, so its not due to cache or else site-related fault. Something in Toolset Forms seems to simply not save the transient for whatever reason. Thus, the SQL query runs each time.
But even if the transient would be saved, the real issue wouldn't be resolved, just hidden:
That SQL query is not exactly the definition of performant.

If a website has just a couple thousand entries then this might be no issue, but with 180k users as here this quickly becomes a nightmare, even if limited to 512 results.

Anything we can do here?

Let's remember that this all happens when a new user is created with a form, and I am pretty positive we don't need to know each field in the Datbase "not starting with a wpcf- prefix" to create a new user (also not the first 512).
Something is fishy with that query to start with - why is it there?
Shouldn't this be a pure backend thing, used for where we can control non-cred fields with Toolset?
(I might be wrong, but that is what it seems to be: a query to get non-toolset fields. Surely we do not want to run that on forms in the front end where we already know what fields the form will be including?)

Cheers!

[smileBeda]

Further debugging this I find that:

1. Transient is checked if existing when user form is shown/submitted
2. If no transient, query is performed, and data put to transient
3. As soon user_register hook fires (which is ALWAYS when we register a new user…) that transient is deleted again, see line 37 of /cred-frontend-editor/application/controllers/cache.php

This is of course thus completely ineffective.

It does indeed save the transient successfully but it is deleted immediately again, because of course, since we are on a form that creates users... it fires the user_register hook.
It makes no sense at all to delete that transient at this point because the array of user meta field keys does not change when we register a new user. It is not an array of _values_, it is an array of _keys_, and those do not change when we register new users, only when we register new fields thru other-than-toolset plugs.

While it is now clear to me why this happens, there is no solution other than to delete the entire code that performs this query or to delete the code that deletes the transient.
Not sure which is best, and what you suggest to resolve this performance issue thus awaiting your reply, thanks!

[Nigel]

Hi Beda

Let me look into this this morning and I'll get back to you.

[Nigel]

Hi Beda

I followed a similar breadcrumb trail to you.

I don't think there is anything that can be done about the queries themselves. As you know, the CRED codebase is very old, pre-dating most of the developers here now, and it is very opaque. As you note, it is a mystery why some of the code runs on the front end after the form is submitted, when it looks appropriate to editing the form itself in the back end. The code includes comments from the developers including "WTF", so I think it is safe to say they are aware of the problematic implementation, but short of a complete rewrite, that isn't going to materially change. (And a complete rewrite would be lovely, but unlikely.)

So taking advantage of the transients seems like the only real avenue to improve things. The transient is created when the page displaying the form is loaded, but as you have seen, is promptly deleted by a callback hooked into one of several actions.

Those hooks are added in application/controllers/cache.php, from line 37 onwards.

When you look at what is cached in the transient—a list of non-Types user fields—it seems entirely innocuous, and there seems little risk from removing some or most of those hooks and letting the transient persist.

You may want to look at those hooks to see if you want any of them to still trigger deleting the transient, bearing in mind that those hooks may be triggered on submitting the form, so you may need a little trial and error. Even if you were to remove them all the transient would expire in a week and be refreshed anyway.

[smileBeda]

That is truly unfortunate.

We will likely switch to another plugin in this case, while temporarily commenting the delete_transient hooks in the CRED Core code...

I do hope this gets addressed at some point. CRED is doomed to die out on the long term if things do not get addressed and/or modernised.

Well, then perhaps that is the actual long term plan. I get it, CRED is a dinosaur and spaghetti code and all that... yet I cannot fully agree with the "let it rot" approach which seems to be the general approach towards CRED issues.

Thanks anyway!