[Resolved] Form notifications not sent when using WP Mail SMTP

[Author]

Posts

[Beda]

Great, at least we know now for sure this is not due to debug modes but to SMTP

I am off unfortunately now until Tuesday. I would like to replicate this if possible locally and analyse it further.
Could you maybe just provide a copy of the site (https://toolset.com/faq/provide-supporters-copy-site/) and instruct me how I can use the SMTP on it (I assume I will be able to use my own Google for this?).

I can then make some tests and eventually we can make some adjustments if I can find a problem in Toolset forms.

[Beda]

Thank you, I received the data and could download it.
I am now deploying it and will update you as soon I have some findings.

[Beda]

OK, deploying the site locally is good for nothing as the SMTP cannot accept fake sites - this is a google restriction.

So I had to set up a full online testing site and deployed SMTP with Toolset Forms there.
The test email sent within the SMTP testing screen was sent successfully, so I can trust I set the Google Application settings correctly.

I then would expect any form, sending notifications to any email, to work just the same.
So I created such forms and tested them, the emails all arrived!

This is good news, it means, it is not due to a conflict but some setting on the site you have, compared to mine, or the field we send notifications to, or the type of form.
I will add the access to the Page in my next (private) reply, so you can log in and see and test the 2 forms I created on the sample page (right now, it's the user form).
It will send the email successfully to the one set as user email.

I would suggest testing on your own site, after my experiences with the SMTP setup:
- that the testing email from Settings > WP Mail SMTP > Email test is sent and received
- that the Google App settings are correctly followed step by step as outlined here hidden link.
- In general, SMTP works with Forms as seen on the testing site, however, it might be that some specific setting was missed by me. Please, can you help me spot this?
If we cannot find any difference I'll see to deploy the very duplicate to the site, however, I have a feeling that after changing the Google App data in the site it'd start to work. I think there is some misconfiguration somewhere in the Google App settings themselves. Had you already tried to set up a new Gmail App?

I will add the access details to the testing site below, please log in there to find the forms under Toolset > Post (and User) Forms, inserted to the Sample Page. You can test them at your will, and change the forms as you think required.
You could also export the form from the site of yours, then import to this new test site, to see if we can spot the difference like that.

[helenmaryC]

Thanks for all the work on this!

I compared the form on your site that works with the form that I have been using and found several differences. So I went back to my test site and started rebuilding the form from the ground up, using your settings for the form and then changing each one to match what I originally had. Everything worked fine until I added in the body text. After that, I no longer received the notification, even though it showed as sent in WP Mail Log. I think it is the html, but not sure. I have to head out to a meeting, so I will look into this more later.

I think we are making progress!

[helenmaryC]

Does Forms set wp_mail_content_type before sending a notification? The WordPress Code reference (https://developer.wordpress.org/reference/functions/wp_mail/) seems to say that this needs to be done before HTML email can be sent. The odd thing is that the test email works with the HTML in it, just the real notification fails. Could the test use the filter and the real notification not?

The default content type is ‘text/plain’ which does not allow using HTML. You can set the content type of the email either by using the ‘wp_mail_content_type‘ filter ( see example below), or by including a header like “Content-type: text/html”. Be careful to reset ‘wp_mail_content_type’ back to ‘text/plain’ after you send your message, though, because failing to do so could lead to unexpected problems with e-mails from WP or plugins/themes.

[helenmaryC]

Hmm. I tried inserting the same HTML text used in the body of my notification into your notification and it is received just fine, so setting the content type shouldn't be the issue. But it is odd that it would work on your test site and not on mine. I'll try to see what other differences I can find.

[Beda]

Everything worked fine until I added in the body text. After that, I no longer received the notification, even though it showed as sent in WP Mail Log.

Then we have made a great step, once again, towards excluding conflicts and limiting the area of infection ( 😛 )
The notification body, as I see on your site, does hold quite some data.

Please follow as below:
1. Ensure, to 100%, that notifications work in your current live install just fine when you remove the content from the body of the notifications
2. Just to be sure, also remove the JS from the Form to test if that has any effects - and if not, then adjust the script to the proper syntax. You should always use "jQuery" instead of "$", see https://toolset.com/documentation/user-guides/adding-custom-javascript-views/#achieving-great-results-with-little-coding-by-using-jquery
3. If at this point you can still positively confirm that the Notification works only if the notification body is omitted, please start with these steps:
- insert the full notification body as you have it now and start subsequently to remove parts.
==> Start with [wpv-user field="user_firstname" id="%%USER_USERID%%"].
==> If not working, remove all HTML

Note, I tested this on the testing site I made, with your HTML and I received the Email but it did not execute the ShortCode [wpv-user field="user_firstname" id="%%USER_USERID%%"].
Forms support HTML and even ShortCodes in the notifications, and should hence not conflict with other Plugins sending them, as the exchange to plain text is done before sending the email, hence, the ShortCodes at this point (inclusive HTML) should be executed.

It was not executed (the shortcode) on the test site because Views was missing - so after installing Views, that part also worked flawlessly

Given the exact same form works on the testing site with the same plugin settings and SPMPT - the only variable left is the server itself and the Google Application you created.
In this case, remains to test with a new Google Application, and if that does not solve the issue, then eventually server admins (and the logs of the servers) may help to narrow down where the sending fails.

Note, you are free to change the SMTP Settings in the testing site and try again - that can show whether your Google API is properly configured or not.

The very last trick I'd resource to is to take the testing site we have now, and add everything you have on the live site to the testing site, to either see when it starts failing or, which I suspect, it should then work just fine.

[helenmaryC]

When I got back to this this morning, I realized that the HTML is NOT the problem. Apparently, when I was testing before, one of the settings which I changed was "After visitors submit," but I must have not saved it, thought it worked, and then decided that the next thing I changed, the body, was the culprit. So I started from the beginning again and realized what was really going on. Sorry for the confusion!

It actually seems to be a combination of issues involving custom user fields and the page redirect. The notification always seems to work if I redirect with at least a 1 second delay. If I redirect with a 0 delay or stay on the same page, the notification is not received. If I change the form to remove the custom user fields, the notification sends no matter what the setting is for After Submit.

I was going to try this on your test site, but Types isn't installed. Can you install it and see if the same problem happens there?

Thanks.

[Beda]

I installed Toolset Types and imported all User Custom Fields from your existing site.
I noted that strangely in your site (hidden link) you have only one Custom Field Group for Users with just 3 fields: hidden link
However, there are more User Fields, like u-userid, and others, which seem to have been maybe deleted in past at some point but they are still in the Fields registered here:
hidden link
Not saying that this is the issue, but if you plan to fully remove Fields, you can (and should) always fully delete them (this will also delete the data in the database stored for them) in the screen where you control the fields (hidden link)

Anyway, after importing all User Fields, the fields are successfully added to hidden link.
So I updated the Toolset Form here hidden link
I already had the form to keep displaying, so there is no redirect delay at all (0)

For me, the email of the form still works.

Can you try to see if I made a mistake in what fields are added, and/or if they need to be completed specifically?

[helenmaryC]

Thanks for installing Types on the Sandbox. I really thought that was going to show that this was the problem, but, as you indicated, the notification works just fine there even with the custom user fields included on the form.

I did not know that there was an additional step to deleting old fields. I went ahead and deleted unused fields on my test site. It did not initially seem to make a difference. In fact, the notification on my simple test form stopped working when the form continued to be displayed even if there were no custom fields on it. I next tried deleting the notification and recreating it from scratch. The notification then started working regardless of custom fields or redirect settings. I'm not sure if it was because the unused fields were no longer there or because something that was causing a problem got cleared out by recreating the notification, but it was progress.

So I tried the same thing on my production site. Deleted the unused fields. Deleted the notification and recreated it. The form includes the custom fields. The notification works if I have a delayed redirect, but not if I keep displaying the form.

There are two major differences, at this point, between the production site and my test site. Elmentor is enabled on production and my "housekeeping" plugin, crfcrc-data, is also enabled. The later includes the following code:

add_action('cred_save_data_6424', 'crfcrc_save_primary',10,2);
function crfcrc_save_primary($post_id, $form_data) {
    if (isset($_POST['crfcrc-primary']))
    {
        if (is_numeric($_POST['crfcrc-primary'])) {
            // add it to saved post meta
            update_user_meta($post_id, 'wpcf-primary-id', $_POST['crfcrc-primary'], true);
        } else {
            update_user_meta($post_id, 'wpcf-primary-id', $post_id);
        }
    } else {
        update_user_meta($post_id, 'wpcf-primary-id', $post_id);
    }
}

I wouldn't think this would cause a problem, but maybe it is. I am going to try putting the test site back together so it is again identical to the main site, then try turning off this code and see if that makes a difference.

I'll let you know what happens.

[helenmaryC]

I am now completely and totally confused.

I went back to my test site to start re-enabling plugins. Before making any changes, I decided to add one more user with the form that had worked perfectly an hour ago. The user was added, but not only did I not receive a notification, it didn't even appear in WP Mail Log! I deleted the user and tried again. The notification then appeared in WP Mail Log, but still isn't received at the new account address. I tired deleting and recreating the notification again. but that didn't fix it this time.

It then occurred to me that I also have old custom post fields which are no longer used, so I found them and deleted them.

I got the same behavior as before: the first time I tried to add a user the notification didn't even show up in WP Mail Log, but on the next try it did, but still wasn't received. I then tried switching the form to redirect with a 1 second delay and the notification is now received.

None of this makes any sense. Could this be some sort of database corruption? I am at a loss as to what else it could be. And have no clue how to fix it if it is.

[Beda]

I think we should not focus on the testing site anymore as confirmed is that probably that site is at some point corrupt, given all the unexpected behaviour.

First, I would like you to connect our sandbox to your Gmail App authentication so we can try your Google Gmail APP access on the sandbox.

If that works we know the Gmail is not the issue and can focus on the live site directly, as then it must be within the form
I would then suggest a totally new form - starting simple - with no code added, and testing this.
If it works, we can add the fields and test again.
If then everything is still working fine we add plugins and/or code back in.

If you cannot make these changes to the live site directly I suggest, re-create a test site using https://toolset.com/faq/provide-supporters-copy-site/ this information.
Then deploy that copy as a new test site, to make fresh tests, given the current testing site seems to react unpredictably.

Since the very same forms and fields work on the sandbox, it could be the Gmail APP you use or anything else on the server.
There is something, that could as well be the source of this issue, which your statement that it works if there is no delay, seems to underline.
See https://toolset.com/forums/topic/cred-post-form-emails-not-sending/page/5/, in short:
It happens on Litespeed servers that have "broken connection" aborting in LiteSpeed (noabort) active, with HTTP2 devices.
This is something that can happen in Litespeed server and it's best avoided by turning off the feature.
You do use Litespeed, so that can be the case.

As a solution, you can try to disable LiteSpeed's abandoned process management by adding this rule to .htaccess:

RewriteEngine On 
RewriteRule .* - [E=noabort:1]

That might be the very reason why on our sandbox that doesn't happen since we do not use Litespeed.
Another workaround for that user was not using any redirects in Toolset Forms (keep display the form instead).
This, however, might be an unreliable workaround.

[helenmaryC]

We are thinking along the same lines.

I managed to get SMTP working on my local machine using a non-gmail server and the notification seems to be working just fine, so I agree that there is probably something about either the test site or the server itself that is causing the problem. I started to re-install the test last night and will finish up that process this morning. I will also check on Lightspeed. I know it is available, I just don't know if it is turned on.

I'll let you know what I find out.