[Escalated to 2nd Tier] I am having repeated JSON response errors when using Toolset.

[Author]

Posts

[peterS-27]

This is a repeat of the issue worked on here: https://toolset.com/forums/topic/wpmu-dev-hosting-web-application-firewall-json-errors/

[Christian Cox]

Hi, I've reopened the original ticket so Jamal can continue assisting you, but he typically does not work Wednesdays or Thursdays. If I understand correctly, Jamal provided an update to .htaccess that solved the problem on the cloned site. Did the problem begin appearing again there, or is the solution provided in the cloned environment not working on the live site? Can you explain a bit more about what happened between the time that ticket was closed and the time you opened the new ticket?

[peterS-27]

I added the code to .htaccess and that appeared to resolve the problem. But today, the problem reappeared. The only substantial change I made (about a week ago) was downgrading the hosting package to a less powerful/large system. But the site seemed to work fine for a few days after I made that change.

[Christian Cox]

It's possible the .htaccess file was modified by the hosting company or a 3rd-party plugin or server process without your knowledge. Have you checked the contents of the .htaccess file since the problem began? Can you download that file with FTP and verify its contents are identical to those recommended before?

[peterS-27]

Hi Christian,

Yes, the .htaccess file still has the code recommended. It looks like this:

# BEGIN WordPress

RewriteEngine On
RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]

# END WordPress

## WP Defender - Prevent information disclosure ##

<FilesMatch "\.(txt|md|exe|sh|bak|inc|pot|po|mo|log|sql)$">
Order allow,deny
Deny from all
</FilesMatch>

<Files robots.txt>
Allow from all
</Files>

<Files ads.txt>
Allow from all
</Files>

## WP Defender - End ##

[Christian Cox]

Okay I tried checking the cloudways site from the other ticket, but it does not appear to be online. I get a 404 error here: hidden link

May I log in to wp-admin and see the problem? Please provide login credentials in the private reply fields here and I will take a closer look. Do I need to try to save a change in some post, like adding a space at the end of a paragraph or some minor text change?

[Christian Cox]

Okay unfortunately I was unable to replicate the JSON errors in a clone of the site on my local environment, but I saw a couple of issues on the Archive of Human Genius page, so I was hoping that resolving those issues on the live site would solve the JSON problem there.

First, I noticed that the sorting control for Post Date did not have a text label, and that was causing a PHP notice generated in the back-end. I added a text label "Date" to this sorting control on the live site to solve the problem for now.

Then I also noticed on my local environment a problem with the featured image block attempting to load an invalid URL, so I recreated the block on my local environment and that solved the image error in my local site. I recreated the featured image block on the live site to resolve the same problem, but the issue is not completely resolved on the live site. It's not causing any visible problems on the front-end or in the editor, but I can see it trying to load an invalid URL in the background.

On the other hand, now I'm not seeing JSON errors in the page editor here:
hidden link

So those two issues seem to be at least partially responsible for the problem editing the Archive of Human Genius page.

However, the site is painfully slow whenever I try to edit this page. It takes over 2 minutes to load the page editor. To try to speed up performance on the front-end and in the backend editor, I have updated the featured image to load a smaller size image rather than rescaling the full-size image. I've also updated the pagination settings to prevent preloading and precaching pages. On sites that have slower response times for AJAX, I've seen this speed up performance considerably, and I see the same results here on the front-end. Results load in much faster as you scroll through content. Also since infinite scrolling is implemented in this View, pagination controls don't really make sense here. I've removed them for now.

As far as the JSON error on this page:
hidden link

I've investigated and found a security issue is blocking access to one of the assets loaded by the editor. The URL of the asset is:
hidden link
See the screenshot attached here for a technical reference. It returns a 403 error, which indicates a security plugin or system on the server blocked access to this API. Normally this indicates a server-side problem that you need your host or security system plugin support team to help resolve. Feel free to pass along this screenshot when explaining the problem. If you need anything from our side to help facilitate that conversation, let me know, but I suspect this is not directly related to Toolset. As a test, you could temporarily deactivate all Toolset plugins and try to edit the page again. You could activate a maintenance mode plugin during testing if you want to block site access during the test. My guess is the problem will return when editing the post, even without Toolset active.

[peterS-27]

Thanks for all the time you put into this Christian! Unfortunately, the problem is still occurring on the Archive page. Is it possible Toolset just can't handle databases this large? It has never quite worked as I'd hoped since I purchased it.

A few things:

Yes, the host (WPMU DEV) has a Web Application Firewall that seems to be interfering. What I've asked for from Toolset previously is IPs or other information that we can exclude from the WAF. See the screenshots. Can we add anything to the allow IPs or URL allowlist from Toolset?

This is a different issue, but the sorting dropdowns are quite strange to me. The label itself is a dropdown with only one option. Doesn't really make sense from a UX perspective. (That's why the label field was blank.)

[peterS-27]

Hi Christian,

Thanks for all the time you've put into this! Really appreciate. Unfortunately, the issue is still occurring on the back end of the Archive page.

Some notes:

The host (WPMU DEV) did mentioned that their Web Application Firewall (WAF) is interfering with Toolset. We went through and tried to "allow" several IPs, URLs, and rules. Are there any more we can add so that the WAH bypasses Toolset?

This is a separate issue, but the label for the sorting dropdowns is strange to me. It basically creates another dropdown with only one entry for the label in addition to the action sorting dropdown itself. I don't really understand this. Can it just be pure text?

[Christian Cox]

Unfortunately, the problem is still occurring on the Archive page.
I don't see the JSON errors when I load either page now in the editor. Maybe your browser is holding onto an older version - can you clear your cache? Are you editing something specific to trigger the message? Can you tell me step-by-step how to make the error message appear?

Is it possible Toolset just can't handle databases this large? It has never quite worked as I'd hoped since I purchased it.
I don't think database size is the issue here, I think the JSON error message is related to server-side security blocking access to specific assets. It is more of an environment issue than a content issue. The block editor expects to receive JSON data when it tries to load some data, but the server is delivering 403 security error messages instead of the data. Those error messages were not in JSON format, so that's why the errors were displayed. It had nothing to do with the database size as far as I can tell. That's why I'm not able to replicate the same issues in my local environment with a clone of the site. If you'd like, I can throw the clone of the site up on a sandbox environment where you can test it out. I think you'll find it is much faster and more reliable in a different environment.

This is a different issue, but the sorting dropdowns are quite strange to me. The label itself is a dropdown with only one option. Doesn't really make sense from a UX perspective. (That's why the label field was blank.)
Sure, when the sorting controls are set up to allow multiple "order by" sorting options, the dropdown makes more sense. In your case the only sorting option is post date, so the "order by" dropdown isn't really intuitive. A simple solution would be to hide the first select field and only display the second dropdown for choosing sort order. There is no built-in way to turn off that dropdown, so you can hide it with CSS. Add this custom CSS snippet in the top-level View block to hide the "order by" dropdown and display the sort order dropdown by itself instead:

.wpv-sorting-block-item.wpv-sorting-block-orderby {
 display:none;
}

There is a bug in Toolset related to that sorting control label field when it is empty. A PHP Notice is being generated by Toolset when the label field is completely empty. I was under the impression that this could be causing problems in the JSON responses for the archive page assets, so I added some text to bypass that problem temporarily during testing. If you're still seeing the JSON error message it must be unrelated. Regardless, the notice should be fixed on our end. I've escalated the issue to my 2nd tier support team for investigation, but I suspect a workaround will be needed for the near future until that issue can be solved in an upcoming release. Anything other than an empty label will be fine, including a blank space character.

Can we add anything to the allow IPs or URL allowlist from Toolset?
The problem I'm seeing is unrelated to external URLs or IPs being blocked. The problem I'm seeing is internal URLs being blocked with 403 errors. See the screenshot here, the full path of the blocked asset shown in the screenshot is:
hidden link
That's a file on your server, not an external server.

So I don't know if the Firewall access tool is going to be helpful - if I understand it correctly Firewalls are designed to manage access to/from external IPs and external URLs. But it can't hurt to try 🙂 If this tool allows URL wildcards you could try explictly allowing all JSON data URLs, which are sometimes throwing security errors in wp-admin.
If relative paths are required, the relative path with wildcard is:
/wp-json/*
If absolute paths are required, the absolute path with wildcard is:
hidden link
Without wildcards, I'm not sure this will work, because the URLs vary per post. When editing post 53633, the autosave URL is
hidden link
But when editing post 104327, the autosave URL is
hidden link
It's dynamic, so adding a URL per post isn't realistic. That's where the wildcard comes in, hopefully it will open up access to all data URLs under the wp-json directory.

[peterS-27]

Hi Christian,

I can't seem to find you last message here. But the issue seems to be resolved for now. Thanks! I'll let you know if it pops back up again. Perhaps keep this thread open for a week or so in case it comes back?

Peter

[Christian Cox]

Sure, I'll check back in a couple of days for an update.

[peterS-27]

Hi Christian,

Unfortunately, this error popped up again on the homepage. See attached screenshot.

Can you assist?
Peter

[Christian Cox]

Okay checking that now, and I see several errors in the JavaScript console periodically when the editor is loading data. That causes a JSON error, which you are seeing in the editor. Here is an example of the error message I see:
-----------------------------------------------------------------------------------------
Error 524
Ray ID: 635ab82b19f8b727 • 2021-03-25 20:01:03 UTC
A timeout occurred
You
Browser
Working
Jacksonville
Cloudflare
Working
k*******.**z
Host
Error
What happened?
The origin web server timed out responding to this request.

What can I do?
If you're a visitor of this website:
Please try again in a few minutes.

If you're the owner of this website:
The connection to the origin web server was made, but the origin web server timed out before responding. The likely cause is an overloaded background task, database or application, stressing the resources on your web server. To resolve, please work with your hosting provider or web development team to free up resources for your database or overloaded application. Additional troubleshooting information here.

Cloudflare Ray ID: 635ab82b19f8b727 • Your IP: 2601:742:200:e7a:4120:5ca9:1323:9692 • Performance & security by Cloudflare
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Instead of serving up data, the server is responding with these error messages saying that the server is overloaded...that's why we're seeing the issue sporadically and in different areas. Right now I'm not seeing the error messages but a few minutes ago I was, so it indicates the server is getting overloaded when traffic is higher. It's incredibly slow and frustrating - I don't know how you tolerate it when editing in wp-admin.

Have you been able to get any feedback from your hosting company about these security errors and general slowness? Do you want me to try copying the site over to cloudways again to give you a good speed and reliability comparison?

[peterS-27]

I went to the host WPMU DEV first. After attempting several fixes, they told me to go to Toolset. That is how this conversation between me and Toolset started weeks ago now. This problem has only ever come up once Toolset was uploaded and activated and on pages with Toolset views.

I don't want a speed and reliability comparison. I want to figure out why this issue keeps popping up and seems to be related strictly to Toolset.

Can you help with that?