[Resolved] Toolset Maps search not showing all results searching for certain cities

[Author]

Posts

[Nigel]

OK, I've identified the source of the problem, and have a potential fix, though I need to pass that to the developers for them to agree and to include it in a Maps release.

It does indeed relate to the address cache, which stores both an unformatted version of the address you enter, as well as a formatted version returned by Google's API.

Our code rightly avoids adding duplicates to the cache table, but it does so based on the canonical formatted address from the Google API.

It turns out that the formatted address the API returns for "Cologne, Germany" and "Köln, Alemania" is the English "Cologne, Germany" in both cases. Because Cologne was already added to the cache, when checking to see if Köln is already in the cache it thinks it is, so Köln does not get added. Therefore posts which have an address of Köln instead of Cologne are not returned in distance searches.

I can understand why the developers would have opted to test the canonical address rather than the unformatted address, but what you experience is probably an unanticipated side effect.

You can "fix" the problem by editing line 1287 of plugins/toolset-maps/includes/toolset-common-functions.php to use $address_passed instead of $address, like so:

		if ( array_key_exists( $address_passed, self::get_cached_coordinates() ) ) {

Try that on your test server (it worked on my local copy of your site).

In the meantime I have raised this with the developers for further discussion.

[simonM-5]

Hi Nigel

Thanks for that update. I have further information and evidence from our dev and www sites for you, which may be relevant also for 2nd tier:

Regarding your statement:
"It turns out that the formatted address the API returns for "Cologne, Germany" and "Köln, Alemania" is the English "Cologne, Germany" in both cases. Because Cologne was already added to the cache, when checking to see if Köln is already in the cache it thinks it is, so Köln does not get added. Therefore posts which have an address of Köln instead of Cologne are not returned in distance searches."

In our production cache, both "Cologne, Germany" and "Köln, Deutschland" exist with the same coordinates.

I can find several instances on our dev and production instances, where the same address coordinates ARE stored multiple times in the cache, both for the same language and differing langauges, all with different "Addresses":

Example screenshot Frankfurt from our dev site, identical coords stored 5 times:
- 3 entries for identical coords from browser in English, ("Frankfurt am Main, Germany", "Frankfurt am Main, Germany, Germany", "Frankfurt am Main, Germany")
- 1 entry for identical coords from browser in German ("Frankfurt am Main, Deutschland")
- 1 entry where it is not clear from which browser language it originated ("Frankfurt")

Example screenshot Munich from our dev site, identical coords stored 2 times:
- 1 entry for identical coords from browser in German ("80797 München, Deutschland")
- 1 entry for identical coords from browser in English ("80797 Munich, Germany")

Example screenshot Bonn from our production site, identical coords stored 3 times:
- 1 entry for identical coords from browser in German ("Bonn, Deutschland")
- 1 entry for identical coords from browser in English ("Bonn, Germany")
- 1 entry for identical coords from browser in French ("Bonn, Allemagne") (which proves that the browser language (French) was being used, since we only offer our site in English and German)

So by that logic, your statement "Because Cologne was already added to the cache, when checking to see if Köln is already in the cache it thinks it is, so Köln does not get added" doesn't seem to hold true in all cases, right? B

This leads me to believe the problem may be due to something else (at least in part).

Hope this helps.

Kind regards
Simon

[Nigel]

Interesting. On my copy of the test site (and on the test site) working with the test data, the search was failing because Köln was not being added to the address cache, because of the code I suggested an edit of, but I see from your screenshots other examples where alternate versions of the canonical addresses are included in the address cache.

I need to broaden the tests with actual data, then.

Could you export just the database of a server which has data I can use for testing searches based on locations such as Frankfurt and Munich which have multiple entries in the address cache?

You can share a link to the package via dropbox or wetransfer or something similar, the link will automatically be hidden.

[simonM-5]

HI Nigel

Here's a copy of our production database, there are several examples there of the same address/coordinates in different languages.

hidden link

Thanks and regards
Simon

[Nigel]

Importing that has added content (Nanny Ads, etc.) but broken the Views, which can neither be edited nor displayed, and I can't continue with that test site any longer. There were evidently some important differences between the production server and the test server.

Sorry, but could I get access to a dev server where the search problem can be reproduced, and I'll take a backup myself, to try again.

I'll set a private reply to get the credentials of that server.

[simonM-5]

Hi Nigel

Here is access to our dev and production servers. Both servers have
- WP Adminer in the toolbar, so you can query the database directly.
- Duplicator Pro so you can take copies and download them yourself if you prefer (please either delete the packages once downloaded via FTP or shoot me a message and I will take care of that for you),
and I'm giving you full FTP access too:

Kind regards
Simon

[Nigel]

Thank you Simon

I have the working copy of the production server installed locally.

Can you give me a concrete example of something to test which you know is not working?

(Referring back to your first post in this thread, I was checking the Nanny Ad with post ID 25734, for example, but the post with that ID isn't a Nanny Ad, it is an attachment post.)

If you can share one or more examples of something that you know does not work I will test with those.

[simonM-5]

HI Nigel

We have been trying to "correct" the Ads as they come in by entering the locations in German or English, but we managed to find at least one example of a Nanny Ad which doesn't appear to be working:

Post ID 31486 created in EN with a DE duplicate.
1) Searching for “Stuttgart” or “Stuttgart, Germany” or “Stuttgart, Deutschland” doesn’t find her (Célia)
Example:
hidden link

2) But just selecting NATIVE LANGUAGE “French” finds the Ad OK (9th card in the list)
hidden link

3) Searching for French and with location Stuttgart, Germany doesn't find her either.
hidden link

My colleague is trying to find you another couple of examples, but I hope this is enough for the meantime for you to make progress (screenshot shows the Ad you're looking for). Both Views "Find a Native Nanny Search and Results View" for logged in users and "Find a Native Nanny Guest Search and Results View" for logged out users should be able to find this ad.

Thanks!

Kind regards
Simon

[simonM-5]

HI Nigel

Found another example:

Nanny ID 30687 Title: Available nanny in Kreuzberg – Portuguese, English and Spanish spoken. (see screenshot)

Found if searching NATIVE LANGUAGE = Portuguese, but if you add Berlin to the search it's not found. Also not found searching "Berlin" on its own. User's location is entered in Portuguese: "Kreuzberg, Berlin, Alemanha"

Kind regards
Simon

[Nigel]

I checked your 2 examples.

The first one I don't see the same on my local copy of your production server, the French-speaking nanny in Stuttgart is returned in the results (screenshot). I'll double-check the production server to see if I can spot any differences.

Regarding the second example, the Portuguese-speaking nanny in Kreuzberg, when I check the address cache her address ("Kreuzberg, Berlin, Alemanha") is missing, it only contains "Kreuzberg, Berlin, Alemanha".

So that looks like the problem I found earlier and shared a fix for. Did you implement that?

[Nigel]

That's also the same issue that should be fixed with the change I suggested.

There is no entry in the cache for the address "Stuttgart, Allemagne" which is what is stored in the post meta for the nanny ad with id = 31486.

[simonM-5]

Hi Nigel

To your first update, Nanny ID 31486 French speaking in Stuttgart:

1) Unfortunately, I am unable to reproduce/confirn finding that Nanny in Stuttgart on our Production server, either for logged in or for guest users (see screenshots):

a) Using logged-in user nativefamily6:

hidden link

b) Using Guest user:
hidden link

2) Removing the French option only leaves 7 nannies found in Stuttgart, none of which has NATIVE LANGUAGE = French, and French is not available to choose, since the query result already excludes that 8th nanny.

3) Even searching for "Stuttgart, Allemagne" does not find the 8th nanny in Stuttgart (shows only results 1 - 7 of 7)

So therefore I'm curious how your result set is showing this nanny and we cannot reproduce this on production?!?

Checking the cache table directly for Stuttgart I'm seeing the same 8 entries.:

SELECT *, AsWKT(`point`) AS `point`
FROM `wp_toolset_maps_address_cache`
WHERE `address_passed` like '%Stuttgart%'

In my eyes however, the bottom two entries you have highlighted in message

https://toolset.com/forums/topic/toolset-maps-search-not-showing-all-results-searching-for-certain-cities/page/2/#post-2567631

are ALREADY duplicates, since they reference exactly the same point on the map. So no matter whether anyone searches "Stuttgart", "Stuttgart, Germany", "Stuttgart, Allemagne", "Stuttgart, Almanya", "Stuttgart, Alemania" etc, it should always compare the COORDINATES returned by Google Maps API, ie POINT(9.182932 48.775846), to what is in the cache table for that point. Or am I missing something completely obvious here? In fact, the address and address_passed columns are in fact technically redundant, except for human-readability purposes, are they not?

Referring to your message:
https://toolset.com/forums/topic/toolset-maps-search-not-showing-all-results-searching-for-certain-cities/page/2/#post-2565345

"It turns out that the formatted address the API returns for "Cologne, Germany" and "Köln, Alemania" is the English "Cologne, Germany" in both cases. Because Cologne was already added to the cache, when checking to see if Köln is already in the cache it thinks it is, so Köln does not get added."

I can find multiple counter examples in production, where English and non-English values are stored in both columns address and address_passed for the same point on the map.

Example Berlin:

SELECT *, AsWKT(`point`) AS `point` 
FROM `wp_toolset_maps_address_cache` 
WHERE `address_passed` like '%Berlin%'

returns
12049 Berlin, Deutschland
Alt-Tegel, 13507 Berlin, Deutschland .. and several more

Example Hamburg

SELECT *, AsWKT(`point`) AS `point` 
FROM `wp_toolset_maps_address_cache` 
WHERE `address_passed` like '%Hamburg%'

returns
20251 Hamburg, Deutschland

and several more examples with Munich. Germany/München, Deutschland

Sorry again for the long-winded message, but I want to be sure how Toolset Maps is working before I modify any of the plugins manually...

Kind regards
Simon

[Nigel]

Sorry Simon, I'm confident that the fix I proposed would resolve the issue, and if it doesn't I'll be happy to look into this further.

Checking the internal ticket a developer has already updated Maps to fix this problem, and the fix will be included in the next plugin update (which I expect within a few weeks at the most).

I'll leave this ticket marked as escalated, and then when the release is available I will let you know.

The nature of the fix means that it should be effective straight away, you shouldn't need to take any special steps for it to work with your front-end searches.

[simonM-5]

Hi Nigel

I implemented your workaround on dev and www.

On dev, we now see the "Cologne" and "Köln" ads.

But on www, we are still not seeing that 8th French speaking nanny from Stuttgart.

Kind regards
Simon

[Nigel]

This is the nanny you are referring to? Or someone else?

(That screenshot is from your production server, the guest user search page.)