[Resolved] WP Allexport/import-Toolset has big bugs? On Post-Export, -Import + -Update

[Author]

Posts

[Fatih]

We are using Toolset for a schooling-course-project, it works well since nearly 2 years now. After reading in your blog, that WP Allimport/export finally supports toolset with a paid addon-plugin, we purchased the Pro. We were hoping that there is finally a way to bulk-update existing posts and create new posts in a much faster way. After 2 days of struggle, trying to get it work, i am pretty sure that i was thinking too naiv.

We have 3 Issues and i hope you can help.

Issue 1: Missing Reference fields completely on export-settings, failing import (empty field in imported posts) of all reference fields

There are 3 Taxonomys:
1. Seminare (courses)
2. Seminar-Orte (course-locations)
3. Seminar-Termine (course-events/dates)

"Seminar-Termine" has 2 Post Reference Fields for creating a many to one connection, fetching data of courses and locations.
1. Reference Field: "Seminare" (Name of a course)
2. Reference Field: "Seminar-Orte" (Name of location)

The Export Options of WP Allexport for "Seminar-Termine" shows all fields of our fieldgroup (except they are empty in all posts) but it does not show both Reference Fields "Serminar-Orte" and "Seminare".

I tried a lot but it seems like a bug for me, because on "WP Allimport" the field shows up as expected in the Import-Settings.

I tried to import a CSV with manual data for the Reference field "Seminar-Orte" of but it did not work (this could be, because i don't know wich value it expects).
I already wrote to the support of WP Allexport, i hope they can help. I don't know how involved Toolset is in the Toolset Extension of WP Allimport, can you confirm my issue or tell me if i can do something to make it work?

Issue 2: Unix Timestamp converting on export works, convert back on import fails (empty)

We use Date / Time fields for the beginning and the end of our course-events in "Seminar-Termine". This is saving Unix-Timestamps. Since we purchased WP-Allimport/export to be able to bulk edit "Seminar-Termine" in Excel, i found out how to convert the Timestamp in WP Allexport with PHP before the Export with PHP (return date("d.m.Y H:i", $timesst);). This works fine and the Time is human-readable in Excel.
I was thinking, that it will be easy to convert back the time with [strtotime({wpcfbeginn[1]})] in WP Allimport Import Settings for the field but it does not work, the string is empty. Is there a mistake in the syntax i am missing? In PHP strtotime('01.12.2020 19:00') works as expected. The Documentation of WP Allimport did not help.

Issue 3: Updating existing Posts – works but for every updated post a new one is pushlished (same content, different ID)

I exportet "Seminar-Termine" as CSV with the column ID (post-id). The values are right.
I modified the CSV to contain only two rows, first row with an ID of an existing post, second row i did not fill the ID-field.
I was hoping that importing the first row will update the existing post and the second row will create a new post. Of course i did not fill fields for slug an permalink in both rows.
While importing with WP Allimport, I checked the option to update fields of existing posts with same ID.
The result was strange, the import did update the existing post but it also created a new post with the data of the first row. The second row did create a post as expected.

My Question: Is the "Update" function in WP Allimport not working well with Toolset? Is this also a bug or a known issue? How can i avoid this strange behavior?

Toolset, WordPress and WP Allimport/export/toolset plugins are up to date (23.01.2021).

[Christian Cox]

Hi, I'll be glad to take a closer look and see what we can do on the Toolset end to make the export/import process run smoothly, but since a 3rd-party plugin is involved our influence is somewhat limited. See my feedback below.

There are 3 Taxonomys:
1. Seminare (courses)
2. Seminar-Orte (course-locations)
3. Seminar-Termine (course-events/dates)
I assume these are actually custom post types, not taxonomies.

I tried to import a CSV with manual data for the Reference field "Seminar-Orte" of but it did not work (this could be, because i don't know wich value it expects).
Post reference fields are technically implemented as one-to-one post relationships, and must be handled like exporting/importing post relationships rather than like exporting/importing custom fields because of this technical difference. After importing the posts, go to Toolset > Export/Import and click the Associations tab. If the post relationships/post reference fields were included in the import, you should see a prompt here to run the automatic import process. That will take the imported information and convert it into usable post relationships/reference field values. If that tab shows no relationships to import, I would need to see the original export CSV document and I would need to know the reference field slugs to determine whether or not the correct information was included in the export process. Can you provide that information for me to review?

I was thinking, that it will be easy to convert back the time with [strtotime({wpcfbeginn[1]})] in WP Allimport Import Settings for the field but it does not work, the string is empty. Is there a mistake in the syntax i am missing?
Conversion of data formats like this should be handled by the import tool, since Toolset's date custom field expects a Unix timestamp and provides no internal conversion processes. If you're trying to import anything other than a Unix timestamp format here, some data conversion must be applied. The syntax of that import tool's conversion process is something you might need to ask their support team about, since their plugin is responsible for performing any conversion during the import. Since you used the "return" operator when exporting data, maybe the "return" operator is needed here as well?

[return strtotime({wpcfbeginn[1]})]

That would be my first guess, but again I'm not trained on their plugin so I'm not the best source of information about that. I'm also not completely clear about why the conversion is there in the first place, except to make the CSV output more human-readable. If you're not manipulating that value directly in the exported CSV, perhaps removing the conversion altogether will resolve this issue? Or if you want a more human-readable format, perhaps you could add a duplicate column in the CSV for display only, where the converted date is visible, and keep the original value in Unix format in the original field column? As long as you're not manipulating that date value directly, it seems that would help circumvent the conversion issue. If you need to manipulate the date value in the CSV document directly as a human-readable date, then conversion would obviously be necessary during the import process.

The result was strange, the import did update the existing post but it also created a new post with the data of the first row. The second row did create a post as expected.
Do you see a duplicate post in wp-admin, or are you referencing the wp_posts table in the database directly? If you're looking directly at the database, then it could be that you're seeing a row added for the post revision, not a duplicate post per se. I would need to see the db to be sure, but you should be able to tell by examining the post_type column in the additional row. If the value is "revision", then the older version of the post is saved in the database as a revision that can be restored at a later date if necessary.

My Question: Is the "Update" function in WP Allimport not working well with Toolset? Is this also a bug or a known issue? How can i avoid this strange behavior?
I'm not aware of any outstanding issues in the Toolset side, but we don't track issues in the WP Allimport plugin so if there is an issue on their end I wouldn't be immediately aware. I ran a quick test on my local environment with a similar, simplified scenario:
- I installed Types and WP All Import Pro, WP All Export Pro, and the WP All Import Toolset Add-on
- I created a custom post type "Books" and published one Book post
- I exported all Book posts into a CSV file format with post ID, title and contents using WP All Export Pro (see here: hidden link)
- I modified the post contents of the existing Book post so I could see if the update processed successfully, and I added one row to the produced CSV and included no post ID to see if the new content was created successfully (see here: hidden link)
- I imported the CSV contents using WP Allimport and configured the importer to update existing items (configurations screenshots attached)
- The existing Book post was modified and the new Book post was created, but there was no duplicate of the original post created. (wp-admin screenshot attached showing only 2 Book posts).

So perhaps there was a configuration issue in the import process? Or something else I overlooked? If you have a backup of the original site database, and a copy of the import CSV file, I can take a closer look to determine why the duplicate post was created.

[Christian Cox]

Also, I did a quick search for date format conversion and found an example here: https://stackoverflow.com/questions/64254986/wp-all-import-pro-function-to-change-date-and-time-format-from-imported-csv
It looks like this person created a custom PHP function that converts the date format, so perhaps a similar approach is needed for your case? PHP code:

function my_fix_date( $date, $format = 'Ymd', $new_format = 'Y/m/d' ) {
   if ( $obj = DateTime::createFromFormat( $format, $date ) ) {
    return $obj->format( $new_format ); 
  } else {
      return null;
  }
}

Then in the import settings for this field:

[my_fix_date({wpcfbeginn[1]},"d.m.Y H:i","U")]

...where "d.m.Y H:i" is the converted date format and "U" is the desired output format of Unix timestamp.

[Fatih]

Hi Christian, thank you for your efforts on a sunday(!). Great support as always.

You are right, i meant custom post types.

Did you also try to export a reference field? This kind of field does not get recognized in the settings. I went so far trying to fetch field contents with a php function on Export in WP Allexport, no chance. Spent 2-3 hours trying, every other wpcf field works. So the Export File definetly can‘t contain the field value for a reference field, there are Options to Import in recognized reference fields in WP Allimport, so i guess someone just forgot to implement support for reference fields in WP Allexport Toolset plugin? Perhaps someone was thinking that this is the same like relationship field, it confused me also at first.

Could you please check if you are able to export Reference fields at all with WP Allexport?

I will try the date conversion, Thank you!

[Christian Cox]

No, as far as I know the WP All Export plugin does not recognize post relationships or reference fields for export by default, though the add-on enables importing them. When exporting data using the standard WordPress export tool, the post reference fields will be included in the generated XML document as postmeta values using a specific format meta key like "_toolset_associations_{reference-field-slug}", which will then be recognized during the import as a post relationship and shown in the Toolset > Import Associations tab. Then you can run the automatic conversion process to convert those into usable post reference fields.

[Fatih]

Thank you for your quick answer, this makes this tool completely useless for toolset users using the reference field ... one of the superpowers of toolset. A real fail purchase for me, too. I would mention this in the toolset blog, this will warn customers of toolset and beware them from loosing time and money ... I really did not think that would have to check every supportet toolset field when the article in toolset blog is praising compatibility with a special paid plugin. Makes me feel upset a little bit, just waiting for the reply of WP Allexport Support. Thank you again for your quick response.

[Fatih]

There is no documentation about the Data format and type the reference field expects with the WP Allimport. Will the ID of the referenced Object work? or the slug? What will work, do you know more?

[Christian Cox]

Usually it's best to use the post's GUID. You can use the post title, but since titles aren't guaranteed to be unique the GUID field is best. When exporting using the standard WordPress export tool, the GUID looks like this:

<guid isPermaLink="false"><em><u>hidden link</u></em>;

Or if you're looking at the original site's database, there is a column called 'guid' in the posts table.

[Fatih]

Thank you, i wanted to have clean regular export as csv and a simple predefined import in a hotfolder with cronjob to update and add courses. This should happen every week, a wordpress export would not help with this.

is the guid also the permalink?
is it possible to get it with php? i could add a function and custom column to make it work as it should when this is possible ...

[Christian Cox]

is the guid also the permalink?
It might be, but the permalink is not always identical to the guid so you can't count on them being the same. From WP docs:
"The guid will appear to be a link, but should not be used as an[sic] link to the post."

is it possible to get it with php?
Yes, there is a WordPress function available get_the_guid that will give you the GUID for any known post ID:
https://developer.wordpress.org/reference/functions/get_the_guid/

[Fatih]

Thank you Christian,

$bez1 = ID-field of current row in export
Fieldname of Reference-field = seminartitel

with this export function i should get the guid of the reference field:

function seminartitel_guid($bez1){
$seminartitel_id = get_post_meta($bez1, "wpcf_seminartitel", true); //this string is empty
$seminarguid = get_the_guid( $seminartitel_id );
return $seminarguid;
}

How can i get the ID of the referenced post? Solving this will sure solve my Issue

[Christian Cox]

You have a couple of options here. One option is to use the Types Field API types_render_field:

function seminartitel_guid($bez1){
// assuming $bez1 is the ID of the post containing the reference field
$seminartitel_id = types_render_field('seminartitel', array('item' => $bez1));

https://toolset.com/documentation/customizing-sites-using-php/functions/

Another option is to use the post relationships API toolset_get_related_post, since reference fields are technically implemented as post relationships:

function seminartitel_guid($bez1){
// assuming $bez1 is the ID of the post containing the reference field
$seminartitel_id = toolset_get_related_post( $post_id, 'seminartitel', 'parent');

https://toolset.com/documentation/customizing-sites-using-php/post-relationships-api/#toolset_get_related_post

Either option should be okay here.

[Fatih]

i tried the second method, toolset functions do not work in WP Allimport/export function fields ...
did you try?

[Christian Cox]

i tried the second method, toolset functions do not work in WP Allimport/export function fields ...
did you try?
I assumed either would work well, but I just ran an export to test the types_render_field function and it seems to work as expected for me. I will try the post relationship method and double check, just a moment.

[Christian Cox]

Yes, both methods work for me in an export. I'm attaching some screenshots here for you to review, maybe you can spot a typo in the function editor? Note that my reference field is "book-ref-1", where yours would be "seminartitel" if I understand your setup correctly.