[Resolved] Importing posts with relationships using intermediary field

[Author]

Posts

[andyS-5]

Hi Christian,

I've readdressed how this will work and decided to remove the Qualifications Suite as a post type and add it as a taxonomy.

So, I can run the import successfully, BUT (there's always a 'but) the relationship for the multi to multi Linked Qualification to Unit combination (now with no post intermediary) is adding a related post "Adde" for each cell in the CSV (changed to CSV for importing) that doesn't have a value. There are now, on the site, exactly 14 related posts for each Unit Combination (as far as I have checked), which exactly matched the number of columns in the CSV to cater for the maximum number of Linked Qualifications.

Long story short - is there a way to prevent the import trying to add a relationship when one doesn't exist?

Cheers
Andy

[andyS-5]

In addition, and rather bizarrely, those Unit Combinations with 14 Linked Qualifications (ie the maximum) aren't importing any Unit Combination/Linked Qualification relationships!

So, as far as I can see, it's working as desired, except for these two slight issues...

[Christian Cox]

I'm not seeing the same behavior in my local tests, but I'm not sure I understand the full process you have in place, which posts have already been imported, how you have configured the relationships fields in the import screen, and so forth. I take it you have set up a staging environment where you run the import, then revert to a prior state to test again? Is it possible for you to provide a clone of the site's prior state, just before the import? I can run through the same steps and try to produce the desired results.

[andyS-5]

What's the best way to share the login credentials with you Christian?

[Christian Cox]

Hi, private reply fields are available here.

[Christian Cox]

I don't have the site before the changes, but the current version has now three CPTs - Units, Linked Qualifications and Unit Combinations (the latter connects the first two and is the CPT we'll be using for display on the site).
You're saying there's no backup of any kind from before the import? If not, then we need to start digging in the database to solve this. I doubt that reimporting will ever fix whatever corruption has occurred here. Since this is a multisite, that makes things more complicated. I have access to Duplicator Pro, which is pretty good at cloning and migrating multisite instances. If it's okay for me to install that on the network level, I can try to get a clone of the development site.

[andyS-5]

Hi Christian,

This is a backup - hidden link - from Nov 19th - prior to the big import. I had started working on the new CPTs, but hadn't run any imports other than a taxonomy import. All the same login credentials.

I don't know if this is any use to you!

Feel free to add Duplicator Pro if you wish... I'm just really happy you're helping me 🙂

Cheers
Andy

[Christian Cox]

Okay I've downloaded a clone of the pre-import site and begun running some tests.
- I installed the wpai-toolset-types-addon-b2.zip plugin.
- The site is still using the old relationships system, so I ran the migration tool in Toolset > Relationships.
- Created a database-only backup.
- I copied data from the master spreadsheet and created a new spreadsheet containing only the Linked Qualifications in a single column. Then I used the spreadsheet tools to filter out all the duplicates and empty rows. This left me with 97 Linked Qualifications, so I saved a new CSV with just these Linked Qualifications and a header cell in a single column. The header cell has just the text "LQs," which WP All Import can use as the column name.
- Uploaded that new CSV with WP All Import Pro, and imported all 97 Linked Qualifications into my cloned site.
- Created a DB-only backup again.
- I copied data from the master spreadsheet and created a new spreadsheet containing only the Laser Unit Codes (Units) in a single column. Then I used the spreadsheet tools to filter out all the duplicates. And created a header cell with the text "Units". This left me with 1815 unique Units, so I saved a new CSV.
- Using the new CSV, I imported all 1815 Units into my cloned site.
- I went to Toolset > Post Types and created a new CPT Unit Combinations.
- I went to Toolset > Relationships and created a one-to-many relationship between Units (one) and Unit Combinations (many).
The system generated this relationship with the slug unit-unit-combination. Your relationship slug may vary.
- Then I created a many-to-many relationship between Unit Combinations and Linked Qualifications. Again, the system generated slug on my side is unit-combination-linked-qualification but yours may vary.
- Created a DB-only backup again.
- Then I uploaded the original master CSV file to create new Unit Combination posts.
- I set the title field as these identifiers with no separator character (see rels.png):

{laserunitcode[1]} + ( {qualificationsuite[1]} )

- For the UC / LQ relationships, I expanded the input field and inserted the following columns one per line with no separator character (again, see rels.png):

{linkedqualification1[1]}
{linkedqualification2[1]}
{linkedqualification3[1]}
{linkedqualification4[1]}
{linkedqualification5[1]}
{linkedqualification6[1]}
{linkedqualification7[1]}
{linkedqualification8[1]}
{linkedqualification9[1]}
{linkedqualification10[1]}
{linkedqualification11[1]}
{linkedqualification12[1]}
{linkedqualification13[1]}
{linkedqualification14[1]}

- I set the unique identifier for the UC posts to be {laserunitcode[1]}-{qualificationsuite[1]} which should be a unique combination of strings, even though some Unit codes repeat. See uniqueid.png.

Then I let the import run for 2021 records. Unfortunately the associations between the Unit Combination and Linked Qualifications were not established. I ran out of time to finish this today so I'll be coming back to it tomorrow. I'll keep you posted!

[Christian Cox]

I've asked my 2nd tier support team to take a closer look. I've had trouble getting all the relationships imported, and I'm not sure if there's an issue in the software or if it's a configuration issue. I'll let you know what I find out as soon as I can.

[Christian Cox]

Hi, just a quick update to let you know our 2nd tier support team is investigating this in more detail, and I will keep you posted as I receive feedback.

[andyS-5]

Sorry for not responding to your earlier email Christian. Been a bit hectic.

Thanks for the update. I've let my client know that any resolution to this will be after the new year 🙂 Wishing you all the best - hope you've some time off!

[Christian Cox]

Hi, I have a quick update for you. My 2nd tier support team has worked with our developers, and unfortunately we have been unable to import the posts with relationships as expected. Given the information we have from the WP All Import team and the documentation provided, it seems that the CSV file is setup correctly and it seems like we are following the directions. However, the import tool is not creating the necessary post relationships during the import process. Our recommendation is to reach out to the import plugin authors to see if they have any additional advice for using their product. If they are having trouble using our APIs, we encourage them to contact us directly.

One note from my 2nd tier support team that may be of value to the import plugin authors:

The first thing to check is \wpai-toolset-types-addon-b2\libraries\toolset-relationships\relationship.php function saveRelationshipToDb() and checkCardinality()

If there is any Toolset information you need to facilitate discussing the problem with their authors, please let me know and I'll do my best to get that information from our team.

[andyS-5]

Thanks for your help Christian - I'll reach out to the WPAllImport team and let you know...

[Christian Cox]

That sounds good, I'll stand by for your input.

[andyS-5]

Hi Christian,

Just want to let you know that I've sorted things out with help from WPAllImport. Thanks for all your help - really appreciated! Below is what they said for your reference.

Thanks
Andy

There are a few problems:

The delimiter for the relationship posts is set to comma, but there are commas in some of your titles, e.g. "LASER Entry Level Award for Learning, Employability and Progression (Entry 3)".

Some of the linked qualifications are missing a space before the open parenthesis, e.g. "LASER Entry Level Certificate in Personal Progress(Entry 1)". Outputting empty titles/slugs to look for was causing an issue.

To remedy these, I did the following:

1. Change WP All Import's multi-glue character from comma to the pipe symbol by saving this in the Function Editor:

add_filter( 'wp_all_import_multi_glue', 'wpai_wp_all_import_multi_glue', 10, 1 );
function wpai_wp_all_import_multi_glue( $delimiter ) {
return "|";
}

2. Use a combination of a PHP function and an XPath query that removes empty entries, fixes spacing between letters and open parentheses, then outputs a comma delimited list of linked qualifications:

[str_replace(array("(", " ("),array(" (", " ("),{*[starts-with(name(),"linkedqualification") and string()]})]

3. Changed the delimiter to pipe