Pim-community-dev: 2.1.8 : Duplicate entry for key 'searchunique_idx' when importing products using the CLI

Created on 6 Apr 2018 · 3Comments · Source: akeneo/pim-community-dev

When I'm importing a feed with only an SKU an images columns in Akeneo using the command line the import fails with the following command:

[2018-04-06 15:35:24] batch.ERROR: Encountered an error executing the step: 
An exception occurred while executing 'INSERT INTO pim_catalog_completeness 
(ratio, missing_count, required_count, locale_id, channel_id, product_id) VALUES (?, ?, ?, ?, ?, ?)'
with params [100, 0, 1, 39, 1, 4404]:  
SQLSTATE[23000]: Integrity constraint violation: 
1062 Duplicate entry '1-39-4404' for key 'searchunique_idx' [] []

Does anyone know what might be causing this issue? Or how I can fix it? The strange part is that the key suggests that there would also be a product with ID 4404 in my database in that table, but there isn't.

update: on further inspecting, I noticed that the file that is uploaded by the client had multiple occurrences of the same SKU. Could this be the cause of this problem? Are these things done in a single transaction? (which could explain why I couldn't find the colliding ID in the database).

Source

kanduvisla

👍1

Most helpful comment

I've experienced this problem in 2 scenario's.

Duplicate import data
You're suggestion is right: when you have multiple instances of the same sku in your import file you get this error.

Elastic Search indexes holds removed product-data
The other case where I had this problem was when elasticsearch has kept a record of a deleted product in its indexes and is somehow 'feeding' the completeness script that runs after importing product models.
This was either caused by removing the product model after which Akeneo did not remove all ES data of the underlaying product variants, or because I had removed a record manually from the database.
I'm not entirely sure which of the two caused it, but it might be the second case and then it's kind of my own fault.

however it was very hard to debug, because I could not find any duplicate record of these product anywhere in my database, nor import files.

The solution:
from commandline, throw out all ES indexes:
bin/console akeneo:elasticsearch:reset-indexes
Then rebuild:
bin/console pim:product:index --all
bin/console pim:product-model:index --all

wigman on 4 Jul 2018

👍4 👎1

All 3 comments

I've experienced this problem in 2 scenario's.

Duplicate import data
You're suggestion is right: when you have multiple instances of the same sku in your import file you get this error.

however it was very hard to debug, because I could not find any duplicate record of these product anywhere in my database, nor import files.

wigman on 4 Jul 2018

👍4 👎1

Got the same issue. The solution provided from @wigman helped me. Thank you
The issue happened cause of me by copying DB from one environment to an other, the index was for sure not the same.

sylvainraye on 27 Jul 2018

The solution suggested by @wigman works only for small shops with a few thousand products.
Command bin/console akeneo:elasticsearch:reset-indexes removes everything from index and then you have to wait some time until data gets back. If you have 150k+ product models and 1kk+ product it's not an option, because reindex can take up to 4 hours.