[CatalogImportExport] Static attribute cleaning asynchronously

[bhennesAdv]

Preconditions and environment

• Magento version : 2.4.7
• Anything else that would help a developer reproduce the bug :
  Actually, I don't have much to reproduce, I just know the technical cause of the error but it's related to garbage collector so it's a bit tricky to reproduce depending on the PHP settings, the amount of modules loaded by the magento instance...

Steps to reproduce

For more context, I'm using a custom module in addition of https://github.com/firegento/FireGento_FastSimpleImport2
This module is using magento product import model \Magento\CatalogImportExport\Model\Import\Product
Numerous instances of the model are created because we have low memory context.

Error is related to \Magento\CatalogImportExport\Model\Import\Product\Type\AbstractType::__destruct()

    /**
     * Clean cached values.
     *
     * @since 100.2.0
     */
    public function __destruct()
    {
        self::$attributeCodeToId = [];
        self::$commonAttributesCache = [];
    }

The thing is that the garbage collector calls the destructor of this class a bit randomly when it requires memory.
And the call of this destructor has an impact on all existing classes child of \Magento\ImportExport\Model\Import\Entity\AbstractEntity

In my case, I encountered and trace the error because the destructor was called between 2 internal call of the class :
Here is the result I got during my calls

protected function _initAttributes()
[...]
     // We have an important array of $unknownAttributeIds containing for example the attribute ID 175
     $addedAttributes[] = $this->attachAttributesByOnlyId($unknownAttributeIds);
    // At this point the self::$commonAttributesCache[175] is not set anymore.
[...]
    // I tracked it down to the lower level 
private function attachAttributesByOnlyId(array $attributeIds) : array
[...]
    foreach [...] {
        // Assuming that the attribute here has ID = 175, the attachAttribute will add it to the self::$commonAttributesCache[$attributeId]
        // If some condition are matching (Attribute must be visible or apply_to must contain the current type
        $this->attachAttribute($attribute)
        // At this point the self::$commonAttributesCache[175] is not set anymore.
    }    
[...]

I added log at the end of the attachAttribute function, in the destructor of the class and just after the attachAttribute call.
The destructor was being called multiple times (For multiple existing instances I suppose) between the return log and the aftercall log
The two logs where both (destructor call log is obvious):

var_dump(isset(self::$commonAttributesCache[175]));

bool(true)
string(50) "==================== DESTRUCTING ================="
string(50) "==================== DESTRUCTING ================="
string(50) "==================== DESTRUCTING ================="
string(50) "==================== DESTRUCTING ================="
string(50) "==================== DESTRUCTING ================="
string(50) "==================== DESTRUCTING ================="
string(50) "==================== DESTRUCTING ================="
string(50) "==================== DESTRUCTING ================="
string(50) "==================== DESTRUCTING ================="
string(50) "==================== DESTRUCTING ================="
string(50) "==================== DESTRUCTING ================="
string(50) "==================== DESTRUCTING ================="
bool(false)

Expected result

The static variables used by a given object during its lifetime must not be modify by the destructor of any other object.
Knowing the randomness of the PHP garbage collector, this is even more risky.

Actual result

The destructor call is cleaning the static caches makes the import process completely random if numerous instances of the classes are allocated and released.

Additional information

Here is the exception log of the error I faced but I had to dig for another couple of hours to find the cause of it :

[2025-01-29T13:00:10.302285+00:00] main.CRITICAL: Exception: Warning: Undefined array key "apply_to" in /magento/vendor/magento/module-catalog-import-export/Model/Import/Product/Type/AbstractType.php on line 233 in /magento/vendor/magento/framework/App/ErrorHandler.php:62
Stack trace:
#0 /magento/vendor/magento/module-catalog-import-export/Model/Import/Product/Type/AbstractType.php(233): Magento\Framework\App\ErrorHandler->handler(2, 'Undefined array...', '/magento/vendor...', 233)
#1 /magento/vendor/magento/module-catalog-import-export/Model/Import/Product/Type/AbstractType.php(338): Magento\CatalogImportExport\Model\Import\Product\Type\AbstractType->_addAttributeParams('Default', Array, Array)
#2 /magento/vendor/magento/module-bundle-import-export/Model/Import/Product/Type/Bundle.php(720): Magento\CatalogImportExport\Model\Import\Product\Type\AbstractType->_initAttributes()
#3 /magento/vendor/magento/module-catalog-import-export/Model/Import/Product/Type/AbstractType.php(203): Magento\BundleImportExport\Model\Import\Product\Type\Bundle->_initAttributes()
#4 /magento/vendor/magento/module-bundle-import-export/Model/Import/Product/Type/Bundle.php(164): Magento\CatalogImportExport\Model\Import\Product\Type\AbstractType->__construct(Object(Magento\Eav\Model\ResourceModel\Entity\Attribute\Set\CollectionFactory), Object(Magento\Catalog\Model\ResourceModel\Product\Attribute\CollectionFactory), Object(Magento\Framework\App\ResourceConnection\Interceptor), Array, NULL)
#5 /magento/generated/code/Magento/BundleImportExport/Model/Import/Product/Type/Bundle/Interceptor.php(14): Magento\BundleImportExport\Model\Import\Product\Type\Bundle->__construct(Object(Magento\Eav\Model\ResourceModel\Entity\Attribute\Set\CollectionFactory), Object(Magento\Catalog\Model\ResourceModel\Product\Attribute\CollectionFactory), Object(Magento\Framework\App\ResourceConnection\Interceptor), Array, NULL, NULL, NULL)
#6 /magento/vendor/magento/framework/ObjectManager/Factory/AbstractFactory.php(121): Magento\BundleImportExport\Model\Import\Product\Type\Bundle\Interceptor->__construct(Object(Magento\Eav\Model\ResourceModel\Entity\Attribute\Set\CollectionFactory), Object(Magento\Catalog\Model\ResourceModel\Product\Attribute\CollectionFactory), Object(Magento\Framework\App\ResourceConnection\Interceptor), Array, NULL, NULL, NULL)
#7 /magento/vendor/magento/framework/ObjectManager/Factory/Compiled.php(108): Magento\Framework\ObjectManager\Factory\AbstractFactory->createObject('Magento\\BundleI...', Array)
#8 /magento/vendor/magento/framework/ObjectManager/ObjectManager.php(59): Magento\Framework\ObjectManager\Factory\Compiled->create('Magento\\BundleI...', Array)
#9 /magento/vendor/magento/module-catalog-import-export/Model/Import/Product/Type/Factory.php(40): Magento\Framework\ObjectManager\ObjectManager->create('Magento\\BundleI...', Array)
#10 /magento/vendor/magento/module-catalog-import-export/Model/Import/Product.php(1254): Magento\CatalogImportExport\Model\Import\Product\Type\Factory->create('Magento\\BundleI...', Array)
#11 /magento/vendor/magento/module-catalog-import-export/Model/Import/Product.php(944): Magento\CatalogImportExport\Model\Import\Product->_initTypeModels()
#12 /magento/generated/code/Kstools/ImportFiregento/Preference/CatalogImportExport/Model/Import/Product2/Interceptor.php(14): Magento\CatalogImportExport\Model\Import\Product->__construct(Object(Magento\Framework\Json\Helper\Data), Object(Magento\ImportExport\Helper\Data), Object(FireGento\FastSimpleImport\ResourceModel\ImportData), Object(Magento\Eav\Model\Config\Interceptor), Object(Magento\Framework\App\ResourceConnection\Interceptor), Object(Magento\ImportExport\Model\ResourceModel\Helper), Object(Magento\Framework\Stdlib\StringUtils), Object(Magento\ImportExport\Model\Import\ErrorProcessing\ProcessingErrorAggregator), Object(Magento\Framework\Event\Manager\Proxy), Object(Magento\CatalogInventory\Model\StockRegistry\Interceptor), Object(Magento\CatalogInventory\Model\Configuration), Object(Magento\CatalogInventory\Model\StockStateProvider\Interceptor), Object(Magento\Catalog\Helper\Data), Object(Magento\ImportExport\Model\Import\Config), Object(Magento\CatalogImportExport\Model\Import\Proxy\Product\ResourceModelFactory), Object(Magento\CatalogImportExport\Model\Import\Product\OptionFactory), Object(Magento\Eav\Model\ResourceModel\Entity\Attribute\Set\CollectionFactory), Object(Magento\CatalogImportExport\Model\Import\Product\Type\Factory), Object(Magento\Catalog\Model\ResourceModel\Product\LinkFactory), Object(Magento\CatalogImportExport\Model\Import\Proxy\ProductFactory), Object(Magento\CatalogImportExport\Model\Import\UploaderFactory), Object(Magento\RemoteStorage\Filesystem), Object(Magento\CatalogInventory\Model\ResourceModel\Stock\ItemFactory), Object(Magento\Framework\Stdlib\DateTime\Timezone), Object(Magento\Framework\Stdlib\DateTime), Object(Magento\Framework\Logger\LoggerProxy), Object(Magento\Framework\Indexer\IndexerRegistry), Object(Magento\CatalogImportExport\Model\Import\Product\StoreResolver), Object(Magento\CatalogImportExport\Model\Import\Product\SkuProcessor), Object(Magento\CatalogImportExport\Model\Import\Product\CategoryProcessor), Object(Magento\CatalogImportExport\Model\Import\Product\Validator), Object(Magento\Framework\Model\ResourceModel\Db\ObjectRelationProcessor), Object(Magento\Framework\Model\ResourceModel\Db\TransactionManager), Object(Magento\CatalogImportExport\Model\Import\Product\TaxClassProcessor), Object(Magento\Framework\App\Config\Interceptor), Object(Magento\Catalog\Model\Product\Url), Array, Array, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL)
#13 /magento/vendor/magento/framework/ObjectManager/Factory/AbstractFactory.php(121): Kstools\ImportFiregento\Preference\CatalogImportExport\Model\Import\Product2\Interceptor->__construct(Object(Magento\Framework\Json\Helper\Data), Object(Magento\ImportExport\Helper\Data), Object(FireGento\FastSimpleImport\ResourceModel\ImportData), Object(Magento\Eav\Model\Config\Interceptor), Object(Magento\Framework\App\ResourceConnection\Interceptor), Object(Magento\ImportExport\Model\ResourceModel\Helper), Object(Magento\Framework\Stdlib\StringUtils), Object(Magento\ImportExport\Model\Import\ErrorProcessing\ProcessingErrorAggregator), Object(Magento\Framework\Event\Manager\Proxy), Object(Magento\CatalogInventory\Model\StockRegistry\Interceptor), Object(Magento\CatalogInventory\Model\Configuration), Object(Magento\CatalogInventory\Model\StockStateProvider\Interceptor), Object(Magento\Catalog\Helper\Data), Object(Magento\ImportExport\Model\Import\Config), Object(Magento\CatalogImportExport\Model\Import\Proxy\Product\ResourceModelFactory), Object(Magento\CatalogImportExport\Model\Import\Product\OptionFactory), Object(Magento\Eav\Model\ResourceModel\Entity\Attribute\Set\CollectionFactory), Object(Magento\CatalogImportExport\Model\Import\Product\Type\Factory), Object(Magento\Catalog\Model\ResourceModel\Product\LinkFactory), Object(Magento\CatalogImportExport\Model\Import\Proxy\ProductFactory), Object(Magento\CatalogImportExport\Model\Import\UploaderFactory), Object(Magento\RemoteStorage\Filesystem), Object(Magento\CatalogInventory\Model\ResourceModel\Stock\ItemFactory), Object(Magento\Framework\Stdlib\DateTime\Timezone), Object(Magento\Framework\Stdlib\DateTime), Object(Magento\Framework\Logger\LoggerProxy), Object(Magento\Framework\Indexer\IndexerRegistry), Object(Magento\CatalogImportExport\Model\Import\Product\StoreResolver), Object(Magento\CatalogImportExport\Model\Import\Product\SkuProcessor), Object(Magento\CatalogImportExport\Model\Import\Product\CategoryProcessor), Object(Magento\CatalogImportExport\Model\Import\Product\Validator), Object(Magento\Framework\Model\ResourceModel\Db\ObjectRelationProcessor), Object(Magento\Framework\Model\ResourceModel\Db\TransactionManager), Object(Magento\CatalogImportExport\Model\Import\Product\TaxClassProcessor), Object(Magento\Framework\App\Config\Interceptor), Object(Magento\Catalog\Model\Product\Url), Array, Array, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL)
#14 /magento/vendor/magento/framework/ObjectManager/Factory/Compiled.php(108): Magento\Framework\ObjectManager\Factory\AbstractFactory->createObject('Kstools\\ImportF...', Array)
#15 /magento/vendor/magento/framework/ObjectManager/ObjectManager.php(59): Magento\Framework\ObjectManager\Factory\Compiled->create('Kstools\\ImportF...', Array)
#16 /magento/vendor/magento/module-import-export/Model/Import/Entity/Factory.php(40): Magento\Framework\ObjectManager\ObjectManager->create('Magento\\Catalog...')
#17 /magento/vendor/magento/module-import-export/Model/Import.php(315): Magento\ImportExport\Model\Import\Entity\Factory->create('Magento\\Catalog...')
#18 /magento/vendor/magento/module-import-export/Model/LocaleEmulator.php(42): Magento\ImportExport\Model\Import->createEntityAdapter()
#19 /magento/generated/code/Magento/ImportExport/Model/LocaleEmulator/Proxy.php(105): Magento\ImportExport\Model\LocaleEmulator->emulate(Object(Closure), 'en_US')
#20 /magento/vendor/magento/module-import-export/Model/Import.php(294): Magento\ImportExport\Model\LocaleEmulator\Proxy->emulate(Object(Closure), 'en_US')
#21 /magento/vendor/magento/module-import-export/Model/Import.php(629): Magento\ImportExport\Model\Import->_getEntityAdapter()
#22 /magento/vendor/magento/module-import-export/Model/Import.php(718): Magento\ImportExport\Model\Import->getErrorAggregator()
#23 /magento/vendor/magento/module-import-export/Model/Import.php(699): Magento\ImportExport\Model\Import->validateSourceCallback(Object(FireGento\FastSimpleImport\Model\Adapters\NestedArrayAdapter))
#24 /magento/vendor/magento/module-import-export/Model/LocaleEmulator.php(53): Magento\ImportExport\Model\Import->Magento\ImportExport\Model\{closure}()
#25 /magento/generated/code/Magento/ImportExport/Model/LocaleEmulator/Proxy.php(105): Magento\ImportExport\Model\LocaleEmulator->emulate(Object(Closure), 'en_US')
#26 /magento/vendor/magento/module-import-export/Model/Import.php(698): Magento\ImportExport\Model\LocaleEmulator\Proxy->emulate(Object(Closure), 'en_US')
#27 /magento/vendor/firegento/fastsimpleimport/Model/Importer.php(72): Magento\ImportExport\Model\Import->validateSource(Object(FireGento\FastSimpleImport\Model\Adapters\NestedArrayAdapter))
#28 /magento/vendor/firegento/fastsimpleimport/Model/Importer.php(57): FireGento\FastSimpleImport\Model\Importer->validateData(Array)
[...] From here the code is specific and has no relation with the error.

Release note

No response

Triage and priority

• Severity: S0 - Affects critical data or functionality and leaves users without workaround.
• Severity: S1 - Affects critical data or functionality and forces users to employ a workaround.
• Severity: S2 - Affects non-critical data or functionality and forces users to employ a workaround.
• Severity: S3 - Affects non-critical data or functionality and does not force users to employ a workaround.
• Severity: S4 - Affects aesthetics, professional look and feel, “quality” or “usability”.

[m2-assistant]

Hi @bhennesAdv. Thank you for your report.
To speed up processing of this issue, make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce.

• For more details, review the Magento Contributor Assistant documentation.
• Add a comment to assign the issue: @magento I am working on this
• To learn more about issue processing workflow, refer to the Code Contributions.

---

Join Magento Community Engineering Slack and ask your questions in #github channel.
⚠️ According to the Magento Contribution requirements, all issues must go through the Community Contributions Triage process. Community Contributions Triage is a public meeting.
🕙 You can find the schedule on the Magento Community Calendar page.
📞 The triage of issues happens in the queue order. If you want to speed up the delivery of your contribution, join the Community Contributions Triage session to discuss the appropriate ticket.