Project Scope

The Scope of the project was integrating products feed supplied by Toolbank (3rd party system) with Magento 2. Toolbank is one of Europe's leading specialist distributors of Hand and Power Tools, in association with many of the leading independent DIY retailers and the world's leading manufacturers

Toolbank Business to Business website provides products feed for Retailers or merchants.

Problem Statement

The client was looking for a solution to sell Toolbank products on his website. As Toolbank provides products in huge numbers so importing such a huge number of products was a challenge.

Along with that, he wanted to keep the product's stocks and prices up to date. Also, the product's images were needed to be imported from a third-party sources.

Challenge

The Client required an application that could import 32k products from Toolbank onto Magento 2 plus keep stock and prices updated from time to time. Importing that amount of products and keeping them updated in terms of stocks and prices should not downgrade site performance.

Solution

Toolbank provides data on the FTP location of Magento 2 website, from where we had read the products feed as per the guide provided by Toolbank. The data was in the CSV file which we first downloaded through PHP FTP connection code and then we parsed the file and wrote the SQL to insert and update products in the Magento 2 database.

To import such huge data for 32k products and keeping stock and prices up to date, we instead of using Magento 2 conventional way that is Object Manager, applied solutions by creating MYSQL scripts that insert/update the products and the value of their attributes. The whole of this required understanding of Magento 2 default database schema and its integrity constraints. To import such huge data, we made use of MYSQL INFILE to read CSV data directly to temporary tables and parse that data using MYSQL advanced query like using joins on multi tables while inserting and updating records. As we were not relying on Magento 2 Object Manager that could slow down the whole import process, we created MYSQL queries for the same to keep site performance high.

Solution Flow Diagram

Technology Used
Magento Open Source