Understanding the Product Export¶
It’s a good start to understand the overall architecture and how to re-use or replace some parts. You can now natively export data into CSV and XLSX format.
Definition of the Job¶
The product export is defined in src/Akeneo/Pim/Enrichment/Bundle/Resources/config/jobs.yml
.
services:
## CSV export
pim_connector.job.csv_product_export:
class: 'Akeneo\Tool\Component\Batch\Job\Job'
arguments:
- 'csv_product_export' # The job name
- '@event_dispatcher'
- '@akeneo_batch.job_repository'
-
- '@pim_connector.step.csv_product.export' # The product export step
- '@akeneo.job_automation.connector.step.upload' # The upload step
- true # Does the job is stoppable ?
tags:
- { name: akeneo_batch.job, connector: 'Akeneo CSV Connector', type: 'export' }
## XLSX export
pim_connector.job.xlsx_product_export:
class: 'Akeneo\Tool\Component\Batch\Job\Job'
arguments:
- 'xlsx_product_export'
- '@event_dispatcher'
- '@akeneo_batch.job_repository'
-
- '@pim_connector.step.xlsx_product.export'
- '@akeneo.job_automation.connector.step.upload'
- true
tags:
- { name: akeneo_batch.job, connector: 'Akeneo XLSX Connector', type: 'export' }
With the type
parameter, we can see that this job is an export.
Product Export Step¶
The purpose of this step is to read products from database, to transform product objects to array and write lines in a csv file.
All steps service definitions are defined in src/Akeneo/Pim/Enrichment/Bundle/Resources/config/steps.yml
.
services:
pim_connector.step.csv_product.export:
class: 'Akeneo\Tool\Component\Batch\Step\ItemStep'
arguments:
- 'export' # Export name
- '@event_dispatcher'
- '@akeneo_batch.job_repository'
- '@pim_connector.reader.database.product' # Reader
- '@pim_connector.processor.normalization.product' # Processor
- '@pim_connector.writer.file.csv_product' # Writer
- 10 # Batch size
- '@akeneo_batch.job.job_stopper'
pim_connector.step.xlsx_product.export:
class: 'Akeneo\Tool\Component\Batch\Step\ItemStep'
arguments:
- 'export'
- '@event_dispatcher'
- '@akeneo_batch.job_repository'
- '@pim_connector.reader.database.product' # Reader
- '@pim_connector.processor.normalization.product' # Processor
- '@pim_connector.writer.file.xlsx_product' # Writer
- 10 # Batch size
- '@akeneo_batch.job.job_stopper'
An ItemStep
always contains 3 elements:
Akeneo\Tool\Bundle\BatchBundle\Item\ItemReaderInterface
Akeneo\Tool\Bundle\BatchBundle\Item\ItemProcessorInterface
Akeneo\Tool\Bundle\BatchBundle\Item\ItemWriterInterface
We provide here specific implementations for these elements, the services are declared with aliases pim_connector.reader.database.product
, pim_connector.processor.normalization.product
, pim_connector.writer.file.csv_product
.
Product Reader¶
This element reads products from database and returns objects one by one.
The service is defined in src/Akeneo/Pim/Enrichment/Bundle/Resources/config/readers.yml
.
The product reader now uses the ProductQueryBuilder, it means that you can now finely select which products to export with the ProductQueryBuilder filters.
services:
pim_connector.reader.database.product:
class: 'Akeneo\Pim\Enrichment\Component\Product\Connector\Reader\Database\ProductReader'
arguments:
- '@pim_catalog.query.product_query_builder_factory_for_reading_purpose'
- '@pim_catalog.repository.channel'
- '@pim_catalog.converter.metric'
Product Processor¶
This element receives product objects one by one, transforms each product object into an array and returns the array.
The service is defined in src/Akeneo/Pim/Enrichment/Bundle/Resources/config/processors.yml
.
services:
pim_connector.processor.normalization.product:
class: 'Akeneo\Pim\Enrichment\Component\Product\Connector\Processor\Normalization\ProductProcessor'
arguments:
- '@pim_catalog.normalizer.standard.product'
- '@pim_catalog.repository.channel'
- '@pim_catalog.repository.attribute'
- '@pim_catalog.product_model.fill_missing_values'
- '@akeneo.pim.structure.query.get_attributes'
- '@pim_connector.processor.normalization.get_normalized_product_quality_scores'
The class Akeneo\Pim\Enrichment\Component\Product\Connector\Processor\Normalization\ProductProcessor
mainly delegates the transformation to the service pim_catalog.normalizer.standard.product
.
We can see here that we normalize each product into the standard
format. It is the writer’s responsibility to convert the standard format to the flat format. (cf How to clean a CSV file during a Product import)
$productStandard = $this->normalizer->normalize($product, 'json', [
'channels' => [$channel->getCode()],
'locales' => array_intersect(
$channel->getLocaleCodes(),
$parameters->get('filters')['structure']['locales']
),
]);
This service pim_catalog.normalizer.standard.product
is declared in src/Akeneo/Pim/Enrichment/Bundle/Resources/config/serializers_standard.yml
and uses the Symfony Serializer
class.
As a product may not have values for all attributes, depending on the product, the normalized array will contain different keys, for instance,
$product1 = [
'sku' => [
['data' => 'AKNTS_BPXS', 'locale' => null, 'scope' => null]
],
'family' => 'tshirts',
'clothing_size' =>
[
[
'locale' => NULL,
'scope' => NULL,
'data' => 'xs'
],
],
'description' =>
[
[
'locale' => 'en_US',
'scope' => 'mobile',
'data' => 'Akeneo T-Shirt'
],
],
];
Here is another example:
$product2 = [
'sku' => [
['data' => 'AKNTS_BPXS', 'locale' => null, 'scope' => null]
],
'family' => 'tshirts',
'main_color' =>
[
[
'locale' => NULL,
'scope' => NULL,
'data' => 'black'
],
],
'name' =>
[
[
'locale' => NULL,
'scope' => NULL,
'data' => 'Akeneo T-Shirt black and purple with short sleeve'
],
],
];
Note
You can find extra information about the Serializer component in the official Symfony documentation https://symfony.com/doc/5.4/components/serializer.html
Product Writer¶
This element receives products in the standard format, converts them in flat format with the converter and writes the lines in a csv file.
The service is defined in src/Akeneo/Pim/Enrichment/Bundle/Resources/config/steps.yml
.
services:
pim_connector.writer.file.csv_product:
class: 'Akeneo\Pim\Enrichment\Component\Product\Connector\Writer\File\Csv\ProductWriter'
arguments:
- '@pim_connector.array_converter.standard_to_flat.product_localized'
- '@pim_connector.factory.flat_item_buffer'
- '@pim_connector.writer.file.product.flat_item_buffer_flusher'
- '@pim_catalog.repository.attribute'
- '@pim_connector.writer.file.media_exporter_path_generator'
- '@akeneo.pim.enrichment.connector.write.file.flat.generate_headers_from_family_codes'
- '@akeneo.pim.enrichment.connector.write.file.flat.generate_headers_from_attribute_codes'
- '@pim_enrich.connector.flat_translators.product_translator'
- '@akeneo_file_storage.repository.file_info'
- '@akeneo_file_storage.file_storage.filesystem_provider'
- ['pim_catalog_file', 'pim_catalog_image']
This service first merges all used columns in all the rows, adds missing cells in each row, then writes the csv file.
$products = [
[
'sku' => 'AKNTS_BPXS',
'family' => 'tshirts',
'clothing_size' => 'xs',
'description-en_US-mobile' => 'Akeneo T-Shirt',
'main_color' => '',
'name' => ''
],
[
'sku' => 'AKNTS_BPXS',
'family' => 'tshirts',
'clothing_size' => '',
'description-en_US-mobile' => '',
'main_color' => 'black',
'name' => 'Akeneo T-Shirt black and purple with short sleeve'
]
];
Found a typo or a hole in the documentation and feel like contributing?
Join us on Github!