How to Import Products from a XML File

Prerequisite

The basics of a connector creation have been covered in the previous chapter (cf. How to Create a New Connector). With the following hands-on practice, we will create our own specific connector.

We assume that we’re using a standard edition with the icecat_demo_dev data set, sku and name already exist as attributes in the PIM and the family is also an existing property.

Overview

In this cookbook, we will create a brand new XML connector to import our products using XML files.

For a recap, here is the process of a product import job execution:

  1. Starting the job and reading the first product

    • The job opens the file to import, it reads the first product in the file and converts it into a standard format.
    • If an error or a warning is thrown during this step, the product is marked as invalid.

  2. Process the product and check the product values

    • If no errors have been thrown in the previous step, the read and converted product is then processed by a product processor.
    • If an error is thrown while processing, the product is marked as invalid.

Note

At this point, an error could be that the the family code does not exist or that the currency set for a price attribute does not match the currency configured in the PIM for this attribute.

  1. Save the product in database

    • The processed product is written in the database using a product writer.

  2. Collect the invalid products found and export them into a separate file

    • When all products have been read, processed and written into the database, the job collects all the errors found in the file at each step and writes them back into a separate file of invalid items.

In this cookbook, our use case is to import new products from the following XML file products.xml:

1
2
3
4
5
6
7
8
<?xml version="1.0" encoding="UTF-8"?>
<products>
    <product sku="sku-1" name="my name 1" family="camcorders" enabled="1"/>
    <product sku="sku-2" name="my name 2" family="camcorders" enabled="1"/>
    <product sku="sku-3" name="my name 3" family="camcorders" enabled="1"/>
    <product sku="sku-4" name="my name 4" family="wrong_family_code" enabled="0"/>
    <product sku="" name="my name 5" family="camcorders" enabled="1"/>
</products>

To stay focused on the main concepts, we will implement the simplest connector possible by avoiding to use too many existing elements.

Note

The code inside this cookbook entry is available in the src directory, you can clone pim-docs (https://github.com/akeneo/pim-docs) and use a symlink to make the Acme bundle available in the src/.

Create the Connector

Create a new bundle:

1
2
3
4
5
6
7
8
9
<?php

namespace Acme\Bundle\XmlConnectorBundle;

use Symfony\Component\HttpKernel\Bundle\Bundle;

class AcmeXmlConnectorBundle extends Bundle
{
}

Register the bundle in AppKernel:

1
2
3
4
5
6
7
8
public function registerProjectBundles()
{
    return [
        // your app bundles should be registered here
        new Acme\Bundle\AppBundle\AcmeAppBundle(),
        new Acme\Bundle\XmlConnectorBundle\AcmeXmlConnectorBundle()
    ];
}

Configure the Job

Configure the job in Resources/config/jobs.yml:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
    acme_xml_connector.job.xml_product_import:
        class: '%pim_connector.job.simple_job.class%'
        arguments:
            - 'xml_product_import'
            - '@event_dispatcher'
            - '@akeneo_batch.job_repository'
            -
                - '@acme_xml_connector.step.xml_product_import.import'
        tags:
            - { name: akeneo_batch.job, connector: 'Akeneo XML Connector', type: '%pim_connector.job.import_type%' }

The default step is Akeneo\Component\Batch\Step\ItemStep.

An item step is configured with 3 elements, a reader, a processor and a writer.

Here is the definition of the Step:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
    acme_xml_connector.step.xml_product_import.import:
        class: '%pim_connector.step.item_step.class%'
        arguments:
            - 'import'
            - '@event_dispatcher'
            - '@akeneo_batch.job_repository'
            - '@acme_xml_connector.reader.file.xml_product'
            - '@pim_connector.processor.denormalization.product'
            - '@pim_connector.writer.database.product'
            - 10

Here, we’ll use a custom reader acme_xml_connector.reader.file.xml_product but we’ll continue to use the default processor and writer.

Then you will need to add the job parameters classes (they define the job configuration, job constraints and job default values):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
parameters:
    acme_xml_connector.job.job_parameters.simple_xml_export.class: Acme\Bundle\XmlConnectorBundle\Job\JobParameters\SimpleXmlImport

services:
    acme_xml_connector.job.job_parameters.simple_xml_product_import:
        class: '%acme_xml_connector.job.job_parameters.simple_xml_export.class%'
        arguments:
            - '%pim_catalog.localization.decimal_separators%'
            - '%pim_catalog.localization.date_formats%'
        tags:
            - { name: akeneo_batch.job.job_parameters.constraint_collection_provider }
            - { name: akeneo_batch.job.job_parameters.default_values_provider }

    acme_xml_connector.job.job_parameters.provider.simple_xml_product_import:
        class: '%pim_enrich.provider.form.job_instance.class%'
        arguments:
            -
                xml_product_import: pim-job-instance-xml-product-import
        tags:
            - { name: pim_enrich.provider.form }

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
<?php

namespace Acme\Bundle\XmlConnectorBundle\Job\JobParameters;

use Akeneo\Component\Batch\Job\JobInterface;
use Akeneo\Component\Batch\Job\JobParameters\ConstraintCollectionProviderInterface;
use Akeneo\Component\Batch\Job\JobParameters\DefaultValuesProviderInterface;
use Akeneo\Component\Localization\Localizer\LocalizerInterface;
use Pim\Component\Catalog\Validator\Constraints\FileExtension;
use Symfony\Component\Validator\Constraints\Collection;
use Symfony\Component\Validator\Constraints\IsTrue;
use Symfony\Component\Validator\Constraints\NotBlank;
use Symfony\Component\Validator\Constraints\Type;

class SimpleXmlImport implements
    DefaultValuesProviderInterface,
    ConstraintCollectionProviderInterface
{
    /** @var ConstraintCollectionProviderInterface */
    protected $constraintCollectionProvider;

    /** @var DefaultValuesProviderInterface */
    protected $defaultValuesProvider;

    /**
     * {@inheritdoc}
     */
    public function getDefaultValues()
    {
        return [
            'filePath'                  => null,
            'uploadAllowed'             => true,
            'dateFormat'                => LocalizerInterface::DEFAULT_DATE_FORMAT,
            'decimalSeparator'          => LocalizerInterface::DEFAULT_DECIMAL_SEPARATOR,
            'enabled'                   => true,
            'categoriesColumn'          => 'categories',
            'familyColumn'              => 'family',
            'groupsColumn'              => 'groups',
            'enabledComparison'         => true,
            'realTimeVersioning'        => true,
            'invalid_items_file_format' => 'xml',
        ];
    }

    /**
     * {@inheritdoc}
     */
    public function getConstraintCollection()
    {
        return new Collection(
            [
                'fields' => [
                    'filePath' => [
                        new NotBlank(['groups' => ['Execution', 'UploadExecution']]),
                        new FileExtension(
                            [
                                'allowedExtensions' => ['xml', 'zip'],
                                'groups'            => ['Execution', 'UploadExecution']
                            ]
                        )
                    ],
                    'uploadAllowed' => [
                        new Type('bool'),
                        new IsTrue(['groups' => 'UploadExecution']),
                    ],
                    'decimalSeparator' => [
                        new NotBlank()
                    ],
                    'dateFormat' => [
                        new NotBlank()
                    ],
                    'enabled' => [
                        new Type('bool')
                    ],
                    'categoriesColumn' => [
                        new NotBlank()
                    ],
                    'familyColumn' => [
                        new NotBlank()
                    ],
                    'groupsColumn' => [
                        new NotBlank()
                    ],
                    'enabledComparison' => [
                        new Type('bool')
                    ],
                    'realTimeVersioning' => [
                        new Type('bool')
                    ],
                    'invalid_items_file_format' => [
                        new Type('string')
                    ]
                ]
            ]
        );
    }

    /**
     * {@inheritdoc}
     */
    public function supports(JobInterface $job)
    {
        return $job->getName() === 'xml_product_import';
    }
}

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
extensions:
    pim-job-instance-xml-product-import-edit:
        module: pim/form/common/edit-form

    pim-job-instance-xml-product-import-edit-cache-invalidator:
        module: pim/cache-invalidator
        parent: pim-job-instance-xml-product-import-edit
        position: 1000

    pim-job-instance-xml-product-import-edit-tabs:
        module: pim/form/common/form-tabs
        parent: pim-job-instance-xml-product-import-edit
        targetZone: content
        position: 100

    pim-job-instance-xml-product-import-edit-properties:
        module: pim/job/common/edit/properties
        parent: pim-job-instance-xml-product-import-edit-tabs
        aclResourceId: pim_importexport_export_profile_property_edit
        targetZone: container
        position: 100
        config:
            tabTitle: pim_enrich.form.job_instance.tab.properties.title
            tabCode: pim-job-instance-properties

    pim-job-instance-xml-product-import-edit-history:
        module: pim/common/tab/history
        parent: pim-job-instance-xml-product-import-edit-tabs
        targetZone: container
        aclResourceId: pim_importexport_import_profile_history
        position: 120
        config:
            class: Akeneo\Component\Batch\Model\JobInstance
            title: pim_enrich.form.job_instance.tab.history.title
            tabCode: pim-job-instance-history

    pim-job-instance-xml-product-import-edit-properties-code:
        module: pim/job/common/edit/field/text
        parent: pim-job-instance-xml-product-import-edit-properties
        position: 100
        targetZone: properties
        config:
            fieldCode: code
            label: pim_enrich.form.job_instance.tab.properties.code.title
            readOnly: true

    pim-job-instance-xml-product-import-edit-properties-label:
        module: pim/job/common/edit/field/text
        parent: pim-job-instance-xml-product-import-edit-properties
        position: 110
        targetZone: properties
        config:
            fieldCode: label
            label: pim_enrich.form.job_instance.tab.properties.label.title

    pim-job-instance-xml-product-import-edit-properties-file-path:
        module: pim/job/common/edit/field/text
        parent: pim-job-instance-xml-product-import-edit-properties
        position: 120
        targetZone: global-settings
        config:
            fieldCode: configuration.filePath
            readOnly: false
            label: pim_enrich.form.job_instance.tab.properties.file_path.title
            tooltip: pim_enrich.form.job_instance.tab.properties.file_path.help

    pim-job-instance-xml-product-import-edit-properties-file-upload:
        module: pim/job/common/edit/field/switch
        parent: pim-job-instance-xml-product-import-edit-properties
        position: 130
        targetZone: global-settings
        config:
            fieldCode: configuration.uploadAllowed
            readOnly: false
            label: pim_enrich.form.job_instance.tab.properties.upload_allowed.title
            tooltip: pim_enrich.form.job_instance.tab.properties.upload_allowed.help
            readOnly: false

    pim-job-instance-xml-product-import-edit-properties-decimal-separator:
        module: pim/job/common/edit/field/decimal-separator
        parent: pim-job-instance-xml-product-import-edit-properties
        position: 170
        targetZone: global-settings
        config:
            fieldCode: configuration.decimalSeparator
            readOnly: false
            label: pim_enrich.form.job_instance.tab.properties.decimal_separator.title
            tooltip: pim_enrich.form.job_instance.tab.properties.decimal_separator.help

    pim-job-instance-xml-product-import-edit-properties-date-format:
        module: pim/job/product/edit/field/date-format
        parent: pim-job-instance-xml-product-import-edit-properties
        position: 180
        targetZone: global-settings
        config:
            fieldCode: configuration.dateFormat
            readOnly: false
            label: pim_enrich.form.job_instance.tab.properties.date_format.title
            tooltip: pim_enrich.form.job_instance.tab.properties.date_format.help

    pim-job-instance-xml-product-import-edit-properties-enabled:
        module: pim/job/common/edit/field/switch
        parent: pim-job-instance-xml-product-import-edit-properties
        position: 190
        targetZone: global-settings
        config:
            fieldCode: configuration.enabled
            readOnly: false
            label: pim_enrich.form.job_instance.tab.properties.enabled.title
            tooltip: pim_enrich.form.job_instance.tab.properties.enabled.help

    pim-job-instance-xml-product-import-edit-properties-categories-column:
        module: pim/job/common/edit/field/text
        parent: pim-job-instance-xml-product-import-edit-properties
        position: 200
        targetZone: global-settings
        config:
            fieldCode: configuration.categoriesColumn
            readOnly: false
            label: pim_enrich.form.job_instance.tab.properties.categories_column.title
            tooltip: pim_enrich.form.job_instance.tab.properties.categories_column.help

    pim-job-instance-xml-product-import-edit-properties-family-column:
        module: pim/job/common/edit/field/text
        parent: pim-job-instance-xml-product-import-edit-properties
        position: 210
        targetZone: global-settings
        config:
            fieldCode: configuration.familyColumn
            readOnly: false
            label: pim_enrich.form.job_instance.tab.properties.family_column.title
            tooltip: pim_enrich.form.job_instance.tab.properties.family_column.help

    pim-job-instance-xml-product-import-edit-properties-groups-column:
        module: pim/job/common/edit/field/text
        parent: pim-job-instance-xml-product-import-edit-properties
        position: 220
        targetZone: global-settings
        config:
            fieldCode: configuration.groupsColumn
            readOnly: false
            label: pim_enrich.form.job_instance.tab.properties.groups_column.title
            tooltip: pim_enrich.form.job_instance.tab.properties.groups_column.help

    pim-job-instance-xml-product-import-edit-properties-enabled-comparison:
        module: pim/job/common/edit/field/switch
        parent: pim-job-instance-xml-product-import-edit-properties
        position: 230
        targetZone: global-settings
        config:
            fieldCode: configuration.enabledComparison
            readOnly: false
            label: pim_enrich.form.job_instance.tab.properties.enabled_comparison.title
            tooltip: pim_enrich.form.job_instance.tab.properties.enabled_comparison.help

    pim-job-instance-xml-product-import-edit-properties-real-time-versioning:
        module: pim/job/common/edit/field/switch
        parent: pim-job-instance-xml-product-import-edit-properties
        position: 240
        targetZone: global-settings
        config:
            fieldCode: configuration.realTimeVersioning
            readOnly: false
            label: pim_enrich.form.job_instance.tab.properties.real_time_versioning.title
            tooltip: pim_enrich.form.job_instance.tab.properties.real_time_versioning.help

    pim-job-instance-xml-product-import-edit-label:
        module: pim/job/common/edit/label
        parent: pim-job-instance-xml-product-import-edit
        targetZone: title
        position: 100

    pim-job-instance-xml-product-import-edit-meta:
        module: pim/job/common/edit/meta
        parent: pim-job-instance-xml-product-import-edit
        targetZone: meta
        position: 100

    pim-job-instance-xml-product-import-edit-back-to-grid:
        module: pim/form/common/back-to-grid
        parent: pim-job-instance-xml-product-import-edit
        targetZone: back
        aclResourceId: pim_importexport_import_profile_index
        position: 80
        config:
            backUrl: pim_importexport_import_profile_index

    pim-job-instance-xml-product-import-edit-delete:
        module: pim/job/import/edit/delete
        parent: pim-job-instance-xml-product-import-edit
        targetZone: buttons
        aclResourceId: pim_importexport_import_profile_remove
        position: 100
        config:
            trans:
                title: confirmation.remove.job_instance
                content: pim_enrich.confirmation.delete_item
                success: flash.job_instance.removed
                failed: error.removing.job_instance
            redirect: pim_importexport_import_profile_index

    pim-job-instance-xml-product-import-edit-save-buttons:
        module: pim/form/common/save-buttons
        parent: pim-job-instance-xml-product-import-edit
        targetZone: buttons
        position: 120

    pim-job-instance-xml-product-import-edit-save:
        module: pim/job-instance-import-edit-form/save
        parent: pim-job-instance-xml-product-import-edit
        targetZone: buttons
        position: 0
        config:
            redirectPath: pim_importexport_import_profile_show

    pim-job-instance-xml-product-import-edit-state:
        module: pim/form/common/state
        parent: pim-job-instance-xml-product-import-edit
        targetZone: state
        position: 900
        config:
            entity: pim_enrich.entity.job_instance.title

    pim-job-instance-xml-product-import-edit-validation:
        module: pim/job/common/edit/validation
        parent: pim-job-instance-xml-product-import-edit

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
extensions:
    pim-job-instance-xml-product-import-show:
        module: pim/form/common/edit-form

    pim-job-instance-xml-product-import-show-tabs:
        module: pim/form/common/form-tabs
        parent: pim-job-instance-xml-product-import-show
        targetZone: content
        position: 100

    pim-job-instance-xml-product-import-show-upload:
        module: pim/job/common/edit/upload
        parent: pim-job-instance-xml-product-import-show
        aclResourceId: pim_importexport_import_profile_launch
        targetZone: content
        position: 90

    pim-job-instance-xml-product-import-show-properties:
        module: pim/job/common/edit/properties
        parent: pim-job-instance-xml-product-import-show-tabs
        aclResourceId: pim_importexport_export_profile_property_show
        targetZone: container
        position: 100
        config:
            tabTitle: pim_enrich.form.job_instance.tab.properties.title
            tabCode: pim-job-instance-properties

    pim-job-instance-xml-product-import-show-history:
        module: pim/common/tab/history
        parent: pim-job-instance-xml-product-import-show-tabs
        targetZone: container
        aclResourceId: pim_importexport_import_profile_history
        position: 120
        config:
            class: Akeneo\Component\Batch\Model\JobInstance
            title: pim_enrich.form.job_instance.tab.history.title
            tabCode: pim-job-instance-history

    pim-job-instance-xml-product-import-show-properties-code:
        module: pim/job/common/edit/field/text
        parent: pim-job-instance-xml-product-import-show-properties
        position: 100
        targetZone: properties
        config:
            fieldCode: code
            label: pim_enrich.form.job_instance.tab.properties.code.title
            readOnly: true

    pim-job-instance-xml-product-import-show-properties-label:
        module: pim/job/common/edit/field/text
        parent: pim-job-instance-xml-product-import-show-properties
        position: 110
        targetZone: properties
        config:
            fieldCode: label
            label: pim_enrich.form.job_instance.tab.properties.label.title
            readOnly: true

    pim-job-instance-xml-product-import-show-properties-file-path:
        module: pim/job/common/edit/field/text
        parent: pim-job-instance-xml-product-import-show-properties
        position: 120
        targetZone: global-settings
        config:
            fieldCode: configuration.filePath
            readOnly: true
            label: pim_enrich.form.job_instance.tab.properties.file_path.title
            tooltip: pim_enrich.form.job_instance.tab.properties.file_path.help

    pim-job-instance-xml-product-import-show-properties-file-upload:
        module: pim/job/common/edit/field/switch
        parent: pim-job-instance-xml-product-import-show-properties
        position: 130
        targetZone: global-settings
        config:
            fieldCode: configuration.uploadAllowed
            readOnly: true
            label: pim_enrich.form.job_instance.tab.properties.upload_allowed.title
            tooltip: pim_enrich.form.job_instance.tab.properties.upload_allowed.help

    pim-job-instance-xml-product-import-show-properties-decimal-separator:
        module: pim/job/common/edit/field/decimal-separator
        parent: pim-job-instance-xml-product-import-show-properties
        position: 170
        targetZone: global-settings
        config:
            fieldCode: configuration.decimalSeparator
            readOnly: true
            label: pim_enrich.form.job_instance.tab.properties.decimal_separator.title
            tooltip: pim_enrich.form.job_instance.tab.properties.decimal_separator.help

    pim-job-instance-xml-product-import-show-properties-date-format:
        module: pim/job/product/edit/field/date-format
        parent: pim-job-instance-xml-product-import-show-properties
        position: 180
        targetZone: global-settings
        config:
            fieldCode: configuration.dateFormat
            readOnly: true
            label: pim_enrich.form.job_instance.tab.properties.date_format.title
            tooltip: pim_enrich.form.job_instance.tab.properties.date_format.help

    pim-job-instance-xml-product-import-show-properties-enabled:
        module: pim/job/common/edit/field/switch
        parent: pim-job-instance-xml-product-import-show-properties
        position: 190
        targetZone: global-settings
        config:
            fieldCode: configuration.enabled
            readOnly: true
            label: pim_enrich.form.job_instance.tab.properties.enabled.title
            tooltip: pim_enrich.form.job_instance.tab.properties.enabled.help

    pim-job-instance-xml-product-import-show-properties-categories-column:
        module: pim/job/common/edit/field/text
        parent: pim-job-instance-xml-product-import-show-properties
        position: 200
        targetZone: global-settings
        config:
            fieldCode: configuration.categoriesColumn
            readOnly: true
            label: pim_enrich.form.job_instance.tab.properties.categories_column.title
            tooltip: pim_enrich.form.job_instance.tab.properties.categories_column.help

    pim-job-instance-xml-product-import-show-properties-family-column:
        module: pim/job/common/edit/field/text
        parent: pim-job-instance-xml-product-import-show-properties
        position: 210
        targetZone: global-settings
        config:
            fieldCode: configuration.familyColumn
            readOnly: true
            label: pim_enrich.form.job_instance.tab.properties.family_column.title
            tooltip: pim_enrich.form.job_instance.tab.properties.family_column.help

    pim-job-instance-xml-product-import-show-properties-groups-column:
        module: pim/job/common/edit/field/text
        parent: pim-job-instance-xml-product-import-show-properties
        position: 220
        targetZone: global-settings
        config:
            fieldCode: configuration.groupsColumn
            readOnly: true
            label: pim_enrich.form.job_instance.tab.properties.groups_column.title
            tooltip: pim_enrich.form.job_instance.tab.properties.groups_column.help

    pim-job-instance-xml-product-import-show-properties-enabled-comparison:
        module: pim/job/common/edit/field/switch
        parent: pim-job-instance-xml-product-import-show-properties
        position: 230
        targetZone: global-settings
        config:
            fieldCode: configuration.enabledComparison
            readOnly: true
            label: pim_enrich.form.job_instance.tab.properties.enabled_comparison.title
            tooltip: pim_enrich.form.job_instance.tab.properties.enabled_comparison.help

    pim-job-instance-xml-product-import-show-properties-real-time-versioning:
        module: pim/job/common/edit/field/switch
        parent: pim-job-instance-xml-product-import-show-properties
        position: 240
        targetZone: global-settings
        config:
            fieldCode: configuration.realTimeVersioning
            readOnly: true
            label: pim_enrich.form.job_instance.tab.properties.real_time_versioning.title
            tooltip: pim_enrich.form.job_instance.tab.properties.real_time_versioning.help

    pim-job-instance-xml-product-import-show-label:
        module: pim/job/common/edit/label
        parent: pim-job-instance-xml-product-import-show
        targetZone: title
        position: 100

    pim-job-instance-xml-product-import-show-meta:
        module: pim/job/common/edit/meta
        parent: pim-job-instance-xml-product-import-show
        targetZone: meta
        position: 100

    pim-job-instance-xml-product-import-show-back-to-grid:
        module: pim/form/common/back-to-grid
        parent: pim-job-instance-xml-product-import-show
        targetZone: back
        aclResourceId: pim_importexport_import_profile_index
        position: 80
        config:
            backUrl: pim_importexport_import_profile_index

    pim-job-instance-xml-product-import-show-edit:
        module: pim/common/redirect
        parent: pim-job-instance-xml-product-import-show
        targetZone: buttons
        position: 100
        config:
            label: pim_enrich.form.job_instance.button.edit.title
            route: pim_importexport_import_profile_edit
            identifier:
                path: code
                name: code

    pim-job-instance-xml-product-import-show-launch:
        module: pim/job/common/edit/upload-launch
        parent: pim-job-instance-xml-product-import-show
        aclResourceId: pim_importexport_import_profile_launch
        targetZone: buttons
        position: 110
        config:
            launch: pim_enrich.form.job_instance.button.import.launch
            upload: pim_enrich.form.job_instance.button.import.upload
            route: pim_enrich_job_instance_rest_import_launch
            identifier:
                path: code
                name: code

For further information you can check the following cookbook: How to Create a New Connector.

Important

We strongly advise to always try to re-use most of the existing pieces, especially processors and writers, it makes sure that all business rules and validation will be properly applied.

Create the Reader

As we don’t have an existing reader which allows to read XML files, we’ll implement a new one that supports it.

The purpose of the reader is to return each item as an array, in the case of XML file, we can have more work to define what is the item.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
<?php

namespace Acme\Bundle\XmlConnectorBundle\Reader\File;

use Akeneo\Component\Batch\Item\FileInvalidItem;
use Akeneo\Component\Batch\Item\FlushableInterface;
use Akeneo\Component\Batch\Item\InvalidItemException;
use Akeneo\Component\Batch\Item\ItemReaderInterface;
use Akeneo\Component\Batch\Model\StepExecution;
use Akeneo\Component\Batch\Step\StepExecutionAwareInterface;
use Pim\Component\Connector\ArrayConverter\ArrayConverterInterface;
use Pim\Component\Connector\Exception\DataArrayConversionException;
use Pim\Component\Connector\Exception\InvalidItemFromViolationsException;

class XmlProductReader implements
    ItemReaderInterface,
    StepExecutionAwareInterface,
    FlushableInterface
{
    /** @var array */
    protected $xml;

    /** @var StepExecution */
    protected $stepExecution;

    /** @var ArrayConverterInterface */
    protected $converter;

    /**
     * @param ArrayConverterInterface $converter
     */
    public function __construct(ArrayConverterInterface $converter)
    {
        $this->converter = $converter;
    }

    public function read()
    {
        if (null === $this->xml) {
            $jobParameters = $this->stepExecution->getJobParameters();
            $filePath = $jobParameters->get('filePath');
            // for example purpose, we should use XML Parser to read line per line
            $this->xml = simplexml_load_file($filePath, 'SimpleXMLIterator');
            $this->xml->rewind();
        }

        if ($data = $this->xml->current()) {
            $item = [];
            foreach ($data->attributes() as $attributeName => $attributeValue) {
                $item[$attributeName] = (string) $attributeValue;
            }
            $this->xml->next();

            if (null !== $this->stepExecution) {
                $this->stepExecution->incrementSummaryInfo('item_position');
            }

            try {
                $item = $this->converter->convert($item);
            } catch (DataArrayConversionException $e) {
                $this->skipItemFromConversionException($this->xml->current(), $e);
            }

            return $item;
        }

        return null;
    }

    /**
     * {@inheritdoc}
     */
    public function flush()
    {
        $this->xml = null;
    }

    /**
     * {@inheritdoc}
     */
    public function setStepExecution(StepExecution $stepExecution)
    {
        $this->stepExecution = $stepExecution;
    }

    /**
     * @param array                        $item
     * @param DataArrayConversionException $exception
     *
     * @throws InvalidItemException
     * @throws InvalidItemFromViolationsException
     */
    protected function skipItemFromConversionException(array $item, DataArrayConversionException $exception)
    {
        if (null !== $this->stepExecution) {
            $this->stepExecution->incrementSummaryInfo('skip');
        }

        if (null !== $exception->getViolations()) {
            throw new InvalidItemFromViolationsException(
                $exception->getViolations(),
                new FileInvalidItem($item, $this->stepExecution->getSummaryInfo('item_position')),
                [],
                0,
                $exception
            );
        }

        $invalidItem = new FileInvalidItem(
            $item,
            $this->stepExecution->getSummaryInfo('item_position')
        );

        throw new InvalidItemException($exception->getMessage(), $invalidItem, [], 0, $exception);
    }
}

The reader processes the file and iterates to return products line by line and then converts them into the Standard format

This element must be configured with the path of the XML file (an example file is provided in XmlConnectorBundle\Resources\fixtures\products.xml).

Then, we need to define this reader as a service in readers.yml:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
parameters:
    acme_xml_connector.reader.file.xml_product.class: Acme\Bundle\XmlConnectorBundle\Reader\File\XmlProductReader
    acme_xml_connector.reader.file.file_iterator.class: Acme\Bundle\XmlConnectorBundle\Reader\File\XmlFileIterator

services:
    acme_xml_connector.reader.file.xml_iterator_factory:
        class: '%pim_connector.reader.file.file_iterator_factory.class%'
        arguments:
            - '%acme_xml_connector.reader.file.file_iterator.class%'
            - 'xml'

    acme_xml_connector.reader.file.xml_product:
        class: '%acme_xml_connector.reader.file.xml_product.class%'
        arguments:
            - '@pim_connector.array_converter.flat_to_standard.product'

And we introduce the following extension to load the services files in configuration:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
<?php

namespace Acme\Bundle\XmlConnectorBundle\DependencyInjection;

use Symfony\Component\HttpKernel\DependencyInjection\Extension;
use Symfony\Component\DependencyInjection\ContainerBuilder;
use Symfony\Component\DependencyInjection\Loader;
use Symfony\Component\Config\FileLocator;

class AcmeXmlConnectorExtension extends Extension
{
    public function load(array $configs, ContainerBuilder $container)
    {
        $loader = new Loader\YamlFileLoader($container, new FileLocator(__DIR__.'/../Resources/config'));
        $loader->load('archiving.yml');
        $loader->load('jobs.yml');
        $loader->load('job_parameters.yml');
        $loader->load('steps.yml');
        $loader->load('readers.yml');
        $loader->load('writers.yml');
    }
}

Translate Job and Step labels in the UI

Behind the scene, the service Pim\Bundle\ImportExportBundle\JobLabel\TranslatedLabelProvider provides translated Job and Step labels to be used in the UI.

This service uses the following conventions:
  • for a job label, given a %%jobName%%, “batch_jobs.%%jobName%%.label”
  • for a step label, given a %%jobName%% and a %%stepName%%, “batch_jobs.%%jobName%%.%%stepName%%.label”

Create a file Resources/translations/messages.en.yml in the Bundle to translate label keys.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
acme_xml_connector:
    jobs:
        xml_product_import:
            title: Product Import XML
            import:
                title: Product Import Step
    steps:
        product_processor.title: Product processor
        import.filePath:
            label: File path
            help: Path of the XML file


batch_jobs:
    xml_product_import:
        label: Product Import XML
        import.label: Product Import XML

pim_import_export:
    download_archive:
      invalid_xml: Download invalid items in XML

pim_connector:
    import:
        xml:
            enabled:
                label: Enable the product
                help: Whether or not the imported product should be enabled
            categoriesColumn:
                label: Categories column
                help: Name of the categories column
            familyColumn:
                label: Family column
                help: Name of the family column
            groupsColumn:
                label: Groups column
                help: Name of the groups column
            filePath:
                label: File
                help: The CSV file to import
            uploadAllowed:
                label: Allow file upload
                help: Whether or not to allow uploading the file directly
            delimiter:
                label: Delimiter
                help: One character used to set the field delimiter for CSV file
            enclosure:
                label: Enclosure
                help: One character used to set the field enclosure
            escape:
                label: Escape
                help: One character used to set the field escape
            circularRefsChecked:
                label: Check circular references
                help: If yes, data will be processed to make sure that there are no circular references between the categories
            realTimeVersioning:
                label: Real time history update
                help: Means that the product history is automatically updated, can be switched off to improve performances
            copyValuesToProducts:
                label: Copy variant group values to products
                help: Means that the products are automatically updated with variant group values, can be switched off to only update variant group
            enabledComparison:
                label: Compare values
                help: Enable the comparison between original values and imported values. Can speed up the import if imported values are very similar to original values
            decimalSeparator:
                label: Decimal separator
                help: One character used to set the field separator for decimal
            dateFormat:
                label: Date format
                help: Specify the format of any date columns in the file, e.g. here DD/MM/YYYY for a 30/04/2014 format.

Use the new Connector

Now if you refresh the cache, the new connector and xml job can be found under Collect > Import profiles > Create import profile.

You can create an instance of this job and give it a name like xml_product_import.

Now you can run the job from the UI or use the following command:

php app/console akeneo:batch:job xml_product_import

Adding support for invalid items export

When the PIM reads the file and processes the entities we want to import, it performs several checks to make sure that the values we import are valid and they respect the constraints we configured the PIM with.

The PIM is then capable of exporting back the invalid items that do not respect those constraints after an import operation.

For our connector to support this feature, we will need to implement a few more parts in our connector:

  • XmlInvalidItemWriter: a registered XML invalid writer service whose work is to export the invalid lines found during the reading and processing steps.
  • XmlFileIterator: which is used by the XmlInvalidItemWriter to read the imported file to find the invalid items.
  • XmlWriter: Its responsibility is to write the invalid items back into a separate file available for download to the user.

Create an XML file iterator class

Let create a class which implements the FileIteratorInterface. This class opens the XML file that was imported thanks to an instance of \SimpleXMLIterator.

We now need to implement the functions of the interface. Here is a working example of the XML iterator:

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
<?php

namespace Acme\Bundle\XmlConnectorBundle\Reader\File;

use Pim\Component\Connector\Reader\File\FileIteratorInterface;
use Symfony\Component\Filesystem\Exception\FileNotFoundException;

class XmlFileIterator implements FileIteratorInterface
{
    /** @var string **/
    protected $type;

    /** @var string **/
    protected $filePath;

    /** @var \SplFileInfo **/
    protected $fileInfo;

    /** @var \SimpleXMLIterator */
    protected $xmlFileIterator;

    /**
     * {@inheritdoc}
     */
    public function __construct($type, $filePath, array $options = [])
    {
        $this->type     = $type;
        $this->filePath = $filePath;
        $this->fileInfo = new \SplFileInfo($filePath);

        if (!$this->fileInfo->isFile()) {
            throw new FileNotFoundException(sprintf('File "%s" could not be found', $this->filePath));
        }

        $this->xmlFileIterator = simplexml_load_file($filePath, 'SimpleXMLIterator');
        $this->xmlFileIterator->rewind();
    }

    /**
     * {@inheritdoc}
     */
    public function getDirectoryPath()
    {
        if (null === $this->archivePath) {
            return $this->fileInfo->getPath();
        }

        return $this->archivePath;
    }

    /**
     * {@inheritdoc}
     */
    public function getHeaders()
    {
        $headers = [];
        foreach ($this->xmlFileIterator->current()->attributes() as $header => $value) {
            $headers[] = $header;
        }

        return $headers;
    }

    /**
     * {@inheritdoc}
     */
    public function current()
    {
        $elem = $this->xmlFileIterator->current();

        return $this->xmlElementToFlat($elem);
    }

    /**
     * {@inheritdoc}
     */
    public function next()
    {
        $this->xmlFileIterator->next();
    }

    /**
     * {@inheritdoc}
     */
    public function key()
    {
        return $this->xmlFileIterator->key();
    }

    /**
     * {@inheritdoc}
     */
    public function valid()
    {
        return $this->xmlFileIterator->valid();
    }

    /**
     * {@inheritdoc}
     */
    public function rewind()
    {
        $this->xmlFileIterator->rewind();
    }

    /**
     * Converts an xml node into an array of values
     *
     * @param \SimpleXMLIterator $elem
     *
     * @return array
     */
    protected function xmlElementToFlat($elem)
    {
        $flatElem = [];

        foreach ($elem->attributes() as $value) {
            $flatElem[] = (string) $value;
        }

        return $flatElem;
    }
}

Now, let’s declare a simple Symfony service for this class. Here is the Resources/config/readers.yml:

    acme_xml_connector.reader.file.xml_iterator_factory:
        class: '%pim_connector.reader.file.file_iterator_factory.class%'
        arguments:
            - '%acme_xml_connector.reader.file.file_iterator.class%'
            - 'xml'

Create an XML writer class

The XML writer will be responsible for writing the invalid items in a specified file path.

An implementation of it could be:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
<?php

namespace Acme\Bundle\XmlConnectorBundle\Writer;

use Akeneo\Component\Batch\Item\FlushableInterface;
use Akeneo\Component\Batch\Item\InitializableInterface;
use Akeneo\Component\Batch\Item\ItemWriterInterface;
use Akeneo\Component\Batch\Step\StepExecutionAwareInterface;
use Pim\Component\Connector\Writer\File\AbstractFileWriter;
use Pim\Component\Connector\Writer\File\ArchivableWriterInterface;

class XmlWriter extends AbstractFileWriter implements
    ItemWriterInterface,
    InitializableInterface,
    FlushableInterface,
    ArchivableWriterInterface,
    StepExecutionAwareInterface
{
    /** @var array */
    protected $writtenFiles = [];

    /** @var \XMLWriter **/
    protected $xml;

    /**
     * {@inheritdoc}
     */
    public function initialize()
    {
        if (null === $this->xml) {
            $filePath = $this->stepExecution->getJobParameters()->get('filePath');

            $this->xml = new \XMLWriter();
            $this->xml->openURI($filePath);
            $this->xml->startDocument('1.0', 'UTF-8');
            $this->xml->setIndent(4);
            $this->xml->startElement('products');
        }
    }

    /**
     * {@inheritdoc}
     */
    public function getWrittenFiles()
    {
        return $this->writtenFiles;
    }

    /**
     * {@inheritdoc}
     */
    public function write(array $items)
    {
        $exportDirectory = dirname($this->getPath());
        if (!is_dir($exportDirectory)) {
            $this->localFs->mkdir($exportDirectory);
        }

        foreach ($items as $item) {
            $this->xml->startElement('product');
            foreach ($item as $property => $value) {
                $this->xml->writeAttribute($property, $value);
            }
            $this->xml->endElement();
        }
    }

    /**
     * {@inheritdoc}
     */
    public function flush()
    {
        $this->xml->endElement();
        $this->xml->endDocument();
        $this->xml->flush();

        $this->writtenFiles = [$this->stepExecution->getJobParameters()->get('filePath')];
    }
}

Let’s declare a Symfony service for our XML writer in Resources/config/writers.yml:

1
2
3
4
5
6
parameters:
  acme_xml_connector.writer.file.xml.class: Acme\Bundle\XmlConnectorBundle\Writer\XmlWriter

services:
    acme_xml_connector.writer.file.invalid_items_xml:
        class: '%acme_xml_connector.writer.file.xml.class%'

Note

Please note that every new configuration file created in the Resources/config folder should be loaded in the Symfony dependency injection for it to be taken into account.

Plug it all together

Now that our XmlFileIterator class and service are defined, let’s use them in our custom implementation of the XmlInvalidWriterInterface.

Let’s use the existing AbstractInvalidItem to implement our custom class. We only need to implement two functions from our abstract superclass.

  • getInputFileIterator(JobParameters $jobParameters): that returns a configured instance of our custom reader the XmlFileIterator class.
  • setupWriter(JobExecution $jobExecution): sets up our custom writer, an instance of the XmlWriter class.

Here is a working example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
<?php

namespace Acme\Bundle\XmlConnectorBundle\Archiver;

use Akeneo\Component\Batch\Job\JobParameters;
use Akeneo\Component\Batch\Model\JobExecution;
use Akeneo\Component\Batch\Model\StepExecution;
use Pim\Component\Connector\Archiver\AbstractInvalidItemWriter;

class XmlInvalidItemWriter extends AbstractInvalidItemWriter
{
    /**
     * {@inheritdoc}
     */
    public function getName()
    {
        return 'invalid_xml';
    }

    /**
     * {@inheritdoc}
     */
    protected function getInputFileIterator(JobParameters $jobParameters)
    {
        $filePath = $jobParameters->get('filePath');
        $fileIterator = $this->fileIteratorFactory->create($filePath);
        $fileIterator->rewind();

        return $fileIterator;
    }

    /**
     * {@inheritdoc}
     */
    protected function setupWriter(JobExecution $jobExecution)
    {
        $fileKey = strtr($this->getRelativeArchivePath($jobExecution), ['%filename%' => 'invalid_items.xml']);
        $this->filesystem->put($fileKey, '');

        $writeParams = $this->defaultValuesProvider->getDefaultValues();
        $writeParams['filePath'] = $this->filesystem->getAdapter()->getPathPrefix() . $fileKey;

        $writeJobParameters = new JobParameters($writeParams);
        $writeJobExecution  = new JobExecution();
        $writeJobExecution->setJobParameters($writeJobParameters);

        $stepExecution = new StepExecution('processor', $writeJobExecution);
        $this->writer->setStepExecution($stepExecution);
        $this->writer->initialize();
    }
}

Let’s define a tagged Symfony service in Resources/config/archiving.yml, so that our custom invalid item writer is taken into account and used by the PIM.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
parameters:
    acme_xml_connector.archiver.invalid_item_xml_writer.class: Acme\Bundle\XmlConnectorBundle\Archiver\XmlInvalidItemWriter

services:
    acme_xml_connector.archiver.invalid_item_xml_writer:
        class: '%acme_xml_connector.archiver.invalid_item_xml_writer.class%'
        arguments:
            - '@pim_connector.event_listener.invalid_items_collector'
            - '@acme_xml_connector.writer.file.invalid_items_xml'
            - '@acme_xml_connector.reader.file.xml_iterator_factory'
            - '@oneup_flysystem.archivist_filesystem'
            - '@acme_xml_connector.job.job_parameters.simple_xml_product_import'
            - 'xml'
        tags:
            - { name: pim_connector.archiver }

Try it out

All parts of our connector are now in place for it to be able to export invalid items.

To try it out, run the XML import with the example file products.xml in the UI. At the end of the job execution a new button should appear with the label “Download invalid items in XML”.

Click it and download the XML file containing the invalid items found by the import job.