How to Import Products from a XML File

Foundations of connector creation have been covered in the previous chapter (cf How to Create a New Connector). With the following hands-on practice, we will create our own specific connector.

To stay focused on the main concepts, we will implement the simplest connector possible by avoiding to use too many existing elements.

Our use case is to import new products from the following XML file:

1
2
3
4
5
6
<?xml version="1.0" encoding="UTF-8"?>
<products>
    <product sku="sku-1" name="my name 1" family="camcorders" enabled="1"/>
    <product sku="sku-2" name="my name 2" enabled="1"/>
    <product sku="sku-3" name="my name 3" enabled="1"/>
</products>

We assume that we’re using a standard edition with the icecat_demo_dev data set, sku and name already exist as real attributes of the PIM, family is also an existing property.

Note

The code inside this cookbook entry is available in the src directory, you can clone pim-docs (https://github.com/akeneo/pim-docs) and use a symlink to make the Acme bundle available in the src/.

Create the Connector

Create a new bundle:

1
2
3
4
5
6
7
8
9
<?php

namespace Acme\Bundle\XmlConnectorBundle;

use Symfony\Component\HttpKernel\Bundle\Bundle;

class AcmeXmlConnectorBundle extends Bundle
{
}

Register the bundle in AppKernel:

1
2
3
4
5
6
public function registerBundles()
{
    // ...
        new Acme\Bundle\XmlConnectorBundle\AcmeXmlConnectorBundle(),
    // ...
}

Configure the Job

Configure a job in Resources/config/jobs.yml:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
    acme_xml_connector.job.xml_product_import:
        class: '%pim_connector.job.simple_job.class%'
        arguments:
            - 'xml_product_import'
            - '@event_dispatcher'
            - '@akeneo_batch.job_repository'
            -
                - '@acme_xml_connector.step.xml_product_import.import'
        tags:
            - { name: akeneo_batch.job, connector: 'Akeneo XML Connector', type: '%pim_connector.job.import_type%' }

The default step is Akeneo\Component\Batch\Step\ItemStep.

An item step is configured with 3 elements, a reader, a processor and a writer.

Here is the definition of the Step:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
    acme_xml_connector.step.xml_product_import.import:
        class: '%pim_connector.step.item_step.class%'
        arguments:
            - 'import'
            - '@event_dispatcher'
            - '@akeneo_batch.job_repository'
            - '@acme_xml_connector.reader.file.xml_product'
            - '@pim_connector.processor.denormalization.product'
            - '@pim_connector.writer.database.product'
            - 10

Here, we’ll use a custom reader acme_xml_connector.reader.file.xml_product but we’ll continue to use the default processor and writer.

Then you will need to add the job parameters classes (they define the job configuration, job constraints and job default values):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
parameters:
    acme_xml_connector.job.job_parameters.product_xml_import.class: Acme\Bundle\XmlConnectorBundle\Job\JobParameters\SimpleXmlImport

services:
    acme_xml_connector.job.job_parameters.product_xml_product_import:
        class: '%acme_xml_connector.job.job_parameters.product_xml_import.class%'
        arguments:
            - '%pim_catalog.localization.decimal_separators%'
            - '%pim_catalog.localization.date_formats%'
        tags:
            - { name: akeneo_batch.job.job_parameters.constraint_collection_provider }
            - { name: akeneo_batch.job.job_parameters.default_values_provider }
            - { name: pim_import_export.job_parameters.form_configuration_provider }

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
<?php

namespace Acme\Bundle\XmlConnectorBundle\Job\JobParameters;

use Akeneo\Component\Batch\Job\JobInterface;
use Akeneo\Component\Batch\Job\JobParameters\ConstraintCollectionProviderInterface;
use Akeneo\Component\Batch\Job\JobParameters\DefaultValuesProviderInterface;
use Akeneo\Component\Localization\Localizer\LocalizerInterface;
use Pim\Bundle\ImportExportBundle\JobParameters\FormConfigurationProviderInterface;
use Pim\Component\Catalog\Validator\Constraints\FileExtension;
use Symfony\Component\Validator\Constraints\Collection;
use Symfony\Component\Validator\Constraints\IsTrue;
use Symfony\Component\Validator\Constraints\NotBlank;
use Symfony\Component\Validator\Constraints\Type;

class SimpleXmlImport implements
    DefaultValuesProviderInterface,
    ConstraintCollectionProviderInterface,
    FormConfigurationProviderInterface
{
    /** @var ConstraintCollectionProviderInterface */
    protected $constraintCollectionProvider;

    /** @var DefaultValuesProviderInterface */
    protected $defaultValuesProvider;

    /** @var FormConfigurationProviderInterface */
    protected $formConfiguration;

    /** @var array */
    protected $decimalSeparators;

    /** @var array */
    protected $dateFormats;

    /**
     * @param array $decimalSeparators
     * @param array $dateFormats
     */
    public function __construct(array $decimalSeparators, array $dateFormats)
    {
        $this->decimalSeparators = $decimalSeparators;
        $this->dateFormats = $dateFormats;
    }

    /**
     * {@inheritdoc}
     */
    public function getDefaultValues()
    {
        return [
            'filePath'           => null,
            'uploadAllowed'      => true,
            'dateFormat'         => LocalizerInterface::DEFAULT_DATE_FORMAT,
            'decimalSeparator'   => LocalizerInterface::DEFAULT_DECIMAL_SEPARATOR,
            'enabled'            => true,
            'categoriesColumn'   => 'categories',
            'familyColumn'       => 'family',
            'groupsColumn'       => 'groups',
            'enabledComparison'  => true,
            'realTimeVersioning' => true,
        ];
    }

    /**
     * {@inheritdoc}
     */
    public function getConstraintCollection()
    {
        return new Collection(
            [
                'fields' => [
                    'filePath' => [
                        new NotBlank(['groups' => ['Execution', 'UploadExecution']]),
                        new FileExtension(
                            [
                                'allowedExtensions' => ['xml', 'zip'],
                                'groups'            => ['Execution', 'UploadExecution']
                            ]
                        )
                    ],
                    'uploadAllowed' => [
                        new Type('bool'),
                        new IsTrue(['groups' => 'UploadExecution']),
                    ],
                    'decimalSeparator' => [
                        new NotBlank()
                    ],
                    'dateFormat' => [
                        new NotBlank()
                    ],
                    'enabled' => [
                        new Type('bool')
                    ],
                    'categoriesColumn' => [
                        new NotBlank()
                    ],
                    'familyColumn' => [
                        new NotBlank()
                    ],
                    'groupsColumn' => [
                        new NotBlank()
                    ],
                    'enabledComparison' => [
                        new Type('bool')
                    ],
                    'realTimeVersioning' => [
                        new Type('bool')
                    ],
                ]
            ]
        );
    }

    /**
     * {@inheritdoc}
     */
    public function getFormConfiguration()
    {
        return [
            'filePath' => [
                'options' => [
                    'label' => 'pim_connector.import.filePath.label',
                    'help'  => 'pim_connector.import.filePath.help'
                ]
            ],
            'decimalSeparator' => [
                'type'    => 'choice',
                'options' => [
                    'choices'  => $this->decimalSeparators,
                    'required' => true,
                    'select2'  => true,
                    'label'    => 'pim_connector.export.decimalSeparator.label',
                    'help'     => 'pim_connector.export.decimalSeparator.help'
                ]
            ],
            'dateFormat' => [
                'type'    => 'choice',
                'options' => [
                    'choices'  => $this->dateFormats,
                    'required' => true,
                    'select2'  => true,
                    'label'    => 'pim_connector.export.dateFormat.label',
                    'help'     => 'pim_connector.export.dateFormat.help',
                ]
            ],
            'enabled' => [
                'type'    => 'switch',
                'options' => [
                    'label' => 'pim_connector.import.enabled.label',
                    'help'  => 'pim_connector.import.enabled.help'
                ]
            ],
            'categoriesColumn' => [
                'options' => [
                    'label' => 'pim_connector.import.categoriesColumn.label',
                    'help'  => 'pim_connector.import.categoriesColumn.help'
                ]
            ],
            'familyColumn' => [
                'options' => [
                    'label' => 'pim_connector.import.familyColumn.label',
                    'help'  => 'pim_connector.import.familyColumn.help'
                ]
            ],
            'groupsColumn' => [
                'options' => [
                    'label' => 'pim_connector.import.groupsColumn.label',
                    'help'  => 'pim_connector.import.groupsColumn.help'
                ]
            ],
            'enabledComparison' => [
                'type'    => 'switch',
                'options' => [
                    'label' => 'pim_connector.import.enabledComparison.label',
                    'help'  => 'pim_connector.import.enabledComparison.help'
                ]
            ],
            'realTimeVersioning' => [
                'type'    => 'switch',
                'options' => [
                    'label' => 'pim_connector.import.realTimeVersioning.label',
                    'help'  => 'pim_connector.import.realTimeVersioning.help'
                ]
            ]
        ];
    }

    /**
     * {@inheritdoc}
     */
    public function supports(JobInterface $job)
    {
        return $job->getName() === 'xml_product_import';
    }
}

For further information you can check the following cookbook: How to Create a New Connector

Important

We strongly advise to always try to re-use most of the existing pieces, especially processor and writer, it ensures that all business rules and validation will be properly applied.

Create the Reader

As we don’t have an existing reader which allows to read this kind of files, we’ll write a new one.

The purpose of the reader is to return each item as an array, in the case of XML file, we can have more work to define what is the item.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
<?php

namespace Acme\Bundle\XmlConnectorBundle\Reader\File;

use Akeneo\Component\Batch\Item\FileInvalidItem;
use Akeneo\Component\Batch\Item\FlushableInterface;
use Akeneo\Component\Batch\Item\InvalidItemException;
use Akeneo\Component\Batch\Item\ItemReaderInterface;
use Akeneo\Component\Batch\Model\StepExecution;
use Akeneo\Component\Batch\Step\StepExecutionAwareInterface;
use Pim\Component\Connector\ArrayConverter\ArrayConverterInterface;
use Pim\Component\Connector\Exception\DataArrayConversionException;
use Pim\Component\Connector\Exception\InvalidItemFromViolationsException;

class XmlProductReader implements
    ItemReaderInterface,
    StepExecutionAwareInterface,
    FlushableInterface
{
    /** @var array */
    protected $xml;

    /** @var StepExecution */
    protected $stepExecution;

    /** @var ArrayConverterInterface */
    protected $converter;

    /**
     * @param ArrayConverterInterface $converter
     */
    public function __construct(ArrayConverterInterface $converter)
    {
        $this->converter = $converter;
    }

    public function read()
    {
        if (null === $this->xml) {
            $jobParameters = $this->stepExecution->getJobParameters();
            $filePath = $jobParameters->get('filePath');
            // for example purpose, we should use XML Parser to read line per line
            $this->xml = simplexml_load_file($filePath, 'SimpleXMLIterator');
            $this->xml->rewind();
        }

        if($data = $this->xml->current()) {
            $item = [];
            foreach ($data->attributes() as $attributeName => $attributeValue) {
                $item[$attributeName] = (string) $attributeValue;
            }
            $this->xml->next();

            try {
                $item = $this->converter->convert($item);
            } catch (DataArrayConversionException $e) {
                $this->skipItemFromConversionException($this->xml->current(), $e);
            }

            return $item;
        }

        return null;
    }

    /**
     * {@inheritdoc}
     */
    public function flush()
    {
        $this->xml = null;
    }

    /**
     * {@inheritdoc}
     */
    public function setStepExecution(StepExecution $stepExecution)
    {
        $this->stepExecution = $stepExecution;
    }

    /**
     * @param array                        $item
     * @param DataArrayConversionException $exception
     *
     * @throws InvalidItemException
     * @throws InvalidItemFromViolationsException
     */
    protected function skipItemFromConversionException(array $item, DataArrayConversionException $exception)
    {
        if (null !== $this->stepExecution) {
            $this->stepExecution->incrementSummaryInfo('skip');
        }

        if (null !== $exception->getViolations()) {
            throw new InvalidItemFromViolationsException(
                $exception->getViolations(),
                new FileInvalidItem($item, ($this->stepExecution->getSummaryInfo('read_lines') + 1)),
                [],
                0,
                $exception
            );
        }

        $invalidItem = new FileInvalidItem(
            $item,
            ($this->stepExecution->getSummaryInfo('read_lines') + 1)
        );

        throw new InvalidItemException($exception->getMessage(), $invalidItem, [], 0, $exception);
    }
}

The reader processes the file and iterates to return products line by line and then converts them into the Standard format

This element must be configured with the path of the XML file (an example file is provided in XmlConnectorBundle\Resources\fixtures\products.xml).

Then, we need to define this reader as a service in readers.yml:

1
2
3
4
5
6
7
8
parameters:
    acme_xml_connector.reader.file.xml_product.class: Acme\Bundle\XmlConnectorBundle\Reader\File\XmlProductReader

services:
    acme_xml_connector.reader.file.xml_product:
        class: '%acme_xml_connector.reader.file.xml_product.class%'
        arguments:
            - '@pim_connector.array_converter.flat_to_standard.product'

And we introduce the following extension to load the services files in configuration:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
<?php

namespace Acme\Bundle\XmlConnectorBundle\DependencyInjection;

use Symfony\Component\HttpKernel\DependencyInjection\Extension;
use Symfony\Component\DependencyInjection\ContainerBuilder;
use Symfony\Component\DependencyInjection\Loader;
use Symfony\Component\Config\FileLocator;

class AcmeXmlConnectorExtension extends Extension
{
    public function load(array $configs, ContainerBuilder $container)
    {
        $loader = new Loader\YamlFileLoader($container, new FileLocator(__DIR__.'/../Resources/config'));
        $loader->load('jobs.yml');
        $loader->load('job_parameters.yml');
        $loader->load('steps.yml');
        $loader->load('readers.yml');
    }
}

Translate Job and Step labels in the UI

Behind the scene, the service Pim\Bundle\ImportExportBundle\JobLabel\TranslatedLabelProvider provides translated Job and Step labels to be used in the UI.

This service uses the following conventions:
  • for a job label, given a %%jobName%%, “batch_jobs.%%jobName%%.label”
  • for a step label, given a %%jobName%% and a %%stepName%%, “batch_jobs.%%jobName%%.%%stepName%%.label”

Create a file Resources/translations/messages.en.yml in the Bundle to translate label keys.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
acme_xml_connector:
    jobs:
        xml_product_import:
            title: Product Import XML
            import:
                title: Product Import Step
    steps:
        product_processor.title: Product processor
        import.filePath:
            label: File path
            help: Path of the XML file

batch_jobs:
    xml_product_import:
        import.label: Product Import XML

Use the new Connector

Now if you refresh the cache, the new export can be found under Extract > Import profiles > Create import profile.

You can run the job from the UI or you can use following command:

php app/console akeneo:batch:job my_job_code