How to Create a Specific Connector

The foundations of connector creation has been covered in the previous chapter (cf How to Create a New Connector). With the following hand-on practice, we will create our own specific connector.

To stay focus on the main concepts, we implement the simplest connector as possible by avoiding to use too many existing elements.

Our use case is to import new products from the following XML file :

1
2
3
4
5
6
<?xml version="1.0" encoding="UTF-8"?>
<products>
    <product sku="sku-1" title="my title 1"/>
    <product sku="sku-2" title="my title 2"/>
    <product sku="sku-3" title="my title 3"/>
</products>

Note

The code inside this cookbook entry is available in the src directory, you can clone pim-doc then use a symlink to make the Acme bundle available in the src/.

Create our Connector

Create a new bundle:

1
2
3
4
5
6
7
8
9
<?php

namespace Acme\Bundle\SpecificConnectorBundle;

use Akeneo\Bundle\BatchBundle\Connector\Connector;

class AcmeSpecificConnectorBundle extends Connector
{
}

Register the bundle in AppKernel:

1
2
3
4
5
6
public function registerBundles()
{
    // ...
        new Acme\Bundle\SpecificConnectorBundle\AcmeSpecificConnectorBundle(),
    // ...
}

Configure our Job

Configure a job in Resources/config/batch_jobs.yml:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
connector:
    name: Specific Connector
    jobs:
        specific_product_import:
            title: acme_specific_connector.jobs.specific_product_import.title
            type:  import
            steps:
                import:
                   title: acme_specific_connector.jobs.specific_product_import.import.title
                   services:
                       reader:    acme_specific_connector.reader.file.xml_product
                       processor: acme_specific_connector.processor.product
                       writer:    acme_specific_connector.writer.orm_product

Here we create an import job which contains a single step import.

The default used step is Akeneo\Bundle\BatchBundle\Step\ItemStep.

An item step is configured with 3 elements, a reader, a processor and a writer.

As seen previously, we can use existing elements, but in this case, we will create our own elements so you will be able to do it by yourself when needed.

During the development, a good practise is to use dummy elements as in this example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
connector:
    name: Demo Connector
    jobs:
        demo_dummy_export:
            title: pim_base_connector.jobs.demo_dummy_export.title
            type:  export
            steps:
                export:
                    title: pim_base_connector.jobs.demo_dummy_export.export.title
                    services:
                        reader:    pim_base_connector.reader.dummy
                        processor: pim_base_connector.processor.dummy
                        writer:    pim_base_connector.writer.dummy

This practice allows to focus on developing each part, element per element, and always be able to run the whole process during the development.

Create our Reader

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
<?php

namespace Acme\Bundle\SpecificConnectorBundle\Reader\File;

use Symfony\Component\Yaml\Yaml;
use Akeneo\Bundle\BatchBundle\Item\ItemReaderInterface;
use Pim\Bundle\BaseConnectorBundle\Reader\File\FileReader;

class XmlProductReader extends FileReader implements ItemReaderInterface
{
    protected $xml;

    public function read()
    {
        if (null === $this->xml) {
            // for exemple purpose, we should use XML Parser to read line per line
            $this->xml = simplexml_load_file($this->filePath, 'SimpleXMLIterator');
            $this->xml->rewind();
        }

        if ($data = $this->xml->current()) {
            $this->xml->next();

            return $data;
        }

        return null;
    }

    public function getConfigurationFields()
    {
        return array(
            'filePath' => array(
                'options' => array(
                    'label' => 'acme_specific_connector.steps.import.filePath.label',
                    'help'  => 'acme_specific_connector.steps.import.filePath.help'
                )
            ),
        );
    }
}

Our element reads the file and iterate to return products line per line.

This element must be configured with the path of the xml file.

Note

It is recommended to provide a label option (otherwise the attribute name will be used, which can lead to translation collision).

The help option allows you to display a hint next to the field in the job edition form.

Then we need to define this reader as a service in readers.yml :

1
2
3
4
5
6
parameters:
    acme_specific_connector.reader.file.xml_product.class: Acme\Bundle\SpecificConnectorBundle\Reader\File\XmlProductReader

services:
    acme_specific_connector.reader.file.xml_product:
        class: '%acme_specific_connector.reader.file.xml_product.class%'

And we introduce the following extension to load the services files in configuration :

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
<?php

namespace Acme\Bundle\SpecificConnectorBundle\DependencyInjection;

use Symfony\Component\HttpKernel\DependencyInjection\Extension;
use Symfony\Component\DependencyInjection\ContainerBuilder;
use Symfony\Component\DependencyInjection\Loader;
use Symfony\Component\Config\FileLocator;

class AcmeSpecificConnectorExtension extends Extension
{
    public function load(array $configs, ContainerBuilder $container)
    {
        $loader = new Loader\YamlFileLoader($container, new FileLocator(__DIR__.'/../Resources/config'));
        $loader->load('readers.yml');
        $loader->load('processors.yml');
        $loader->load('writers.yml');
    }
}

Create our Processor

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
<?php

namespace Acme\Bundle\SpecificConnectorBundle\Processor;

use Akeneo\Bundle\BatchBundle\Entity\StepExecution;
use Akeneo\Bundle\BatchBundle\Item\AbstractConfigurableStepElement;
use Akeneo\Bundle\BatchBundle\Item\InvalidItemException;
use Akeneo\Bundle\BatchBundle\Item\ItemProcessorInterface;
use Akeneo\Bundle\BatchBundle\Step\StepExecutionAwareInterface;
use Pim\Bundle\CatalogBundle\Manager\ProductManager;

class ProductProcessor extends AbstractConfigurableStepElement implements
    ItemProcessorInterface,
    StepExecutionAwareInterface
{
    /** @var StepExecution */
    protected $stepExecution;

    /** @var ProductManager */
    protected $productManager;

    public function __construct($manager)
    {
        $this->productManager = $manager;
    }

    public function process($item)
    {
        $sku       = $item['sku'];
        $attribute = $this->productManager->getIdentifierAttribute();
        $product   = $this->productManager->findByIdentifier($sku);

        if (!$product) {
            $product = $this->productManager->createProduct();
            $value   = $this->productManager->createProductValue();
            $value->setAttribute($attribute);
            $value->setData($sku);
            $product->addValue($value);
            $this->stepExecution->incrementSummaryInfo('create');

            return $product;

        } else {

            $data = current(((array) $item));
            $this->stepExecution->incrementSummaryInfo('skip');

            throw new InvalidItemException(sprintf('Skip the existing %s product', $sku), $data);
        }
    }

    public function getConfigurationFields()
    {
        return array();
    }

    public function setStepExecution(StepExecution $stepExecution)
    {
        $this->stepExecution = $stepExecution;
    }
}

Our processor receives each item passed by our reader and converts it to product.

If the product is already known, we skip the item. Of course, in the case of production import, we will update the product as well by changing the properties of the loaded product.

We create a minimal product, to go further, you can take a look on How to Programmatically Manipulate Products

This processor needs to know the product manager that is injected in the following service definition in processors.yml :

1
2
3
4
5
6
7
8
parameters:
    acme_specific_connector.processor.product.class: Acme\Bundle\SpecificConnectorBundle\Processor\ProductProcessor

services:
    acme_specific_connector.processor.product:
        class: '%acme_specific_connector.processor.product.class%'
        arguments:
            - '@pim_catalog.manager.product'

Add Details in Summary

The execution details page presents a summary and the errors encountered during the execution. Your own information and counter can be easily added with following methods:

$this->stepExecution->incrementSummaryInfo('skip');
$this->stepExecution->incrementSummaryInfo('mycounter');
$this->stepExecution->addSummaryInfo('myinfo', 'my value');

Skip Erroneous Data

To skip the current line and go to the next one, you need to throw the following exception:

throw new InvalidItemException($message, $item);

Note

You can use this exception in reader, processor or writer, it will be handled by the ItemStep. Other exceptions will stop the whole job.

Create our Writer

Finaly we define our product writer :

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
<?php

namespace Acme\Bundle\SpecificConnectorBundle\Writer\ORM;

use Doctrine\ORM\EntityManager;

use Akeneo\Bundle\BatchBundle\Entity\StepExecution;
use Akeneo\Bundle\BatchBundle\Item\ItemWriterInterface;
use Akeneo\Bundle\BatchBundle\Item\AbstractConfigurableStepElement;
use Akeneo\Bundle\BatchBundle\Step\StepExecutionAwareInterface;
use Pim\Bundle\CatalogBundle\Manager\ProductManager;

class ProductWriter extends AbstractConfigurableStepElement implements
    ItemWriterInterface,
    StepExecutionAwareInterface
{
    /** @var StepExecution */
    protected $stepExecution;

    /** @var ProductManager */
    protected $productManager;

    public function __construct($manager)
    {
        $this->productManager = $manager;
    }

    public function write(array $items)
    {
        foreach ($items as $product) {
            $this->productManager->save($product);
            $this->stepExecution->incrementSummaryInfo('save');
        }
    }

    public function getConfigurationFields()
    {
        return array();
    }

    public function setStepExecution(StepExecution $stepExecution)
    {
        $this->stepExecution = $stepExecution;
    }
}

The writer element receives an array of items, as a writer can be able to do some mass writing that could be more efficient than writing item one by one.

In this example, the items are products and the writer persist them.

In order to do that, this writer needs to know the product manager that is injected in the following service definition in writers.yml :

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
services: ~

parameters:
    acme_specific_connector.writer.orm_product.class: Acme\Bundle\SpecificConnectorBundle\Writer\ORM\ProductWriter

services:
    acme_specific_connector.writer.orm_product:
        class: '%acme_specific_connector.writer.orm_product.class%'
        arguments:
            - '@pim_catalog.manager.product'

Note

Keep in mind that for example purpose, we define by hand our own reader, processor, writer. In fact, we should use existing elements from the Base Connector. We’ll see how to re-use and customize existing elements in following examples.

Use our new Connector

Now if you refresh the cache, your new export can be found under Extract > Import profiles > Create import profile.

You can run the job from UI or you can use following command:

php app/console akeneo:batch:job my_job_code

Create a Custom Step

The default ItemStep answers to the majority of cases but sometimes you need to create more custom logic with no need for a reader, processor or writer.

For instance, at the end of an export you may want send a custom email, copy the result to a FTP server or call a specific url to report the result.

Let’s take this last example to illustrate How to Create a Custom Step