How to Create a Specific Connector¶
The foundations of connector creation has been covered in the previous chapter (cf How to Create a New Connector). With the following hand-on practice, we will create our own specific connector.
To stay focus on the main concepts, we implement the simplest connector as possible by avoiding to use too many existing elements.
Our use case is to import new products from the following XML file :
1 2 3 4 5 6 | <?xml version="1.0" encoding="UTF-8"?>
<products>
<product sku="sku-1" title="my title 1"/>
<product sku="sku-2" title="my title 2"/>
<product sku="sku-3" title="my title 3"/>
</products>
|
Note
The code inside this cookbook entry is available in the src directory, you can clone pim-doc then use a symlink to make the Acme bundle available in the src/.
Create our Connector¶
Create a new bundle:
1 2 3 4 5 6 7 8 9 | <?php
namespace Acme\Bundle\SpecificConnectorBundle;
use Akeneo\Bundle\BatchBundle\Connector\Connector;
class AcmeSpecificConnectorBundle extends Connector
{
}
|
Register the bundle in AppKernel:
1 2 3 4 5 6 | public function registerBundles()
{
// ...
new Acme\Bundle\SpecificConnectorBundle\AcmeSpecificConnectorBundle(),
// ...
}
|
Configure our Job¶
Configure a job in Resources/config/batch_jobs.yml
:
1 2 3 4 5 6 7 8 9 10 11 12 13 | connector:
name: Specific Connector
jobs:
specific_product_import:
title: acme_specific_connector.jobs.specific_product_import.title
type: import
steps:
import:
title: acme_specific_connector.jobs.specific_product_import.import.title
services:
reader: acme_specific_connector.reader.file.xml_product
processor: acme_specific_connector.processor.product
writer: acme_specific_connector.writer.orm_product
|
Here we create an import job which contains a single step import.
The default used step is Akeneo\Bundle\BatchBundle\Step\ItemStep
.
An item step is configured with 3 elements, a reader, a processor and a writer.
As seen previously, we can use existing elements, but in this case, we will create our own elements so you will be able to do it by yourself when needed.
During the development, a good practise is to use dummy elements as in this example:
1 2 3 4 5 6 7 8 9 10 11 12 13 | connector:
name: Demo Connector
jobs:
demo_dummy_export:
title: pim_base_connector.jobs.demo_dummy_export.title
type: export
steps:
export:
title: pim_base_connector.jobs.demo_dummy_export.export.title
services:
reader: pim_base_connector.reader.dummy
processor: pim_base_connector.processor.dummy
writer: pim_base_connector.writer.dummy
|
This practice allows to focus on developing each part, element per element, and always be able to run the whole process during the development.
Create our Reader¶
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 | <?php
namespace Acme\Bundle\SpecificConnectorBundle\Reader\File;
use Symfony\Component\Yaml\Yaml;
use Akeneo\Bundle\BatchBundle\Item\ItemReaderInterface;
use Pim\Bundle\BaseConnectorBundle\Reader\File\FileReader;
class XmlProductReader extends FileReader implements ItemReaderInterface
{
protected $xml;
public function read()
{
if (null === $this->xml) {
// for exemple purpose, we should use XML Parser to read line per line
$this->xml = simplexml_load_file($this->filePath, 'SimpleXMLIterator');
$this->xml->rewind();
}
if ($data = $this->xml->current()) {
$this->xml->next();
return $data;
}
return null;
}
public function getConfigurationFields()
{
return array(
'filePath' => array(
'options' => array(
'label' => 'acme_specific_connector.steps.import.filePath.label',
'help' => 'acme_specific_connector.steps.import.filePath.help'
)
),
);
}
}
|
Our element reads the file and iterate to return products line per line.
This element must be configured with the path of the xml file.
Note
It is recommended to provide a label option (otherwise the attribute name will be used, which can lead to translation collision).
The help option allows you to display a hint next to the field in the job edition form.
Then we need to define this reader as a service in readers.yml :
1 2 3 4 5 6 | parameters:
acme_specific_connector.reader.file.xml_product.class: Acme\Bundle\SpecificConnectorBundle\Reader\File\XmlProductReader
services:
acme_specific_connector.reader.file.xml_product:
class: '%acme_specific_connector.reader.file.xml_product.class%'
|
And we introduce the following extension to load the services files in configuration :
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 | <?php
namespace Acme\Bundle\SpecificConnectorBundle\DependencyInjection;
use Symfony\Component\HttpKernel\DependencyInjection\Extension;
use Symfony\Component\DependencyInjection\ContainerBuilder;
use Symfony\Component\DependencyInjection\Loader;
use Symfony\Component\Config\FileLocator;
class AcmeSpecificConnectorExtension extends Extension
{
public function load(array $configs, ContainerBuilder $container)
{
$loader = new Loader\YamlFileLoader($container, new FileLocator(__DIR__.'/../Resources/config'));
$loader->load('readers.yml');
$loader->load('processors.yml');
$loader->load('writers.yml');
}
}
|
Create our Processor¶
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 | <?php
namespace Acme\Bundle\SpecificConnectorBundle\Processor;
use Akeneo\Bundle\BatchBundle\Entity\StepExecution;
use Akeneo\Bundle\BatchBundle\Item\AbstractConfigurableStepElement;
use Akeneo\Bundle\BatchBundle\Item\InvalidItemException;
use Akeneo\Bundle\BatchBundle\Item\ItemProcessorInterface;
use Akeneo\Bundle\BatchBundle\Step\StepExecutionAwareInterface;
use Pim\Bundle\CatalogBundle\Manager\ProductManager;
class ProductProcessor extends AbstractConfigurableStepElement implements
ItemProcessorInterface,
StepExecutionAwareInterface
{
/** @var StepExecution */
protected $stepExecution;
/** @var ProductManager */
protected $productManager;
public function __construct($manager)
{
$this->productManager = $manager;
}
public function process($item)
{
$sku = $item['sku'];
$attribute = $this->productManager->getIdentifierAttribute();
$product = $this->productManager->findByIdentifier($sku);
if (!$product) {
$product = $this->productManager->createProduct();
$value = $this->productManager->createProductValue();
$value->setAttribute($attribute);
$value->setData($sku);
$product->addValue($value);
$this->stepExecution->incrementSummaryInfo('create');
return $product;
} else {
$data = current(((array) $item));
$this->stepExecution->incrementSummaryInfo('skip');
throw new InvalidItemException(sprintf('Skip the existing %s product', $sku), $data);
}
}
public function getConfigurationFields()
{
return array();
}
public function setStepExecution(StepExecution $stepExecution)
{
$this->stepExecution = $stepExecution;
}
}
|
Our processor receives each item passed by our reader and converts it to product.
If the product is already known, we skip the item. Of course, in the case of production import, we will update the product as well by changing the properties of the loaded product.
We create a minimal product, to go further, you can take a look on How to Programmatically Manipulate Products
This processor needs to know the product manager that is injected in the following service definition in processors.yml :
1 2 3 4 5 6 7 8 | parameters:
acme_specific_connector.processor.product.class: Acme\Bundle\SpecificConnectorBundle\Processor\ProductProcessor
services:
acme_specific_connector.processor.product:
class: '%acme_specific_connector.processor.product.class%'
arguments:
- '@pim_catalog.manager.product'
|
Add Details in Summary¶
The execution details page presents a summary and the errors encountered during the execution. Your own information and counter can be easily added with following methods:
$this->stepExecution->incrementSummaryInfo('skip');
$this->stepExecution->incrementSummaryInfo('mycounter');
$this->stepExecution->addSummaryInfo('myinfo', 'my value');
Skip Erroneous Data¶
To skip the current line and go to the next one, you need to throw the following exception:
throw new InvalidItemException($message, $item);
Note
You can use this exception in reader, processor or writer, it will be handled by the ItemStep. Other exceptions will stop the whole job.
Create our Writer¶
Finaly we define our product writer :
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 | <?php
namespace Acme\Bundle\SpecificConnectorBundle\Writer\ORM;
use Doctrine\ORM\EntityManager;
use Akeneo\Bundle\BatchBundle\Entity\StepExecution;
use Akeneo\Bundle\BatchBundle\Item\ItemWriterInterface;
use Akeneo\Bundle\BatchBundle\Item\AbstractConfigurableStepElement;
use Akeneo\Bundle\BatchBundle\Step\StepExecutionAwareInterface;
use Pim\Bundle\CatalogBundle\Manager\ProductManager;
class ProductWriter extends AbstractConfigurableStepElement implements
ItemWriterInterface,
StepExecutionAwareInterface
{
/** @var StepExecution */
protected $stepExecution;
/** @var ProductManager */
protected $productManager;
public function __construct($manager)
{
$this->productManager = $manager;
}
public function write(array $items)
{
foreach ($items as $product) {
$this->productManager->save($product);
$this->stepExecution->incrementSummaryInfo('save');
}
}
public function getConfigurationFields()
{
return array();
}
public function setStepExecution(StepExecution $stepExecution)
{
$this->stepExecution = $stepExecution;
}
}
|
The writer element receives an array of items, as a writer can be able to do some mass writing that could be more efficient than writing item one by one.
In this example, the items are products and the writer persist them.
In order to do that, this writer needs to know the product manager that is injected in the following service definition in writers.yml :
1 2 3 4 5 6 7 8 9 10 | services: ~
parameters:
acme_specific_connector.writer.orm_product.class: Acme\Bundle\SpecificConnectorBundle\Writer\ORM\ProductWriter
services:
acme_specific_connector.writer.orm_product:
class: '%acme_specific_connector.writer.orm_product.class%'
arguments:
- '@pim_catalog.manager.product'
|
Note
Keep in mind that for example purpose, we define by hand our own reader, processor, writer. In fact, we should use existing elements from the Base Connector. We’ll see how to re-use and customize existing elements in following examples.
Use our new Connector¶
Now if you refresh the cache, your new export can be found under Extract > Import profiles > Create import profile.
You can run the job from UI or you can use following command:
php app/console akeneo:batch:job my_job_code
Create a Custom Step¶
The default ItemStep answers to the majority of cases but sometimes you need to create more custom logic with no need for a reader, processor or writer.
For instance, at the end of an export you may want send a custom email, copy the result to a FTP server or call a specific url to report the result.
Let’s take this last example to illustrate How to Create a Custom Step