How to import Products from a XML file¶
Prerequisite¶
The basics of a connector creation have been covered in the previous chapter (cf. How to create a new Connector). With the following hands-on practice, we will create our own specific connector.
We assume that we’re using a standard edition with the icecat_demo_dev data set, sku and name already exist as attributes in the PIM and the family is also an existing property.
Overview¶
In this cookbook, we will create a brand new XML connector to import our products using XML files.
For a recap, here is the process of a product import job execution:
Starting the job and reading the first product
The job opens the file to import, it reads the first product in the file and converts it into a standard format.
If an error or a warning is thrown during this step, the product is marked as invalid.
Process the product and check the product values
If no errors have been thrown in the previous step, the read and converted product is then processed by a product processor.
If an error is thrown while processing, the product is marked as invalid.
Note
At this point, an error could be that the the family code does not exist or that the currency set for a price attribute does not match the currency configured in the PIM for this attribute.
Save the product in database
The processed product is written in the database using a product writer.
Collect the invalid products found and export them into a separate file
When all products have been read, processed and written into the database, the job collects all the errors found in the file at each step and writes them back into a separate file of invalid items.
In this cookbook, our use case is to import new products from the following XML file products.xml:
1<?xml version="1.0" encoding="UTF-8"?>
2<products>
3 <product sku="sku-1" name="my name 1" family="camcorders" enabled="1"/>
4 <product sku="sku-2" name="my name 2" family="camcorders" enabled="1"/>
5 <product sku="sku-3" name="my name 3" family="camcorders" enabled="1"/>
6 <product sku="sku-4" name="my name 4" family="wrong_family_code" enabled="0"/>
7 <product sku="" name="my name 5" family="camcorders" enabled="1"/>
8</products>
To stay focused on the main concepts, we will implement the simplest connector possible by avoiding to use too many existing elements.
Note
The code inside this cookbook entry is available in the src directory, you can clone pim-docs (https://github.com/akeneo/pim-docs) and use a symlink to make the Acme bundle available in the src/.
Create the Connector¶
Create a new bundle:
1<?php
2
3namespace Acme\Bundle\XmlConnectorBundle;
4
5use Symfony\Component\HttpKernel\Bundle\Bundle;
6
7class AcmeXmlConnectorBundle extends Bundle
8{
9}
Register the bundle in AppKernel:
1public function registerProjectBundles()
2{
3 return [
4 // your app bundles should be registered here
5 new Acme\Bundle\AppBundle\AcmeAppBundle(),
6 new Acme\Bundle\XmlConnectorBundle\AcmeXmlConnectorBundle()
7 ];
8}
Configure the Job¶
Configure the job in Resources/config/jobs.yml:
1 acme_xml_connector.job.xml_product_import:
2 class: '%pim_connector.job.simple_job.class%'
3 arguments:
4 - 'xml_product_import'
5 - '@event_dispatcher'
6 - '@akeneo_batch.job_repository'
7 -
8 - '@acme_xml_connector.step.xml_product_import.import'
9 tags:
10 - { name: akeneo_batch.job, connector: 'Akeneo XML Connector', type: '%pim_connector.job.import_type%' }
The default step is Akeneo\Tool\Component\Batch\Step\ItemStep.
An item step is configured with 3 elements, a reader, a processor and a writer.
Here is the definition of the Step:
1 acme_xml_connector.step.xml_product_import.import:
2 class: '%pim_connector.step.item_step.class%'
3 arguments:
4 - 'import'
5 - '@event_dispatcher'
6 - '@akeneo_batch.job_repository'
7 - '@acme_xml_connector.reader.file.xml_product'
8 - '@pim_connector.processor.denormalization.product'
9 - '@pim_connector.writer.database.product'
10 - 10
Here, we’ll use a custom reader acme_xml_connector.reader.file.xml_product but we’ll continue to use the default processor and writer.
Then you will need to add the job parameters classes (they define the job configuration, job constraints and job default values):
1parameters:
2 acme_xml_connector.job.job_parameters.simple_xml_import.class: Acme\Bundle\XmlConnectorBundle\Job\JobParameters\SimpleXmlImport
3
4services:
5 acme_xml_connector.job.job_parameters.simple_xml_product_import:
6 class: '%acme_xml_connector.job.job_parameters.simple_xml_import.class%'
7 arguments:
8 - '%pim_catalog.localization.decimal_separators%'
9 - '%pim_catalog.localization.date_formats%'
10 tags:
11 - { name: akeneo_batch.job.job_parameters.constraint_collection_provider }
12 - { name: akeneo_batch.job.job_parameters.default_values_provider }
13
14 acme_xml_connector.job.job_parameters.provider.simple_xml_product_import:
15 class: 'Akeneo\Platform\Bundle\ImportExportBundle\Provider\Form\JobInstanceFormProvider'
16 arguments:
17 -
18 xml_product_import: pim-job-instance-xml-product-import
19 tags:
20 - { name: pim_enrich.provider.form }
1<?php
2
3namespace Acme\Bundle\XmlConnectorBundle\Job\JobParameters;
4
5use Akeneo\Tool\Component\Batch\Job\JobInterface;
6use Akeneo\Tool\Component\Batch\Job\JobParameters\ConstraintCollectionProviderInterface;
7use Akeneo\Tool\Component\Batch\Job\JobParameters\DefaultValuesProviderInterface;
8use Akeneo\Tool\Component\Localization\Localizer\LocalizerInterface;
9use Akeneo\Pim\Enrichment\Component\Product\Validator\Constraints\FileExtension;
10use Symfony\Component\Validator\Constraints\Collection;
11use Symfony\Component\Validator\Constraints\IsTrue;
12use Symfony\Component\Validator\Constraints\NotBlank;
13use Symfony\Component\Validator\Constraints\Type;
14
15class SimpleXmlImport implements
16 DefaultValuesProviderInterface,
17 ConstraintCollectionProviderInterface
18{
19 /** @var ConstraintCollectionProviderInterface */
20 protected $constraintCollectionProvider;
21
22 /** @var DefaultValuesProviderInterface */
23 protected $defaultValuesProvider;
24
25 /**
26 * {@inheritdoc}
27 */
28 public function getDefaultValues()
29 {
30 return [
31 'filePath' => null,
32 'uploadAllowed' => true,
33 'dateFormat' => LocalizerInterface::DEFAULT_DATE_FORMAT,
34 'decimalSeparator' => LocalizerInterface::DEFAULT_DECIMAL_SEPARATOR,
35 'enabled' => true,
36 'categoriesColumn' => 'categories',
37 'familyColumn' => 'family',
38 'groupsColumn' => 'groups',
39 'enabledComparison' => true,
40 'realTimeVersioning' => true,
41 'invalid_items_file_format' => 'xml',
42 ];
43 }
44
45 /**
46 * {@inheritdoc}
47 */
48 public function getConstraintCollection()
49 {
50 return new Collection(
51 [
52 'fields' => [
53 'filePath' => [
54 new NotBlank(['groups' => ['Execution', 'UploadExecution']]),
55 new FileExtension(
56 [
57 'allowedExtensions' => ['xml', 'zip'],
58 'groups' => ['Execution', 'UploadExecution']
59 ]
60 )
61 ],
62 'uploadAllowed' => [
63 new Type('bool'),
64 new IsTrue(['groups' => 'UploadExecution']),
65 ],
66 'decimalSeparator' => [
67 new NotBlank()
68 ],
69 'dateFormat' => [
70 new NotBlank()
71 ],
72 'enabled' => [
73 new Type('bool')
74 ],
75 'categoriesColumn' => [
76 new NotBlank()
77 ],
78 'familyColumn' => [
79 new NotBlank()
80 ],
81 'groupsColumn' => [
82 new NotBlank()
83 ],
84 'enabledComparison' => [
85 new Type('bool')
86 ],
87 'realTimeVersioning' => [
88 new Type('bool')
89 ],
90 'invalid_items_file_format' => [
91 new Type('string')
92 ]
93 ]
94 ]
95 );
96 }
97
98 /**
99 * {@inheritdoc}
100 */
101 public function supports(JobInterface $job)
102 {
103 return $job->getName() === 'xml_product_import';
104 }
105}
1extensions:
2 pim-job-instance-xml-product-import-edit:
3 module: pim/form/common/edit-form
4
5 pim-job-instance-xml-product-import-edit-cache-invalidator:
6 module: pim/cache-invalidator
7 parent: pim-job-instance-xml-product-import-edit
8 position: 1000
9
10 pim-job-instance-xml-product-import-edit-tabs:
11 module: pim/form/common/form-tabs
12 parent: pim-job-instance-xml-product-import-edit
13 targetZone: content
14 position: 100
15
16 pim-job-instance-xml-product-import-edit-properties:
17 module: pim/job/common/edit/properties
18 parent: pim-job-instance-xml-product-import-edit-tabs
19 aclResourceId: pim_importexport_export_profile_property_edit
20 targetZone: container
21 position: 100
22 config:
23 tabTitle: pim_enrich.form.job_instance.tab.properties.title
24 tabCode: pim-job-instance-properties
25
26 pim-job-instance-xml-product-import-edit-history:
27 module: pim/common/tab/history
28 parent: pim-job-instance-xml-product-import-edit-tabs
29 targetZone: container
30 aclResourceId: pim_importexport_import_profile_history
31 position: 120
32 config:
33 class: Akeneo\Tool\Component\Batch\Model\JobInstance
34 title: pim_enrich.form.job_instance.tab.history.title
35 tabCode: pim-job-instance-history
36
37 pim-job-instance-xml-product-import-edit-properties-code:
38 module: pim/job/common/edit/field/text
39 parent: pim-job-instance-xml-product-import-edit-properties
40 position: 100
41 targetZone: properties
42 config:
43 fieldCode: code
44 label: pim_enrich.form.job_instance.tab.properties.code.title
45 readOnly: true
46
47 pim-job-instance-xml-product-import-edit-properties-label:
48 module: pim/job/common/edit/field/text
49 parent: pim-job-instance-xml-product-import-edit-properties
50 position: 110
51 targetZone: properties
52 config:
53 fieldCode: label
54 label: pim_enrich.form.job_instance.tab.properties.label.title
55
56 pim-job-instance-xml-product-import-edit-properties-file-path:
57 module: pim/job/common/edit/field/text
58 parent: pim-job-instance-xml-product-import-edit-properties
59 position: 120
60 targetZone: global-settings
61 config:
62 fieldCode: configuration.filePath
63 readOnly: false
64 label: pim_enrich.form.job_instance.tab.properties.file_path.title
65 tooltip: pim_enrich.form.job_instance.tab.properties.file_path.help
66
67 pim-job-instance-xml-product-import-edit-properties-file-upload:
68 module: pim/job/common/edit/field/switch
69 parent: pim-job-instance-xml-product-import-edit-properties
70 position: 130
71 targetZone: global-settings
72 config:
73 fieldCode: configuration.uploadAllowed
74 readOnly: false
75 label: pim_enrich.form.job_instance.tab.properties.upload_allowed.title
76 tooltip: pim_enrich.form.job_instance.tab.properties.upload_allowed.help
77 readOnly: false
78
79 pim-job-instance-xml-product-import-edit-properties-decimal-separator:
80 module: pim/job/common/edit/field/decimal-separator
81 parent: pim-job-instance-xml-product-import-edit-properties
82 position: 170
83 targetZone: global-settings
84 config:
85 fieldCode: configuration.decimalSeparator
86 readOnly: false
87 label: pim_enrich.form.job_instance.tab.properties.decimal_separator.title
88 tooltip: pim_enrich.form.job_instance.tab.properties.decimal_separator.help
89
90 pim-job-instance-xml-product-import-edit-properties-date-format:
91 module: pim/job/product/edit/field/date-format
92 parent: pim-job-instance-xml-product-import-edit-properties
93 position: 180
94 targetZone: global-settings
95 config:
96 fieldCode: configuration.dateFormat
97 readOnly: false
98 label: pim_enrich.form.job_instance.tab.properties.date_format.title
99 tooltip: pim_enrich.form.job_instance.tab.properties.date_format.help
100
101 pim-job-instance-xml-product-import-edit-properties-enabled:
102 module: pim/job/common/edit/field/switch
103 parent: pim-job-instance-xml-product-import-edit-properties
104 position: 190
105 targetZone: global-settings
106 config:
107 fieldCode: configuration.enabled
108 readOnly: false
109 label: pim_enrich.form.job_instance.tab.properties.enabled.title
110 tooltip: pim_enrich.form.job_instance.tab.properties.enabled.help
111
112 pim-job-instance-xml-product-import-edit-properties-categories-column:
113 module: pim/job/common/edit/field/text
114 parent: pim-job-instance-xml-product-import-edit-properties
115 position: 200
116 targetZone: global-settings
117 config:
118 fieldCode: configuration.categoriesColumn
119 readOnly: false
120 label: pim_enrich.form.job_instance.tab.properties.categories_column.title
121 tooltip: pim_enrich.form.job_instance.tab.properties.categories_column.help
122
123 pim-job-instance-xml-product-import-edit-properties-family-column:
124 module: pim/job/common/edit/field/text
125 parent: pim-job-instance-xml-product-import-edit-properties
126 position: 210
127 targetZone: global-settings
128 config:
129 fieldCode: configuration.familyColumn
130 readOnly: false
131 label: pim_enrich.form.job_instance.tab.properties.family_column.title
132 tooltip: pim_enrich.form.job_instance.tab.properties.family_column.help
133
134 pim-job-instance-xml-product-import-edit-properties-groups-column:
135 module: pim/job/common/edit/field/text
136 parent: pim-job-instance-xml-product-import-edit-properties
137 position: 220
138 targetZone: global-settings
139 config:
140 fieldCode: configuration.groupsColumn
141 readOnly: false
142 label: pim_enrich.form.job_instance.tab.properties.groups_column.title
143 tooltip: pim_enrich.form.job_instance.tab.properties.groups_column.help
144
145 pim-job-instance-xml-product-import-edit-properties-enabled-comparison:
146 module: pim/job/common/edit/field/switch
147 parent: pim-job-instance-xml-product-import-edit-properties
148 position: 230
149 targetZone: global-settings
150 config:
151 fieldCode: configuration.enabledComparison
152 readOnly: false
153 label: pim_enrich.form.job_instance.tab.properties.enabled_comparison.title
154 tooltip: pim_enrich.form.job_instance.tab.properties.enabled_comparison.help
155
156 pim-job-instance-xml-product-import-edit-properties-real-time-versioning:
157 module: pim/job/common/edit/field/switch
158 parent: pim-job-instance-xml-product-import-edit-properties
159 position: 240
160 targetZone: global-settings
161 config:
162 fieldCode: configuration.realTimeVersioning
163 readOnly: false
164 label: pim_enrich.form.job_instance.tab.properties.real_time_versioning.title
165 tooltip: pim_enrich.form.job_instance.tab.properties.real_time_versioning.help
166
167 pim-job-instance-xml-product-import-edit-label:
168 module: pim/job/common/edit/label
169 parent: pim-job-instance-xml-product-import-edit
170 targetZone: title
171 position: 100
172
173 pim-job-instance-xml-product-import-edit-meta:
174 module: pim/job/common/edit/meta
175 parent: pim-job-instance-xml-product-import-edit
176 targetZone: meta
177 position: 100
178
179 pim-job-instance-xml-product-import-edit-back-to-grid:
180 module: pim/form/common/back-to-grid
181 parent: pim-job-instance-xml-product-import-edit
182 targetZone: back
183 aclResourceId: pim_importexport_import_profile_index
184 position: 80
185 config:
186 backUrl: pim_importexport_import_profile_index
187
188 pim-job-instance-xml-product-import-edit-delete:
189 module: pim/job/import/edit/delete
190 parent: pim-job-instance-xml-product-import-edit
191 targetZone: buttons
192 aclResourceId: pim_importexport_import_profile_remove
193 position: 100
194 config:
195 trans:
196 title: confirmation.remove.job_instance
197 content: pim_enrich.confirmation.delete_item
198 success: flash.job_instance.removed
199 failed: error.removing.job_instance
200 redirect: pim_importexport_import_profile_index
201
202 pim-job-instance-xml-product-import-edit-save-buttons:
203 module: pim/form/common/save-buttons
204 parent: pim-job-instance-xml-product-import-edit
205 targetZone: buttons
206 position: 120
207
208 pim-job-instance-xml-product-import-edit-save:
209 module: pim/job-instance-import-edit-form/save
210 parent: pim-job-instance-xml-product-import-edit
211 targetZone: buttons
212 position: 0
213 config:
214 redirectPath: pim_importexport_import_profile_show
215
216 pim-job-instance-xml-product-import-edit-state:
217 module: pim/form/common/state
218 parent: pim-job-instance-xml-product-import-edit
219 targetZone: state
220 position: 900
221 config:
222 entity: pim_enrich.entity.job_instance.title
223
224 pim-job-instance-xml-product-import-edit-validation:
225 module: pim/job/common/edit/validation
226 parent: pim-job-instance-xml-product-import-edit
1extensions:
2 pim-job-instance-xml-product-import-show:
3 module: pim/form/common/edit-form
4
5 pim-job-instance-xml-product-import-show-tabs:
6 module: pim/form/common/form-tabs
7 parent: pim-job-instance-xml-product-import-show
8 targetZone: content
9 position: 100
10
11 pim-job-instance-xml-product-import-show-upload:
12 module: pim/job/common/edit/upload
13 parent: pim-job-instance-xml-product-import-show
14 aclResourceId: pim_importexport_import_profile_launch
15 targetZone: content
16 position: 90
17
18 pim-job-instance-xml-product-import-show-properties:
19 module: pim/job/common/edit/properties
20 parent: pim-job-instance-xml-product-import-show-tabs
21 aclResourceId: pim_importexport_export_profile_property_show
22 targetZone: container
23 position: 100
24 config:
25 tabTitle: pim_enrich.form.job_instance.tab.properties.title
26 tabCode: pim-job-instance-properties
27
28 pim-job-instance-xml-product-import-show-history:
29 module: pim/common/tab/history
30 parent: pim-job-instance-xml-product-import-show-tabs
31 targetZone: container
32 aclResourceId: pim_importexport_import_profile_history
33 position: 120
34 config:
35 class: Akeneo\Tool\Component\Batch\Model\JobInstance
36 title: pim_enrich.form.job_instance.tab.history.title
37 tabCode: pim-job-instance-history
38
39 pim-job-instance-xml-product-import-show-properties-code:
40 module: pim/job/common/edit/field/text
41 parent: pim-job-instance-xml-product-import-show-properties
42 position: 100
43 targetZone: properties
44 config:
45 fieldCode: code
46 label: pim_enrich.form.job_instance.tab.properties.code.title
47 readOnly: true
48
49 pim-job-instance-xml-product-import-show-properties-label:
50 module: pim/job/common/edit/field/text
51 parent: pim-job-instance-xml-product-import-show-properties
52 position: 110
53 targetZone: properties
54 config:
55 fieldCode: label
56 label: pim_enrich.form.job_instance.tab.properties.label.title
57 readOnly: true
58
59 pim-job-instance-xml-product-import-show-properties-file-path:
60 module: pim/job/common/edit/field/text
61 parent: pim-job-instance-xml-product-import-show-properties
62 position: 120
63 targetZone: global-settings
64 config:
65 fieldCode: configuration.filePath
66 readOnly: true
67 label: pim_enrich.form.job_instance.tab.properties.file_path.title
68 tooltip: pim_enrich.form.job_instance.tab.properties.file_path.help
69
70 pim-job-instance-xml-product-import-show-properties-file-upload:
71 module: pim/job/common/edit/field/switch
72 parent: pim-job-instance-xml-product-import-show-properties
73 position: 130
74 targetZone: global-settings
75 config:
76 fieldCode: configuration.uploadAllowed
77 readOnly: true
78 label: pim_enrich.form.job_instance.tab.properties.upload_allowed.title
79 tooltip: pim_enrich.form.job_instance.tab.properties.upload_allowed.help
80
81 pim-job-instance-xml-product-import-show-properties-decimal-separator:
82 module: pim/job/common/edit/field/decimal-separator
83 parent: pim-job-instance-xml-product-import-show-properties
84 position: 170
85 targetZone: global-settings
86 config:
87 fieldCode: configuration.decimalSeparator
88 readOnly: true
89 label: pim_enrich.form.job_instance.tab.properties.decimal_separator.title
90 tooltip: pim_enrich.form.job_instance.tab.properties.decimal_separator.help
91
92 pim-job-instance-xml-product-import-show-properties-date-format:
93 module: pim/job/product/edit/field/date-format
94 parent: pim-job-instance-xml-product-import-show-properties
95 position: 180
96 targetZone: global-settings
97 config:
98 fieldCode: configuration.dateFormat
99 readOnly: true
100 label: pim_enrich.form.job_instance.tab.properties.date_format.title
101 tooltip: pim_enrich.form.job_instance.tab.properties.date_format.help
102
103 pim-job-instance-xml-product-import-show-properties-enabled:
104 module: pim/job/common/edit/field/switch
105 parent: pim-job-instance-xml-product-import-show-properties
106 position: 190
107 targetZone: global-settings
108 config:
109 fieldCode: configuration.enabled
110 readOnly: true
111 label: pim_enrich.form.job_instance.tab.properties.enabled.title
112 tooltip: pim_enrich.form.job_instance.tab.properties.enabled.help
113
114 pim-job-instance-xml-product-import-show-properties-categories-column:
115 module: pim/job/common/edit/field/text
116 parent: pim-job-instance-xml-product-import-show-properties
117 position: 200
118 targetZone: global-settings
119 config:
120 fieldCode: configuration.categoriesColumn
121 readOnly: true
122 label: pim_enrich.form.job_instance.tab.properties.categories_column.title
123 tooltip: pim_enrich.form.job_instance.tab.properties.categories_column.help
124
125 pim-job-instance-xml-product-import-show-properties-family-column:
126 module: pim/job/common/edit/field/text
127 parent: pim-job-instance-xml-product-import-show-properties
128 position: 210
129 targetZone: global-settings
130 config:
131 fieldCode: configuration.familyColumn
132 readOnly: true
133 label: pim_enrich.form.job_instance.tab.properties.family_column.title
134 tooltip: pim_enrich.form.job_instance.tab.properties.family_column.help
135
136 pim-job-instance-xml-product-import-show-properties-groups-column:
137 module: pim/job/common/edit/field/text
138 parent: pim-job-instance-xml-product-import-show-properties
139 position: 220
140 targetZone: global-settings
141 config:
142 fieldCode: configuration.groupsColumn
143 readOnly: true
144 label: pim_enrich.form.job_instance.tab.properties.groups_column.title
145 tooltip: pim_enrich.form.job_instance.tab.properties.groups_column.help
146
147 pim-job-instance-xml-product-import-show-properties-enabled-comparison:
148 module: pim/job/common/edit/field/switch
149 parent: pim-job-instance-xml-product-import-show-properties
150 position: 230
151 targetZone: global-settings
152 config:
153 fieldCode: configuration.enabledComparison
154 readOnly: true
155 label: pim_enrich.form.job_instance.tab.properties.enabled_comparison.title
156 tooltip: pim_enrich.form.job_instance.tab.properties.enabled_comparison.help
157
158 pim-job-instance-xml-product-import-show-properties-real-time-versioning:
159 module: pim/job/common/edit/field/switch
160 parent: pim-job-instance-xml-product-import-show-properties
161 position: 240
162 targetZone: global-settings
163 config:
164 fieldCode: configuration.realTimeVersioning
165 readOnly: true
166 label: pim_enrich.form.job_instance.tab.properties.real_time_versioning.title
167 tooltip: pim_enrich.form.job_instance.tab.properties.real_time_versioning.help
168
169 pim-job-instance-xml-product-import-show-label:
170 module: pim/job/common/edit/label
171 parent: pim-job-instance-xml-product-import-show
172 targetZone: title
173 position: 100
174
175 pim-job-instance-xml-product-import-show-meta:
176 module: pim/job/common/edit/meta
177 parent: pim-job-instance-xml-product-import-show
178 targetZone: meta
179 position: 100
180
181 pim-job-instance-xml-product-import-show-back-to-grid:
182 module: pim/form/common/back-to-grid
183 parent: pim-job-instance-xml-product-import-show
184 targetZone: back
185 aclResourceId: pim_importexport_import_profile_index
186 position: 80
187 config:
188 backUrl: pim_importexport_import_profile_index
189
190 pim-job-instance-xml-product-import-show-edit:
191 module: pim/common/redirect
192 parent: pim-job-instance-xml-product-import-show
193 targetZone: buttons
194 position: 100
195 config:
196 label: pim_enrich.form.job_instance.button.edit.title
197 route: pim_importexport_import_profile_edit
198 identifier:
199 path: code
200 name: code
201
202 pim-job-instance-xml-product-import-show-launch:
203 module: pim/job/common/edit/upload-launch
204 parent: pim-job-instance-xml-product-import-show
205 aclResourceId: pim_importexport_import_profile_launch
206 targetZone: buttons
207 position: 110
208 config:
209 launch: pim_enrich.form.job_instance.button.import.launch
210 upload: pim_enrich.form.job_instance.button.import.upload
211 route: pim_enrich_job_instance_rest_import_launch
212 identifier:
213 path: code
214 name: code
For further information you can check the following cookbook: How to create a new Connector.
Important
We strongly advise to always try to re-use most of the existing pieces, especially processors and writers, it makes sure that all business rules and validation will be properly applied.
Create the Reader¶
As we don’t have an existing reader which allows to read XML files, we’ll implement a new one that supports it.
The purpose of the reader is to return each item as an array, in the case of XML file, we can have more work to define what is the item.
1<?php
2
3namespace Acme\Bundle\XmlConnectorBundle\Reader\File;
4
5use Akeneo\Tool\Component\Batch\Item\FileInvalidItem;
6use Akeneo\Tool\Component\Batch\Item\FlushableInterface;
7use Akeneo\Tool\Component\Batch\Item\InvalidItemException;
8use Akeneo\Tool\Component\Batch\Item\ItemReaderInterface;
9use Akeneo\Tool\Component\Batch\Model\StepExecution;
10use Akeneo\Tool\Component\Batch\Step\StepExecutionAwareInterface;
11use Akeneo\Tool\Component\Connector\ArrayConverter\ArrayConverterInterface;
12use Akeneo\Tool\Component\Connector\Exception\DataArrayConversionException;
13use Akeneo\Tool\Component\Connector\Exception\InvalidItemFromViolationsException;
14
15class XmlProductReader implements
16 ItemReaderInterface,
17 StepExecutionAwareInterface,
18 FlushableInterface
19{
20 /** @var array */
21 protected $xml;
22
23 /** @var StepExecution */
24 protected $stepExecution;
25
26 /** @var ArrayConverterInterface */
27 protected $converter;
28
29 /**
30 * @param ArrayConverterInterface $converter
31 */
32 public function __construct(ArrayConverterInterface $converter)
33 {
34 $this->converter = $converter;
35 }
36
37 public function read()
38 {
39 if (null === $this->xml) {
40 $jobParameters = $this->stepExecution->getJobParameters();
41 $filePath = $jobParameters->get('filePath');
42 // for example purpose, we should use XML Parser to read line per line
43 $this->xml = simplexml_load_file($filePath, 'SimpleXMLIterator');
44 $this->xml->rewind();
45 }
46
47 if ($data = $this->xml->current()) {
48 $item = [];
49 foreach ($data->attributes() as $attributeName => $attributeValue) {
50 $item[$attributeName] = (string) $attributeValue;
51 }
52 $this->xml->next();
53
54 if (null !== $this->stepExecution) {
55 $this->stepExecution->incrementSummaryInfo('item_position');
56 }
57
58 try {
59 $item = $this->converter->convert($item);
60 } catch (DataArrayConversionException $e) {
61 $this->skipItemFromConversionException($this->xml->current(), $e);
62 }
63
64 return $item;
65 }
66
67 return null;
68 }
69
70 /**
71 * {@inheritdoc}
72 */
73 public function flush()
74 {
75 $this->xml = null;
76 }
77
78 /**
79 * {@inheritdoc}
80 */
81 public function setStepExecution(StepExecution $stepExecution)
82 {
83 $this->stepExecution = $stepExecution;
84 }
85
86 /**
87 * @param array $item
88 * @param DataArrayConversionException $exception
89 *
90 * @throws InvalidItemException
91 * @throws InvalidItemFromViolationsException
92 */
93 protected function skipItemFromConversionException(array $item, DataArrayConversionException $exception)
94 {
95 if (null !== $this->stepExecution) {
96 $this->stepExecution->incrementSummaryInfo('skip');
97 }
98
99 if (null !== $exception->getViolations()) {
100 throw new InvalidItemFromViolationsException(
101 $exception->getViolations(),
102 new FileInvalidItem($item, $this->stepExecution->getSummaryInfo('item_position')),
103 [],
104 0,
105 $exception
106 );
107 }
108
109 $invalidItem = new FileInvalidItem(
110 $item,
111 $this->stepExecution->getSummaryInfo('item_position')
112 );
113
114 throw new InvalidItemException($exception->getMessage(), $invalidItem, [], 0, $exception);
115 }
116}
The reader processes the file and iterates to return products line by line and then converts them into the Standard format
This element must be configured with the path of the XML file (an example file is provided in XmlConnectorBundle\Resources\fixtures\products.xml).
Then, we need to define this reader as a service in readers.yml:
1parameters:
2 acme_xml_connector.reader.file.xml_product.class: Acme\Bundle\XmlConnectorBundle\Reader\File\XmlProductReader
3 acme_xml_connector.reader.file.file_iterator.class: Acme\Bundle\XmlConnectorBundle\Reader\File\XmlFileIterator
4
5services:
6 acme_xml_connector.reader.file.xml_iterator_factory:
7 class: '%pim_connector.reader.file.file_iterator_factory.class%'
8 arguments:
9 - '%acme_xml_connector.reader.file.file_iterator.class%'
10 - 'xml'
11
12 acme_xml_connector.reader.file.xml_product:
13 class: '%acme_xml_connector.reader.file.xml_product.class%'
14 arguments:
15 - '@pim_connector.array_converter.flat_to_standard.product'
And we introduce the following extension to load the services files in configuration:
1<?php
2
3namespace Acme\Bundle\XmlConnectorBundle\DependencyInjection;
4
5use Symfony\Component\HttpKernel\DependencyInjection\Extension;
6use Symfony\Component\DependencyInjection\ContainerBuilder;
7use Symfony\Component\DependencyInjection\Loader\YamlFileLoader;
8use Symfony\Component\Config\FileLocator;
9
10class AcmeXmlConnectorExtension extends Extension
11{
12 public function load(array $configs, ContainerBuilder $container)
13 {
14 $loader = new YamlFileLoader($container, new FileLocator(__DIR__.'/../Resources/config'));
15 $loader->load('archiving.yml');
16 $loader->load('jobs.yml');
17 $loader->load('job_parameters.yml');
18 $loader->load('steps.yml');
19 $loader->load('readers.yml');
20 $loader->load('writers.yml');
21 }
22}
Translate Job and Step labels in the UI¶
Behind the scene, the service Akeneo\Platform\Bundle\ImportExportBundle\JobLabel\TranslatedLabelProvider provides translated Job and Step labels to be used in the UI.
This service uses the following conventions: - for a job label, given a %%jobName%%, “batch_jobs.%%jobName%%.label” - for a step label, given a %%jobName%% and a %%stepName%%, “batch_jobs.%%jobName%%.%%stepName%%.label”
Create a file Resources/translations/messages.en.yml in the Bundle to translate label keys.
1acme_xml_connector:
2 jobs:
3 xml_product_import:
4 title: Product Import XML
5 import:
6 title: Product Import Step
7 steps:
8 product_processor.title: Product processor
9 import.filePath:
10 label: File path
11 help: Path of the XML file
12
13
14batch_jobs:
15 xml_product_import:
16 label: Product Import XML
17 import.label: Product Import XML
18
19pim_import_export:
20 download_archive:
21 invalid_xml: Download invalid items in XML
22
23pim_connector:
24 import:
25 xml:
26 enabled:
27 label: Enable the product
28 help: Whether or not the imported product should be enabled
29 categoriesColumn:
30 label: Categories column
31 help: Name of the categories column
32 familyColumn:
33 label: Family column
34 help: Name of the family column
35 groupsColumn:
36 label: Groups column
37 help: Name of the groups column
38 filePath:
39 label: File
40 help: The CSV file to import
41 uploadAllowed:
42 label: Allow file upload
43 help: Whether or not to allow uploading the file directly
44 delimiter:
45 label: Delimiter
46 help: One character used to set the field delimiter for CSV file
47 enclosure:
48 label: Enclosure
49 help: One character used to set the field enclosure
50 escape:
51 label: Escape
52 help: One character used to set the field escape
53 circularRefsChecked:
54 label: Check circular references
55 help: If yes, data will be processed to make sure that there are no circular references between the categories
56 realTimeVersioning:
57 label: Real time history update
58 help: Means that the product history is automatically updated, can be switched off to improve performances
59 copyValuesToProducts:
60 label: Copy variant group values to products
61 help: Means that the products are automatically updated with variant group values, can be switched off to only update variant group
62 enabledComparison:
63 label: Compare values
64 help: Enable the comparison between original values and imported values. Can speed up the import if imported values are very similar to original values
65 decimalSeparator:
66 label: Decimal separator
67 help: One character used to set the field separator for decimal
68 dateFormat:
69 label: Date format
70 help: Specify the format of any date columns in the file, e.g. here DD/MM/YYYY for a 30/04/2014 format.
Use the new Connector¶
Now if you refresh the cache, the new connector and xml job can be found under Collect > Import profiles > Create import profile.
You can create an instance of this job and give it a name like xml_product_import.
Now you can run the job from the UI or use the following command:
php bin/console akeneo:batch:job xml_product_import
Warning
In production, use this command instead:
php bin/console akeneo:batch:publish-job-to-queue my_app_product_export --env=prod
One daemon or several daemon processes have to be started to execute the jobs. Please follow the documentation Setting up the job queue daemon if it’s not the case.
Adding support for invalid items export¶
When the PIM reads the file and processes the entities we want to import, it performs several checks to make sure that the values we import are valid and they respect the constraints we configured the PIM with.
The PIM is then capable of exporting back the invalid items that do not respect those constraints after an import operation.
For our connector to support this feature, we will need to implement a few more parts in our connector:
XmlInvalidItemWriter: a registered XML invalid writer service whose work is to export the invalid lines found during the reading and processing steps.XmlFileIterator: which is used by theXmlInvalidItemWriterto read the imported file to find the invalid items.XmlWriter: Its responsibility is to write the invalid items back into a separate file available for download to the user.
Create an XML file iterator class¶
Let create a class which implements the FileIteratorInterface. This class opens the XML file that was imported thanks to an instance of \SimpleXMLIterator.
We now need to implement the functions of the interface. Here is a working example of the XML iterator:
1<?php
2
3namespace Acme\Bundle\XmlConnectorBundle\Reader\File;
4
5use Akeneo\Tool\Component\Connector\Reader\File\FileIteratorInterface;
6use Symfony\Component\Filesystem\Exception\FileNotFoundException;
7
8class XmlFileIterator implements FileIteratorInterface
9{
10 /** @var string **/
11 protected $type;
12
13 /** @var string **/
14 protected $filePath;
15
16 /** @var \SplFileInfo **/
17 protected $fileInfo;
18
19 /** @var \SimpleXMLIterator */
20 protected $xmlFileIterator;
21
22 /**
23 * {@inheritdoc}
24 */
25 public function __construct($type, $filePath, array $options = [])
26 {
27 $this->type = $type;
28 $this->filePath = $filePath;
29 $this->fileInfo = new \SplFileInfo($filePath);
30
31 if (!$this->fileInfo->isFile()) {
32 throw new FileNotFoundException(sprintf('File "%s" could not be found', $this->filePath));
33 }
34
35 $this->xmlFileIterator = simplexml_load_file($filePath, 'SimpleXMLIterator');
36 $this->xmlFileIterator->rewind();
37 }
38
39 /**
40 * {@inheritdoc}
41 */
42 public function getDirectoryPath()
43 {
44 if (null === $this->archivePath) {
45 return $this->fileInfo->getPath();
46 }
47
48 return $this->archivePath;
49 }
50
51 /**
52 * {@inheritdoc}
53 */
54 public function getHeaders()
55 {
56 $headers = [];
57 foreach ($this->xmlFileIterator->current()->attributes() as $header => $value) {
58 $headers[] = $header;
59 }
60
61 return $headers;
62 }
63
64 /**
65 * {@inheritdoc}
66 */
67 public function current()
68 {
69 $elem = $this->xmlFileIterator->current();
70
71 return $this->xmlElementToFlat($elem);
72 }
73
74 /**
75 * {@inheritdoc}
76 */
77 public function next()
78 {
79 $this->xmlFileIterator->next();
80 }
81
82 /**
83 * {@inheritdoc}
84 */
85 public function key()
86 {
87 return $this->xmlFileIterator->key();
88 }
89
90 /**
91 * {@inheritdoc}
92 */
93 public function valid()
94 {
95 return $this->xmlFileIterator->valid();
96 }
97
98 /**
99 * {@inheritdoc}
100 */
101 public function rewind()
102 {
103 $this->xmlFileIterator->rewind();
104 }
105
106 /**
107 * Converts an xml node into an array of values
108 *
109 * @param \SimpleXMLIterator $elem
110 *
111 * @return array
112 */
113 protected function xmlElementToFlat($elem)
114 {
115 $flatElem = [];
116
117 foreach ($elem->attributes() as $value) {
118 $flatElem[] = (string) $value;
119 }
120
121 return $flatElem;
122 }
123}
Now, let’s declare a simple Symfony service for this class. Here is the Resources/config/readers.yml:
acme_xml_connector.reader.file.xml_iterator_factory:
class: '%pim_connector.reader.file.file_iterator_factory.class%'
arguments:
- '%acme_xml_connector.reader.file.file_iterator.class%'
- 'xml'
Create an XML writer class¶
The XML writer will be responsible for writing the invalid items in a specified file path.
An implementation of it could be:
1<?php
2
3namespace Acme\Bundle\XmlConnectorBundle\Writer;
4
5use Akeneo\Tool\Component\Batch\Item\FlushableInterface;
6use Akeneo\Tool\Component\Batch\Item\InitializableInterface;
7use Akeneo\Tool\Component\Batch\Item\ItemWriterInterface;
8use Akeneo\Tool\Component\Batch\Step\StepExecutionAwareInterface;
9use Akeneo\Tool\Component\Connector\Writer\File\AbstractFileWriter;
10use Akeneo\Tool\Component\Connector\Writer\File\ArchivableWriterInterface;
11
12class XmlWriter extends AbstractFileWriter implements
13 ItemWriterInterface,
14 InitializableInterface,
15 FlushableInterface,
16 ArchivableWriterInterface,
17 StepExecutionAwareInterface
18{
19 /** @var array */
20 protected $writtenFiles = [];
21
22 /** @var \XMLWriter **/
23 protected $xml;
24
25 /**
26 * {@inheritdoc}
27 */
28 public function initialize()
29 {
30 if (null === $this->xml) {
31 $filePath = $this->stepExecution->getJobParameters()->get('filePath');
32
33 $this->xml = new \XMLWriter();
34 $this->xml->openURI($filePath);
35 $this->xml->startDocument('1.0', 'UTF-8');
36 $this->xml->setIndent(4);
37 $this->xml->startElement('products');
38 }
39 }
40
41 /**
42 * {@inheritdoc}
43 */
44 public function getWrittenFiles()
45 {
46 return $this->writtenFiles;
47 }
48
49 /**
50 * {@inheritdoc}
51 */
52 public function write(array $items)
53 {
54 $exportDirectory = dirname($this->getPath());
55 if (!is_dir($exportDirectory)) {
56 $this->localFs->mkdir($exportDirectory);
57 }
58
59 foreach ($items as $item) {
60 $this->xml->startElement('product');
61 foreach ($item as $property => $value) {
62 $this->xml->writeAttribute($property, $value);
63 }
64 $this->xml->endElement();
65 }
66 }
67
68 /**
69 * {@inheritdoc}
70 */
71 public function flush()
72 {
73 $this->xml->endElement();
74 $this->xml->endDocument();
75 $this->xml->flush();
76
77 $this->writtenFiles = [$this->stepExecution->getJobParameters()->get('filePath')];
78 }
79}
Let’s declare a Symfony service for our XML writer in Resources/config/writers.yml:
1parameters:
2 acme_xml_connector.writer.file.xml.class: Acme\Bundle\XmlConnectorBundle\Writer\XmlWriter
3
4services:
5 acme_xml_connector.writer.file.invalid_items_xml:
6 class: '%acme_xml_connector.writer.file.xml.class%'
Note
Please note that every new configuration file created in the Resources/config folder should be loaded in the Symfony dependency injection for it to be taken into account.
Plug it all together¶
Now that our XmlFileIterator class and service are defined, let’s use them in our custom implementation of the XmlInvalidWriterInterface.
Let’s use the existing AbstractInvalidItem to implement our custom class. We only need to implement two functions from our abstract superclass.
getInputFileIterator(JobParameters $jobParameters): that returns a configured instance of our custom reader theXmlFileIteratorclass.setupWriter(JobExecution $jobExecution): sets up our custom writer, an instance of theXmlWriterclass.
Here is a working example:
1<?php
2
3namespace Acme\Bundle\XmlConnectorBundle\Archiver;
4
5use Akeneo\Tool\Component\Batch\Job\JobParameters;
6use Akeneo\Tool\Component\Batch\Model\JobExecution;
7use Akeneo\Tool\Component\Batch\Model\StepExecution;
8use Akeneo\Tool\Component\Connector\Archiver\AbstractInvalidItemWriter;
9
10class XmlInvalidItemWriter extends AbstractInvalidItemWriter
11{
12 /**
13 * {@inheritdoc}
14 */
15 public function getName()
16 {
17 return 'invalid_xml';
18 }
19
20 /**
21 * {@inheritdoc}
22 */
23 protected function getInputFileIterator(JobParameters $jobParameters)
24 {
25 $filePath = $jobParameters->get('filePath');
26 $fileIterator = $this->fileIteratorFactory->create($filePath);
27 $fileIterator->rewind();
28
29 return $fileIterator;
30 }
31
32 /**
33 * {@inheritdoc}
34 */
35 protected function setupWriter(JobExecution $jobExecution)
36 {
37 $fileKey = strtr($this->getRelativeArchivePath($jobExecution), ['%filename%' => 'invalid_items.xml']);
38 $this->filesystem->put($fileKey, '');
39
40 $writeParams = $this->defaultValuesProvider->getDefaultValues();
41 $writeParams['filePath'] = $this->filesystem->getAdapter()->getPathPrefix() . $fileKey;
42
43 $writeJobParameters = new JobParameters($writeParams);
44 $writeJobExecution = new JobExecution();
45 $writeJobExecution->setJobParameters($writeJobParameters);
46
47 $stepExecution = new StepExecution('processor', $writeJobExecution);
48 $this->writer->setStepExecution($stepExecution);
49 $this->writer->initialize();
50 }
51}
Let’s define a tagged Symfony service in Resources/config/archiving.yml, so that our custom invalid item writer is taken into account and used by the PIM.
1parameters:
2 acme_xml_connector.archiver.invalid_item_xml_writer.class: Acme\Bundle\XmlConnectorBundle\Archiver\XmlInvalidItemWriter
3
4services:
5 acme_xml_connector.archiver.invalid_item_xml_writer:
6 class: '%acme_xml_connector.archiver.invalid_item_xml_writer.class%'
7 arguments:
8 - '@pim_connector.event_listener.invalid_items_collector'
9 - '@acme_xml_connector.writer.file.invalid_items_xml'
10 - '@acme_xml_connector.reader.file.xml_iterator_factory'
11 - '@oneup_flysystem.archivist_filesystem'
12 - '@acme_xml_connector.job.job_parameters.simple_xml_product_import'
13 - 'xml'
14 tags:
15 - { name: pim_connector.archiver }
Try it out¶
All parts of our connector are now in place for it to be able to export invalid items.
To try it out, run the XML import with the example file products.xml in the UI. At the end of the job execution a new button should appear with the label “Download invalid items in XML”.
Click it and download the XML file containing the invalid items found by the import job.
Found a typo or a hole in the documentation and feel like contributing?
Join us on Github!