How to import Products from a XML file¶
Prerequisite¶
The basics of a connector creation have been covered in the previous chapter (cf. How to create a new Connector). With the following hands-on practice, we will create our own specific connector.
We assume that we’re using a standard edition with the icecat_demo_dev data set, sku and name already exist as attributes in the PIM and the family is also an existing property.
Overview¶
In this cookbook, we will create a brand new XML connector to import our products using XML files.
For a recap, here is the process of a product import job execution:
Starting the job and reading the first product
The job opens the file to import, it reads the first product in the file and converts it into a standard format.
If an error or a warning is thrown during this step, the product is marked as invalid.
Process the product and check the product values
If no errors have been thrown in the previous step, the read and converted product is then processed by a product processor.
If an error is thrown while processing, the product is marked as invalid.
Note
At this point, an error could be that the the family code does not exist or that the currency set for a price attribute does not match the currency configured in the PIM for this attribute.
Save the product in database
The processed product is written in the database using a product writer.
Collect the invalid products found and export them into a separate file
When all products have been read, processed and written into the database, the job collects all the errors found in the file at each step and writes them back into a separate file of invalid items.
In this cookbook, our use case is to import new products from the following XML file products.xml:
1<?xml version="1.0" encoding="UTF-8"?>
2<products>
3 <product sku="sku-1" name="my name 1" family="camcorders" enabled="1"/>
4 <product sku="sku-2" name="my name 2" family="camcorders" enabled="1"/>
5 <product sku="sku-3" name="my name 3" family="camcorders" enabled="1"/>
6 <product sku="sku-4" name="my name 4" family="wrong_family_code" enabled="0"/>
7 <product sku="" name="my name 5" family="camcorders" enabled="1"/>
8</products>
To stay focused on the main concepts, we will implement the simplest connector possible by avoiding to use too many existing elements.
Note
The code inside this cookbook entry is available in the src directory, you can clone pim-docs (https://github.com/akeneo/pim-docs) and use a symlink to make the Acme bundle available in the src/.
Create the Connector¶
Create a new bundle:
1<?php
2
3namespace Acme\Bundle\XmlConnectorBundle;
4
5use Symfony\Component\HttpKernel\Bundle\Bundle;
6
7class AcmeXmlConnectorBundle extends Bundle
8{
9}
Register the bundle in AppKernel:
1public function registerProjectBundles()
2{
3 return [
4 // your app bundles should be registered here
5 new Acme\Bundle\AppBundle\AcmeAppBundle(),
6 new Acme\Bundle\XmlConnectorBundle\AcmeXmlConnectorBundle()
7 ];
8}
Configure the Job¶
Configure the job in Resources/config/jobs.yml:
1 acme_xml_connector.job.xml_product_import:
2 class: '%pim_connector.job.simple_job.class%'
3 arguments:
4 - 'xml_product_import'
5 - '@event_dispatcher'
6 - '@akeneo_batch.job_repository'
7 -
8 - '@acme_xml_connector.step.xml_product_import.import'
9 tags:
10 - { name: akeneo_batch.job, connector: 'Akeneo XML Connector', type: '%pim_connector.job.import_type%' }
The default step is Akeneo\Tool\Component\Batch\Step\ItemStep.
An item step is configured with 3 elements, a reader, a processor and a writer.
Here is the definition of the Step:
1 acme_xml_connector.step.xml_product_import.import:
2 class: '%pim_connector.step.item_step.class%'
3 arguments:
4 - 'import'
5 - '@event_dispatcher'
6 - '@akeneo_batch.job_repository'
7 - '@acme_xml_connector.reader.file.xml_product'
8 - '@pim_connector.processor.denormalization.product'
9 - '@pim_connector.writer.database.product'
10 - 10
Here, we’ll use a custom reader acme_xml_connector.reader.file.xml_product but we’ll continue to use the default processor and writer.
Then you will need to add the job parameters classes (they define the job configuration, job constraints and job default values):
1parameters:
2 acme_xml_connector.job.job_parameters.simple_xml_import.class: Acme\Bundle\XmlConnectorBundle\Job\JobParameters\SimpleXmlImport
3
4services:
5 acme_xml_connector.job.job_parameters.simple_xml_product_import:
6 class: '%acme_xml_connector.job.job_parameters.simple_xml_import.class%'
7 arguments:
8 - '%pim_catalog.localization.decimal_separators%'
9 - '%pim_catalog.localization.date_formats%'
10 tags:
11 - { name: akeneo_batch.job.job_parameters.constraint_collection_provider }
12 - { name: akeneo_batch.job.job_parameters.default_values_provider }
13
14 acme_xml_connector.job.job_parameters.provider.simple_xml_product_import:
15 class: 'Akeneo\Platform\Bundle\ImportExportBundle\Provider\Form\JobInstanceFormProvider'
16 arguments:
17 -
18 xml_product_import: pim-job-instance-xml-product-import
19 tags:
20 - { name: pim_enrich.provider.form }
1<?php
2
3namespace Acme\Bundle\XmlConnectorBundle\Job\JobParameters;
4
5use Akeneo\Platform\Bundle\ImportExportBundle\Infrastructure\Validation\Storage;
6use Akeneo\Tool\Component\Batch\Job\JobInterface;
7use Akeneo\Tool\Component\Batch\Job\JobParameters\ConstraintCollectionProviderInterface;
8use Akeneo\Tool\Component\Batch\Job\JobParameters\DefaultValuesProviderInterface;
9use Akeneo\Tool\Component\Localization\Localizer\LocalizerInterface;
10use Akeneo\Pim\Enrichment\Component\Product\Validator\Constraints\FileExtension;
11use Symfony\Component\Validator\Constraints\Collection;
12use Symfony\Component\Validator\Constraints\IsTrue;
13use Symfony\Component\Validator\Constraints\NotBlank;
14use Symfony\Component\Validator\Constraints\Type;
15
16class SimpleXmlImport implements
17 DefaultValuesProviderInterface,
18 ConstraintCollectionProviderInterface
19{
20 /** @var ConstraintCollectionProviderInterface */
21 protected $constraintCollectionProvider;
22
23 /** @var DefaultValuesProviderInterface */
24 protected $defaultValuesProvider;
25
26 /**
27 * {@inheritdoc}
28 */
29 public function getDefaultValues()
30 {
31 return [
32 'storage' => [
33 'type' => 'none',
34 'file_path' => sys_get_temp_dir() . DIRECTORY_SEPARATOR . 'export_%job_label%_%datetime%.xml'
35 ],
36 'uploadAllowed' => true,
37 'dateFormat' => LocalizerInterface::DEFAULT_DATE_FORMAT,
38 'decimalSeparator' => LocalizerInterface::DEFAULT_DECIMAL_SEPARATOR,
39 'enabled' => true,
40 'categoriesColumn' => 'categories',
41 'familyColumn' => 'family',
42 'groupsColumn' => 'groups',
43 'enabledComparison' => true,
44 'realTimeVersioning' => true,
45 'invalid_items_file_format' => 'xml',
46 ];
47 }
48
49 /**
50 * {@inheritdoc}
51 */
52 public function getConstraintCollection()
53 {
54 return new Collection(
55 [
56 'fields' => [
57 'storage' => new Storage(['xml']),
58 'uploadAllowed' => [
59 new Type('bool'),
60 new IsTrue(['groups' => 'UploadExecution']),
61 ],
62 'decimalSeparator' => [
63 new NotBlank()
64 ],
65 'dateFormat' => [
66 new NotBlank()
67 ],
68 'enabled' => [
69 new Type('bool')
70 ],
71 'categoriesColumn' => [
72 new NotBlank()
73 ],
74 'familyColumn' => [
75 new NotBlank()
76 ],
77 'groupsColumn' => [
78 new NotBlank()
79 ],
80 'enabledComparison' => [
81 new Type('bool')
82 ],
83 'realTimeVersioning' => [
84 new Type('bool')
85 ],
86 'invalid_items_file_format' => [
87 new Type('string')
88 ]
89 ]
90 ]
91 );
92 }
93
94 /**
95 * {@inheritdoc}
96 */
97 public function supports(JobInterface $job)
98 {
99 return $job->getName() === 'xml_product_import';
100 }
101}
1extensions:
2 pim-job-instance-xml-product-import-edit:
3 module: pim/form/common/edit-form
4
5 pim-job-instance-xml-product-import-edit-cache-invalidator:
6 module: pim/cache-invalidator
7 parent: pim-job-instance-xml-product-import-edit
8 position: 1000
9
10 pim-job-instance-xml-product-import-edit-tabs:
11 module: pim/form/common/form-tabs
12 parent: pim-job-instance-xml-product-import-edit
13 targetZone: content
14 position: 100
15
16 pim-job-instance-xml-product-import-edit-properties:
17 module: pim/job/common/edit/properties
18 parent: pim-job-instance-xml-product-import-edit-tabs
19 aclResourceId: pim_importexport_export_profile_property_edit
20 targetZone: container
21 position: 100
22 config:
23 tabTitle: pim_enrich.form.job_instance.tab.properties.title
24 tabCode: pim-job-instance-properties
25
26 pim-job-instance-xml-product-import-edit-history:
27 module: pim/common/tab/history
28 parent: pim-job-instance-xml-product-import-edit-tabs
29 targetZone: container
30 aclResourceId: pim_importexport_import_profile_history
31 position: 120
32 config:
33 class: Akeneo\Tool\Component\Batch\Model\JobInstance
34 title: pim_enrich.form.job_instance.tab.history.title
35 tabCode: pim-job-instance-history
36
37 pim-job-instance-xml-product-import-edit-properties-code:
38 module: pim/job/common/edit/field/text
39 parent: pim-job-instance-xml-product-import-edit-properties
40 position: 100
41 targetZone: properties
42 config:
43 fieldCode: code
44 label: pim_enrich.form.job_instance.tab.properties.code.title
45 readOnly: true
46
47 pim-job-instance-xml-product-import-edit-properties-label:
48 module: pim/job/common/edit/field/text
49 parent: pim-job-instance-xml-product-import-edit-properties
50 position: 110
51 targetZone: properties
52 config:
53 fieldCode: label
54 label: pim_enrich.form.job_instance.tab.properties.label.title
55
56 pim-job-instance-xml-product-import-edit-properties-storage:
57 module: pimimportexport/js/job/common/edit/storage-form
58 parent: pim-job-instance-xml-product-import-edit-properties
59 position: 120
60 targetZone: additional-properties
61 config:
62 tabCode: pim-job-instance-properties
63 jobType: import
64 fileExtension: xml
65
66 pim-job-instance-xml-product-import-edit-properties-file-upload:
67 module: pim/job/common/edit/field/switch
68 parent: pim-job-instance-xml-product-import-edit-properties
69 position: 130
70 targetZone: global-settings
71 config:
72 fieldCode: configuration.uploadAllowed
73 readOnly: false
74 label: pim_enrich.form.job_instance.tab.properties.upload_allowed.title
75 tooltip: pim_enrich.form.job_instance.tab.properties.upload_allowed.help
76 readOnly: false
77
78 pim-job-instance-xml-product-import-edit-properties-decimal-separator:
79 module: pim/job/common/edit/field/decimal-separator
80 parent: pim-job-instance-xml-product-import-edit-properties
81 position: 170
82 targetZone: global-settings
83 config:
84 fieldCode: configuration.decimalSeparator
85 readOnly: false
86 label: pim_enrich.form.job_instance.tab.properties.decimal_separator.title
87 tooltip: pim_enrich.form.job_instance.tab.properties.decimal_separator.help
88
89 pim-job-instance-xml-product-import-edit-properties-date-format:
90 module: pim/job/product/edit/field/date-format
91 parent: pim-job-instance-xml-product-import-edit-properties
92 position: 180
93 targetZone: global-settings
94 config:
95 fieldCode: configuration.dateFormat
96 readOnly: false
97 label: pim_enrich.form.job_instance.tab.properties.date_format.title
98 tooltip: pim_enrich.form.job_instance.tab.properties.date_format.help
99
100 pim-job-instance-xml-product-import-edit-properties-enabled:
101 module: pim/job/common/edit/field/switch
102 parent: pim-job-instance-xml-product-import-edit-properties
103 position: 190
104 targetZone: global-settings
105 config:
106 fieldCode: configuration.enabled
107 readOnly: false
108 label: pim_enrich.form.job_instance.tab.properties.enabled.title
109 tooltip: pim_enrich.form.job_instance.tab.properties.enabled.help
110
111 pim-job-instance-xml-product-import-edit-properties-categories-column:
112 module: pim/job/common/edit/field/text
113 parent: pim-job-instance-xml-product-import-edit-properties
114 position: 200
115 targetZone: global-settings
116 config:
117 fieldCode: configuration.categoriesColumn
118 readOnly: false
119 label: pim_enrich.form.job_instance.tab.properties.categories_column.title
120 tooltip: pim_enrich.form.job_instance.tab.properties.categories_column.help
121
122 pim-job-instance-xml-product-import-edit-properties-family-column:
123 module: pim/job/common/edit/field/text
124 parent: pim-job-instance-xml-product-import-edit-properties
125 position: 210
126 targetZone: global-settings
127 config:
128 fieldCode: configuration.familyColumn
129 readOnly: false
130 label: pim_enrich.form.job_instance.tab.properties.family_column.title
131 tooltip: pim_enrich.form.job_instance.tab.properties.family_column.help
132
133 pim-job-instance-xml-product-import-edit-properties-groups-column:
134 module: pim/job/common/edit/field/text
135 parent: pim-job-instance-xml-product-import-edit-properties
136 position: 220
137 targetZone: global-settings
138 config:
139 fieldCode: configuration.groupsColumn
140 readOnly: false
141 label: pim_enrich.form.job_instance.tab.properties.groups_column.title
142 tooltip: pim_enrich.form.job_instance.tab.properties.groups_column.help
143
144 pim-job-instance-xml-product-import-edit-properties-enabled-comparison:
145 module: pim/job/common/edit/field/switch
146 parent: pim-job-instance-xml-product-import-edit-properties
147 position: 230
148 targetZone: global-settings
149 config:
150 fieldCode: configuration.enabledComparison
151 readOnly: false
152 label: pim_enrich.form.job_instance.tab.properties.enabled_comparison.title
153 tooltip: pim_enrich.form.job_instance.tab.properties.enabled_comparison.help
154
155 pim-job-instance-xml-product-import-edit-properties-real-time-versioning:
156 module: pim/job/common/edit/field/switch
157 parent: pim-job-instance-xml-product-import-edit-properties
158 position: 240
159 targetZone: global-settings
160 config:
161 fieldCode: configuration.realTimeVersioning
162 readOnly: false
163 label: pim_enrich.form.job_instance.tab.properties.real_time_versioning.title
164 tooltip: pim_enrich.form.job_instance.tab.properties.real_time_versioning.help
165
166 pim-job-instance-xml-product-import-edit-label:
167 module: pim/job/common/edit/label
168 parent: pim-job-instance-xml-product-import-edit
169 targetZone: title
170 position: 100
171
172 pim-job-instance-xml-product-import-edit-meta:
173 module: pim/job/common/edit/meta
174 parent: pim-job-instance-xml-product-import-edit
175 targetZone: meta
176 position: 100
177
178 pim-job-instance-xml-product-import-edit-back-to-grid:
179 module: pim/form/common/back-to-grid
180 parent: pim-job-instance-xml-product-import-edit
181 targetZone: back
182 aclResourceId: pim_importexport_import_profile_index
183 position: 80
184 config:
185 backUrl: pim_importexport_import_profile_index
186
187 pim-job-instance-xml-product-import-edit-delete:
188 module: pim/job/import/edit/delete
189 parent: pim-job-instance-xml-product-import-edit
190 targetZone: buttons
191 aclResourceId: pim_importexport_import_profile_remove
192 position: 100
193 config:
194 trans:
195 title: confirmation.remove.job_instance
196 content: pim_enrich.confirmation.delete_item
197 success: flash.job_instance.removed
198 failed: error.removing.job_instance
199 redirect: pim_importexport_import_profile_index
200
201 pim-job-instance-xml-product-import-edit-save-buttons:
202 module: pim/form/common/save-buttons
203 parent: pim-job-instance-xml-product-import-edit
204 targetZone: buttons
205 position: 120
206
207 pim-job-instance-xml-product-import-edit-save:
208 module: pim/job-instance-import-edit-form/save
209 parent: pim-job-instance-xml-product-import-edit
210 targetZone: buttons
211 position: 0
212 config:
213 redirectPath: pim_importexport_import_profile_show
214
215 pim-job-instance-xml-product-import-edit-state:
216 module: pim/form/common/state
217 parent: pim-job-instance-xml-product-import-edit
218 targetZone: state
219 position: 900
220 config:
221 entity: pim_enrich.entity.job_instance.title
222
223 pim-job-instance-xml-product-import-edit-validation:
224 module: pim/job/common/edit/validation
225 parent: pim-job-instance-xml-product-import-edit
1extensions:
2 pim-job-instance-xml-product-import-show:
3 module: pim/form/common/edit-form
4
5 pim-job-instance-xml-product-import-show-tabs:
6 module: pim/form/common/form-tabs
7 parent: pim-job-instance-xml-product-import-show
8 targetZone: content
9 position: 100
10
11 pim-job-instance-xml-product-import-show-upload:
12 module: pim/job/common/edit/upload
13 parent: pim-job-instance-xml-product-import-show
14 aclResourceId: pim_importexport_import_profile_launch
15 targetZone: content
16 position: 90
17
18 pim-job-instance-xml-product-import-show-properties:
19 module: pim/job/common/edit/properties
20 parent: pim-job-instance-xml-product-import-show-tabs
21 aclResourceId: pim_importexport_export_profile_property_show
22 targetZone: container
23 position: 100
24 config:
25 tabTitle: pim_enrich.form.job_instance.tab.properties.title
26 tabCode: pim-job-instance-properties
27
28 pim-job-instance-xml-product-import-show-history:
29 module: pim/common/tab/history
30 parent: pim-job-instance-xml-product-import-show-tabs
31 targetZone: container
32 aclResourceId: pim_importexport_import_profile_history
33 position: 120
34 config:
35 class: Akeneo\Tool\Component\Batch\Model\JobInstance
36 title: pim_enrich.form.job_instance.tab.history.title
37 tabCode: pim-job-instance-history
38
39 pim-job-instance-xml-product-import-show-properties-code:
40 module: pim/job/common/edit/field/text
41 parent: pim-job-instance-xml-product-import-show-properties
42 position: 100
43 targetZone: properties
44 config:
45 fieldCode: code
46 label: pim_enrich.form.job_instance.tab.properties.code.title
47 readOnly: true
48
49 pim-job-instance-xml-product-import-show-properties-label:
50 module: pim/job/common/edit/field/text
51 parent: pim-job-instance-xml-product-import-show-properties
52 position: 110
53 targetZone: properties
54 config:
55 fieldCode: label
56 label: pim_enrich.form.job_instance.tab.properties.label.title
57 readOnly: true
58
59
60 pim-job-instance-xml-product-import-show-properties-storage:
61 module: pimimportexport/js/job/common/edit/storage-form
62 parent: pim-job-instance-xml-product-import-show-properties
63 position: 120
64 targetZone: additional-properties
65 config:
66 tabCode: pim-job-instance-properties
67 jobType: import
68 fileExtension: xml
69
70 pim-job-instance-xml-product-import-show-properties-file-upload:
71 module: pim/job/common/edit/field/switch
72 parent: pim-job-instance-xml-product-import-show-properties
73 position: 130
74 targetZone: global-settings
75 config:
76 fieldCode: configuration.uploadAllowed
77 readOnly: true
78 label: pim_enrich.form.job_instance.tab.properties.upload_allowed.title
79 tooltip: pim_enrich.form.job_instance.tab.properties.upload_allowed.help
80
81 pim-job-instance-xml-product-import-show-properties-decimal-separator:
82 module: pim/job/common/edit/field/decimal-separator
83 parent: pim-job-instance-xml-product-import-show-properties
84 position: 170
85 targetZone: global-settings
86 config:
87 fieldCode: configuration.decimalSeparator
88 readOnly: true
89 label: pim_enrich.form.job_instance.tab.properties.decimal_separator.title
90 tooltip: pim_enrich.form.job_instance.tab.properties.decimal_separator.help
91
92 pim-job-instance-xml-product-import-show-properties-date-format:
93 module: pim/job/product/edit/field/date-format
94 parent: pim-job-instance-xml-product-import-show-properties
95 position: 180
96 targetZone: global-settings
97 config:
98 fieldCode: configuration.dateFormat
99 readOnly: true
100 label: pim_enrich.form.job_instance.tab.properties.date_format.title
101 tooltip: pim_enrich.form.job_instance.tab.properties.date_format.help
102
103 pim-job-instance-xml-product-import-show-properties-enabled:
104 module: pim/job/common/edit/field/switch
105 parent: pim-job-instance-xml-product-import-show-properties
106 position: 190
107 targetZone: global-settings
108 config:
109 fieldCode: configuration.enabled
110 readOnly: true
111 label: pim_enrich.form.job_instance.tab.properties.enabled.title
112 tooltip: pim_enrich.form.job_instance.tab.properties.enabled.help
113
114 pim-job-instance-xml-product-import-show-properties-categories-column:
115 module: pim/job/common/edit/field/text
116 parent: pim-job-instance-xml-product-import-show-properties
117 position: 200
118 targetZone: global-settings
119 config:
120 fieldCode: configuration.categoriesColumn
121 readOnly: true
122 label: pim_enrich.form.job_instance.tab.properties.categories_column.title
123 tooltip: pim_enrich.form.job_instance.tab.properties.categories_column.help
124
125 pim-job-instance-xml-product-import-show-properties-family-column:
126 module: pim/job/common/edit/field/text
127 parent: pim-job-instance-xml-product-import-show-properties
128 position: 210
129 targetZone: global-settings
130 config:
131 fieldCode: configuration.familyColumn
132 readOnly: true
133 label: pim_enrich.form.job_instance.tab.properties.family_column.title
134 tooltip: pim_enrich.form.job_instance.tab.properties.family_column.help
135
136 pim-job-instance-xml-product-import-show-properties-groups-column:
137 module: pim/job/common/edit/field/text
138 parent: pim-job-instance-xml-product-import-show-properties
139 position: 220
140 targetZone: global-settings
141 config:
142 fieldCode: configuration.groupsColumn
143 readOnly: true
144 label: pim_enrich.form.job_instance.tab.properties.groups_column.title
145 tooltip: pim_enrich.form.job_instance.tab.properties.groups_column.help
146
147 pim-job-instance-xml-product-import-show-properties-enabled-comparison:
148 module: pim/job/common/edit/field/switch
149 parent: pim-job-instance-xml-product-import-show-properties
150 position: 230
151 targetZone: global-settings
152 config:
153 fieldCode: configuration.enabledComparison
154 readOnly: true
155 label: pim_enrich.form.job_instance.tab.properties.enabled_comparison.title
156 tooltip: pim_enrich.form.job_instance.tab.properties.enabled_comparison.help
157
158 pim-job-instance-xml-product-import-show-properties-real-time-versioning:
159 module: pim/job/common/edit/field/switch
160 parent: pim-job-instance-xml-product-import-show-properties
161 position: 240
162 targetZone: global-settings
163 config:
164 fieldCode: configuration.realTimeVersioning
165 readOnly: true
166 label: pim_enrich.form.job_instance.tab.properties.real_time_versioning.title
167 tooltip: pim_enrich.form.job_instance.tab.properties.real_time_versioning.help
168
169 pim-job-instance-xml-product-import-show-label:
170 module: pim/job/common/edit/label
171 parent: pim-job-instance-xml-product-import-show
172 targetZone: title
173 position: 100
174
175 pim-job-instance-xml-product-import-show-meta:
176 module: pim/job/common/edit/meta
177 parent: pim-job-instance-xml-product-import-show
178 targetZone: meta
179 position: 100
180
181 pim-job-instance-xml-product-import-show-back-to-grid:
182 module: pim/form/common/back-to-grid
183 parent: pim-job-instance-xml-product-import-show
184 targetZone: back
185 aclResourceId: pim_importexport_import_profile_index
186 position: 80
187 config:
188 backUrl: pim_importexport_import_profile_index
189
190 pim-job-instance-xml-product-import-show-edit:
191 module: pim/common/redirect
192 parent: pim-job-instance-xml-product-import-show
193 targetZone: buttons
194 position: 100
195 config:
196 label: pim_enrich.form.job_instance.button.edit.title
197 route: pim_importexport_import_profile_edit
198 identifier:
199 path: code
200 name: code
201
202 pim-job-instance-xml-product-import-show-launch:
203 module: pim/job/common/edit/upload-launch
204 parent: pim-job-instance-xml-product-import-show
205 aclResourceId: pim_importexport_import_profile_launch
206 targetZone: buttons
207 position: 110
208 config:
209 launch: pim_enrich.form.job_instance.button.import.launch
210 upload: pim_enrich.form.job_instance.button.import.upload
211 route: pim_enrich_job_instance_rest_import_launch
212 identifier:
213 path: code
214 name: code
For further information you can check the following cookbook: How to create a new Connector.
Important
We strongly advise to always try to re-use most of the existing pieces, especially processors and writers, it makes sure that all business rules and validation will be properly applied.
Create the Reader¶
As we don’t have an existing reader which allows to read XML files, we’ll implement a new one that supports it.
The purpose of the reader is to return each item as an array, in the case of XML file, we can have more work to define what is the item.
1<?php
2
3namespace Acme\Bundle\XmlConnectorBundle\Reader\File;
4
5use Akeneo\Tool\Component\Batch\Item\FileInvalidItem;
6use Akeneo\Tool\Component\Batch\Item\FlushableInterface;
7use Akeneo\Tool\Component\Batch\Item\InvalidItemException;
8use Akeneo\Tool\Component\Batch\Item\ItemReaderInterface;
9use Akeneo\Tool\Component\Batch\Model\StepExecution;
10use Akeneo\Tool\Component\Batch\Step\StepExecutionAwareInterface;
11use Akeneo\Tool\Component\Connector\ArrayConverter\ArrayConverterInterface;
12use Akeneo\Tool\Component\Connector\Exception\DataArrayConversionException;
13use Akeneo\Tool\Component\Connector\Exception\InvalidItemFromViolationsException;
14
15class XmlProductReader implements
16 ItemReaderInterface,
17 StepExecutionAwareInterface,
18 FlushableInterface
19{
20 /** @var array */
21 protected $xml;
22
23 /** @var StepExecution */
24 protected $stepExecution;
25
26 /** @var ArrayConverterInterface */
27 protected $converter;
28
29 /**
30 * @param ArrayConverterInterface $converter
31 */
32 public function __construct(ArrayConverterInterface $converter)
33 {
34 $this->converter = $converter;
35 }
36
37 public function read()
38 {
39 if (null === $this->xml) {
40 $jobParameters = $this->stepExecution->getJobParameters();
41 $filePath = $jobParameters->get('storage')['file_path'];
42
43 // for example purpose, we should use XML Parser to read line per line
44 $this->xml = simplexml_load_file($filePath, 'SimpleXMLIterator');
45 $this->xml->rewind();
46 }
47
48 if ($data = $this->xml->current()) {
49 $item = [];
50 foreach ($data->attributes() as $attributeName => $attributeValue) {
51 $item[$attributeName] = (string) $attributeValue;
52 }
53 $this->xml->next();
54
55 if (null !== $this->stepExecution) {
56 $this->stepExecution->incrementSummaryInfo('item_position');
57 }
58
59 try {
60 $item = $this->converter->convert($item);
61 } catch (DataArrayConversionException $e) {
62 $this->skipItemFromConversionException($this->xml->current(), $e);
63 }
64
65 return $item;
66 }
67
68 return null;
69 }
70
71 /**
72 * {@inheritdoc}
73 */
74 public function flush()
75 {
76 $this->xml = null;
77 }
78
79 /**
80 * {@inheritdoc}
81 */
82 public function setStepExecution(StepExecution $stepExecution)
83 {
84 $this->stepExecution = $stepExecution;
85 }
86
87 /**
88 * @param array $item
89 * @param DataArrayConversionException $exception
90 *
91 * @throws InvalidItemException
92 * @throws InvalidItemFromViolationsException
93 */
94 protected function skipItemFromConversionException(array $item, DataArrayConversionException $exception)
95 {
96 if (null !== $this->stepExecution) {
97 $this->stepExecution->incrementSummaryInfo('skip');
98 }
99
100 if (null !== $exception->getViolations()) {
101 throw new InvalidItemFromViolationsException(
102 $exception->getViolations(),
103 new FileInvalidItem($item, $this->stepExecution->getSummaryInfo('item_position')),
104 [],
105 0,
106 $exception
107 );
108 }
109
110 $invalidItem = new FileInvalidItem(
111 $item,
112 $this->stepExecution->getSummaryInfo('item_position')
113 );
114
115 throw new InvalidItemException($exception->getMessage(), $invalidItem, [], 0, $exception);
116 }
117}
The reader processes the file and iterates to return products line by line and then converts them into the Standard format
This element must be configured with the path of the XML file (an example file is provided in XmlConnectorBundle\Resources\fixtures\products.xml).
Then, we need to define this reader as a service in readers.yml:
1parameters:
2 acme_xml_connector.reader.file.xml_product.class: Acme\Bundle\XmlConnectorBundle\Reader\File\XmlProductReader
3 acme_xml_connector.reader.file.file_iterator.class: Acme\Bundle\XmlConnectorBundle\Reader\File\XmlFileIterator
4
5services:
6 acme_xml_connector.reader.file.xml_iterator_factory:
7 class: '%pim_connector.reader.file.file_iterator_factory.class%'
8 arguments:
9 - '%acme_xml_connector.reader.file.file_iterator.class%'
10 - 'xml'
11
12 acme_xml_connector.reader.file.xml_product:
13 class: '%acme_xml_connector.reader.file.xml_product.class%'
14 arguments:
15 - '@pim_connector.array_converter.flat_to_standard.product'
And we introduce the following extension to load the services files in configuration:
1<?php
2
3namespace Acme\Bundle\XmlConnectorBundle\DependencyInjection;
4
5use Symfony\Component\HttpKernel\DependencyInjection\Extension;
6use Symfony\Component\DependencyInjection\ContainerBuilder;
7use Symfony\Component\DependencyInjection\Loader\YamlFileLoader;
8use Symfony\Component\Config\FileLocator;
9
10class AcmeXmlConnectorExtension extends Extension
11{
12 public function load(array $configs, ContainerBuilder $container)
13 {
14 $loader = new YamlFileLoader($container, new FileLocator(__DIR__.'/../Resources/config'));
15 $loader->load('archiving.yml');
16 $loader->load('jobs.yml');
17 $loader->load('job_parameters.yml');
18 $loader->load('steps.yml');
19 $loader->load('readers.yml');
20 $loader->load('writers.yml');
21 }
22}
Translate Job and Step labels in the UI¶
Behind the scene, the service Akeneo\Platform\Bundle\ImportExportBundle\JobLabel\TranslatedLabelProvider provides translated Job and Step labels to be used in the UI.
This service uses the following conventions: - for a job label, given a %%jobName%%, “batch_jobs.%%jobName%%.label” - for a step label, given a %%jobName%% and a %%stepName%%, “batch_jobs.%%jobName%%.%%stepName%%.label”
Create a file Resources/translations/messages.en.yml in the Bundle to translate label keys.
1acme_xml_connector:
2 jobs:
3 xml_product_import:
4 title: Product Import XML
5 import:
6 title: Product Import Step
7 steps:
8 product_processor.title: Product processor
9 import.filePath:
10 label: File path
11 help: Path of the XML file
12
13
14batch_jobs:
15 xml_product_import:
16 label: Product Import XML
17 import.label: Product Import XML
18
19pim_import_export:
20 download_archive:
21 invalid_xml: Download invalid items in XML
22
23pim_connector:
24 import:
25 xml:
26 enabled:
27 label: Enable the product
28 help: Whether or not the imported product should be enabled
29 categoriesColumn:
30 label: Categories column
31 help: Name of the categories column
32 familyColumn:
33 label: Family column
34 help: Name of the family column
35 groupsColumn:
36 label: Groups column
37 help: Name of the groups column
38 filePath:
39 label: File
40 help: The CSV file to import
41 uploadAllowed:
42 label: Allow file upload
43 help: Whether or not to allow uploading the file directly
44 delimiter:
45 label: Delimiter
46 help: One character used to set the field delimiter for CSV file
47 enclosure:
48 label: Enclosure
49 help: One character used to set the field enclosure
50 escape:
51 label: Escape
52 help: One character used to set the field escape
53 circularRefsChecked:
54 label: Check circular references
55 help: If yes, data will be processed to make sure that there are no circular references between the categories
56 realTimeVersioning:
57 label: Real time history update
58 help: Means that the product history is automatically updated, can be switched off to improve performances
59 copyValuesToProducts:
60 label: Copy variant group values to products
61 help: Means that the products are automatically updated with variant group values, can be switched off to only update variant group
62 enabledComparison:
63 label: Compare values
64 help: Enable the comparison between original values and imported values. Can speed up the import if imported values are very similar to original values
65 decimalSeparator:
66 label: Decimal separator
67 help: One character used to set the field separator for decimal
68 dateFormat:
69 label: Date format
70 help: Specify the format of any date columns in the file, e.g. here DD/MM/YYYY for a 30/04/2014 format.
Use the new Connector¶
Now if you refresh the cache, the new connector and xml job can be found under Collect > Import profiles > Create import profile.
You can create an instance of this job and give it a name like xml_product_import.
Now you can run the job from the UI or use the following command:
php bin/console akeneo:batch:job xml_product_import
Warning
In production, use this command instead:
php bin/console akeneo:batch:publish-job-to-queue my_app_product_export --env=prod
One daemon or several daemon processes have to be started to execute the jobs. Please follow the documentation Setting up the job queue daemon if it’s not the case.
Adding support for invalid items export¶
When the PIM reads the file and processes the entities we want to import, it performs several checks to make sure that the values we import are valid and they respect the constraints we configured the PIM with.
The PIM is then capable of exporting back the invalid items that do not respect those constraints after an import operation.
For our connector to support this feature, we will need to implement a few more parts in our connector:
XmlInvalidItemWriter: a registered XML invalid writer service whose work is to export the invalid lines found during the reading and processing steps.XmlFileIterator: which is used by theXmlInvalidItemWriterto read the imported file to find the invalid items.XmlWriter: Its responsibility is to write the invalid items back into a separate file available for download to the user.
Create an XML file iterator class¶
Let create a class which implements the FileIteratorInterface. This class opens the XML file that was imported thanks to an instance of \SimpleXMLIterator.
We now need to implement the functions of the interface. Here is a working example of the XML iterator:
1<?php
2
3namespace Acme\Bundle\XmlConnectorBundle\Reader\File;
4
5use Akeneo\Tool\Component\Connector\Reader\File\FileIteratorInterface;
6use Symfony\Component\Filesystem\Exception\FileNotFoundException;
7
8class XmlFileIterator implements FileIteratorInterface
9{
10 /** @var string **/
11 protected $type;
12
13 /** @var string **/
14 protected $filePath;
15
16 /** @var \SplFileInfo **/
17 protected $fileInfo;
18
19 /** @var \SimpleXMLIterator */
20 protected $xmlFileIterator;
21
22 /**
23 * {@inheritdoc}
24 */
25 public function __construct($type, $filePath, array $options = [])
26 {
27 $this->type = $type;
28 $this->filePath = $filePath;
29 $this->fileInfo = new \SplFileInfo($filePath);
30
31 if (!$this->fileInfo->isFile()) {
32 throw new FileNotFoundException(sprintf('File "%s" could not be found', $this->filePath));
33 }
34
35 $this->xmlFileIterator = simplexml_load_file($filePath, 'SimpleXMLIterator');
36 $this->xmlFileIterator->rewind();
37 }
38
39 /**
40 * {@inheritdoc}
41 */
42 public function getDirectoryPath()
43 {
44 if (null === $this->archivePath) {
45 return $this->fileInfo->getPath();
46 }
47
48 return $this->archivePath;
49 }
50
51 /**
52 * {@inheritdoc}
53 */
54 public function getHeaders()
55 {
56 $headers = [];
57 foreach ($this->xmlFileIterator->current()->attributes() as $header => $value) {
58 $headers[] = $header;
59 }
60
61 return $headers;
62 }
63
64 /**
65 * {@inheritdoc}
66 */
67 public function current()
68 {
69 $elem = $this->xmlFileIterator->current();
70
71 return $this->xmlElementToFlat($elem);
72 }
73
74 /**
75 * {@inheritdoc}
76 */
77 public function next()
78 {
79 $this->xmlFileIterator->next();
80 }
81
82 /**
83 * {@inheritdoc}
84 */
85 public function key()
86 {
87 return $this->xmlFileIterator->key();
88 }
89
90 /**
91 * {@inheritdoc}
92 */
93 public function valid()
94 {
95 return $this->xmlFileIterator->valid();
96 }
97
98 /**
99 * {@inheritdoc}
100 */
101 public function rewind()
102 {
103 $this->xmlFileIterator->rewind();
104 }
105
106 /**
107 * Converts an xml node into an array of values
108 *
109 * @param \SimpleXMLIterator $elem
110 *
111 * @return array
112 */
113 protected function xmlElementToFlat($elem)
114 {
115 $flatElem = [];
116
117 foreach ($elem->attributes() as $value) {
118 $flatElem[] = (string) $value;
119 }
120
121 return $flatElem;
122 }
123}
Now, let’s declare a simple Symfony service for this class. Here is the Resources/config/readers.yml:
acme_xml_connector.reader.file.xml_iterator_factory:
class: '%pim_connector.reader.file.file_iterator_factory.class%'
arguments:
- '%acme_xml_connector.reader.file.file_iterator.class%'
- 'xml'
Create an XML writer class¶
The XML writer will be responsible for writing the invalid items in a specified file path.
An implementation of it could be:
1<?php
2
3namespace Acme\Bundle\XmlConnectorBundle\Writer;
4
5use Akeneo\Tool\Component\Batch\Item\FlushableInterface;
6use Akeneo\Tool\Component\Batch\Item\InitializableInterface;
7use Akeneo\Tool\Component\Batch\Item\ItemWriterInterface;
8use Akeneo\Tool\Component\Batch\Step\StepExecutionAwareInterface;
9use Akeneo\Tool\Component\Connector\Writer\File\AbstractFileWriter;
10use Akeneo\Tool\Component\Connector\Writer\File\ArchivableWriterInterface;
11
12class XmlWriter extends AbstractFileWriter implements
13 ItemWriterInterface,
14 InitializableInterface,
15 FlushableInterface,
16 ArchivableWriterInterface,
17 StepExecutionAwareInterface
18{
19 /** @var array */
20 protected $writtenFiles = [];
21
22 /** @var \XMLWriter **/
23 protected $xml;
24
25 /**
26 * {@inheritdoc}
27 */
28 public function initialize()
29 {
30 if (null === $this->xml) {
31 $jobParameters = $this->stepExecution->getJobParameters();
32 $filePath = $jobParameters->get('storage')['file_path'];
33
34 $this->xml = new \XMLWriter();
35 $this->xml->openURI($filePath);
36 $this->xml->startDocument('1.0', 'UTF-8');
37 $this->xml->setIndent(4);
38 $this->xml->startElement('products');
39 }
40 }
41
42 /**
43 * {@inheritdoc}
44 */
45 public function getWrittenFiles()
46 {
47 return $this->writtenFiles;
48 }
49
50 /**
51 * {@inheritdoc}
52 */
53 public function write(array $items)
54 {
55 $exportDirectory = dirname($this->getPath());
56 if (!is_dir($exportDirectory)) {
57 $this->localFs->mkdir($exportDirectory);
58 }
59
60 foreach ($items as $item) {
61 $this->xml->startElement('product');
62 foreach ($item as $property => $value) {
63 $this->xml->writeAttribute($property, $value);
64 }
65 $this->xml->endElement();
66 }
67 }
68
69 /**
70 * {@inheritdoc}
71 */
72 public function flush()
73 {
74 $this->xml->endElement();
75 $this->xml->endDocument();
76 $this->xml->flush();
77 $jobParameters = $this->stepExecution->getJobParameters();
78
79 $this->writtenFiles = [$this->stepExecution->getJobParameters()->get('storage')['file_path']];
80 }
81}
Let’s declare a Symfony service for our XML writer in Resources/config/writers.yml:
1parameters:
2 acme_xml_connector.writer.file.xml.class: Acme\Bundle\XmlConnectorBundle\Writer\XmlWriter
3
4services:
5 acme_xml_connector.writer.file.invalid_items_xml:
6 class: '%acme_xml_connector.writer.file.xml.class%'
Note
Please note that every new configuration file created in the Resources/config folder should be loaded in the Symfony dependency injection for it to be taken into account.
Plug it all together¶
Now that our XmlFileIterator class and service are defined, let’s use them in our custom implementation of the XmlInvalidWriterInterface.
Let’s use the existing AbstractInvalidItem to implement our custom class. We only need to implement two functions from our abstract superclass.
getInputFileIterator(JobParameters $jobParameters): that returns a configured instance of our custom reader theXmlFileIteratorclass.setupWriter(JobExecution $jobExecution): sets up our custom writer, an instance of theXmlWriterclass.
Here is a working example:
1<?php
2
3namespace Acme\Bundle\XmlConnectorBundle\Archiver;
4
5use Akeneo\Tool\Component\Batch\Job\JobParameters;
6use Akeneo\Tool\Component\Batch\Model\JobExecution;
7use Akeneo\Tool\Component\Batch\Model\StepExecution;
8use Akeneo\Tool\Component\Connector\Archiver\AbstractInvalidItemWriter;
9
10class XmlInvalidItemWriter extends AbstractInvalidItemWriter
11{
12 /**
13 * {@inheritdoc}
14 */
15 public function getName()
16 {
17 return 'invalid_xml';
18 }
19
20 /**
21 * {@inheritdoc}
22 */
23 protected function getInputFileIterator(JobParameters $jobParameters)
24 {
25 $filePath = $jobParameters->get('storage')['file_path'];
26
27 $fileIterator = $this->fileIteratorFactory->create($filePath);
28 $fileIterator->rewind();
29
30 return $fileIterator;
31 }
32
33 /**
34 * {@inheritdoc}
35 */
36 protected function setupWriter(JobExecution $jobExecution)
37 {
38 $fileKey = strtr($this->getRelativeArchivePath($jobExecution), ['%filename%' => 'invalid_items.xml']);
39 $this->filesystem->put($fileKey, '');
40
41 $writeParams = $this->defaultValuesProvider->getDefaultValues();
42 $writeParams['storage'] = [
43 'type' => 'local',
44 'file_path' => $this->filesystem->getAdapter()->getPathPrefix() . $fileKey,
45 ];
46
47 $writeJobParameters = new JobParameters($writeParams);
48 $writeJobExecution = new JobExecution();
49 $writeJobExecution->setJobParameters($writeJobParameters);
50
51 $stepExecution = new StepExecution('processor', $writeJobExecution);
52 $this->writer->setStepExecution($stepExecution);
53 $this->writer->initialize();
54 }
55}
Let’s define a tagged Symfony service in Resources/config/archiving.yml, so that our custom invalid item writer is taken into account and used by the PIM.
1parameters:
2 acme_xml_connector.archiver.invalid_item_xml_writer.class: Acme\Bundle\XmlConnectorBundle\Archiver\XmlInvalidItemWriter
3
4services:
5 acme_xml_connector.archiver.invalid_item_xml_writer:
6 class: '%acme_xml_connector.archiver.invalid_item_xml_writer.class%'
7 arguments:
8 - '@pim_connector.event_listener.invalid_items_collector'
9 - '@acme_xml_connector.writer.file.invalid_items_xml'
10 - '@acme_xml_connector.reader.file.xml_iterator_factory'
11 - '@oneup_flysystem.archivist_filesystem'
12 - '@acme_xml_connector.job.job_parameters.simple_xml_product_import'
13 - 'xml'
14 tags:
15 - { name: pim_connector.archiver }
Try it out¶
All parts of our connector are now in place for it to be able to export invalid items.
To try it out, run the XML import with the example file products.xml in the UI. At the end of the job execution a new button should appear with the label “Download invalid items in XML”.
Click it and download the XML file containing the invalid items found by the import job.
Found a typo or a hole in the documentation and feel like contributing?
Join us on Github!