Audit with 3 Representative Catalogs

Warning

This page is an early version, we’ll continue to complete it with more use cases.

We’ve audited the application with 3 different representative catalogs:

Catalog Small Medium Large
Products 5.000 50.000 1.000.000
Categories 500 2.000 4.000
Categories / product 2 2 4
Attributes 100 400 1.000
Attribute Groups 8 15 20
Attributes / Families 50 100 100
% filled attributes 75% 75% 50%
%localisable attributes 10% 5% 2%
%scopable attributes 5% 2% 1%
%scopable + localisable attributes 2% 1% < 1%
Families 20 50 400
Channels 2 2 2
Enabled Locales 1 4 4
Audit Status for Community Edition Ok Ok Ok
Audit Status for Enterprise Edition Ok Ok Ok

These catalogs are available in our dedicated repository https://github.com/akeneo/catalogs, you can also use our data generator to build your own test catalog https://github.com/akeneo-labs/DataGeneratorBundle.

Several of our customers strongly push these limitations in their custom projects, you can consult different use cases in this page. We adapt these representative catalogs between minor versions when we improve the application scalability. Don’t hesitate to consult this page for other versions.

How we tested?

Installation

The application is installed on a server following the recommended architecture Application Technical Information.

Depending on the catalog, we used a different database storage. We installed the data fixtures via the installer before importing the products through the default product csv import job (for a large product import, we split the catalog into 10 files + parallel imports + custom optimisations).

The targeted amount of product values implies to choose a relevant database storage, if you want to know more about how to choose the right storage strategy, please read More than 5 million product values?.

Depending on the data volume, the number of field indexes in MongoDB also impacts the performances, please read More than 64 indexes with MongoDB? for more information.

  Storage Product values Note
Small MySQL 159.676  
Medium MySQL 3.661.981  
Large MongoDB 52.699.463 With more than ~300k products or ~15M product values, you should use the additional ElasticSearchBundle (contact us)

Audit User Interface

We use the application in production mode, with xdebug disabled, and we expect an optimal user experience for each page and action.

Audit Backend Processes

We run backend processes (bulk actions, imports, exports, rules execution, etc) in production mode, with xdebug disabled. Processes may run for quite a long time depending on the amount of data but do not consume more memory than the volume advised in Application Technical Information. Please note that for some project data set, several extra configurations are required (for instance, changing the size of a bulk of products for the rules execution).

Automation

We have built several tools to automate these performance and scalability tests. Basically, our continuous integration loads a target catalog and then runs different scenarios. The build is considered to fail when thresholds on time execution and memory usage are reached. These tools are not open sourced for now.

Known limitations on representative catalogs

We observed two kinds of limitations during the audit: memory leaks and scalability limitations.

Memory leaks when a process consumes more memory than recommended. These issues are qualified as bugs, their fixes are released in version patches.

Scalability limitations, when we try to support larger data volume for an axis. These issues are qualified as improvements which are released in minor versions.

The following limitations have been encountered with standard installations without any custom code:

Type Catalog Edition Released Note
memleak All All v1.4.23 (PIM-5507) Memory leak when executing mass edit or mass publish on several thousands of products
improv. Medium All TODO (PIM-5467) First load of the completeness widget is too long (ORM)
improv. Large All TODO (PIM-5518) Timeout with synchronous update of products when remove ‘AssociationType’, ‘Attribute’, ‘AttributeOption’, ‘Category’, ‘Family’, ‘Group’, ‘Channel’
improv. Large All v1.6.0 (PIM-5542) the request /configuration/family/rest slow down the UI (dashboard, grid, pef)
improv. Large Enterprise TODO (PIM-5544) the request /enrich/product-category-tree/list-tree.json allowing to load the tree on the grid is very slow (improved with Elastic Search Bundle)
improv. Large All TODO MongoDB timeout when filtering and sorting on product grid when using not indexed fields (improved with Elastic Search Bundle)

Examples of customers instance

Several customers challenge the limitations even more in their custom projects and it requires custom optimizations sometimes. We continuously improve the product scalability in each minor version and we are always interested in new use cases to investigate. Don’t hesitate to contact us if you need help to scale your instance.

On standard axes:

Catalog Customer 1 Customer 2 Customer 3 Customer 4 Details about limitations
Storage MongoDB + ES MongoDB + ES MySQL MySQL ES: ElasticSearch Bundle
Products 2.000.000 1.100.041 80.000 10.000  
Products values 43.398.847 78.606.501 6.000.000 70.000 6 millions product values is a high limit for MySQL storage
Attributes 1.800 8.272 240 355 More than 10k attributes?
Families 131 3.546 44 3 More than 10k families?
Categories 2613 14.238 740 60 More than 10k categories?
Channels 1 2 2 14  
Enabled Locales 1 1 36 1  

On other axes or combinations:

  Tested In custom project Details about limitations
Attribute options 95.000    
Attribute options per attribute 500    
Reference data [WIP]    
Reference data per attribute [WIP]    
Products per family [WIP] 1.000.000 cf following PIM-5563
Product groups 10.000   cf following PIM-5519, PIM-5363
Products per product group 50    
Product variant groups 10.000   cf following PIM-5467, PIM-5520, PIM-5363
Products per product variant group 50    
Product values per variant group 50    
Product associations [WIP]   cf following PIM-5363, PIM-5562
Attributes per family 150    
Attributes per attribute group 150 1.500  
Product values per product 200    
Rules 150 3.000 Memory usage of rules execution (Enterprise Edition)
Product assets [WIP]    
Product drafts [WIP]    

Known limitations on other axes or combinations

Type Catalog Edition Released Note
improv. All All TODO (PIM-5519) Mass edit products, display the add to a group configuration is too long with a lot of product groups (use a paginated select2 and not checkboxes)
improv. All All TODO (PIM-5520) Mass edit products, display the add to a variant group configuration is too long with a lot of product groups (use a paginated select2)
improv. All All TODO (PIM-5467) When saving a variant group, variant group values are synchronously copied in products, it may cause timeout issue
improv. All All TODO (PIM-5463) When associating a lot of products to a group, variant group or association, you may encounter “The requested URL’s length exceeds the capacity”
improv. All All TODO (PIM-5562) When delete a product associated to other products, run a backend process to cleanup all associations
improv. All All TODO (PIM-5563) Query for completeness rescheduling when saving a family with 50k products inside is too long to execute, exec as backend process
improv. All All TODO (PIM-5861) Cannot remove an attribute on big (~1M products) MongoDB catalog because of timeout when we update MongoDB database to remove it from all products
improv. All All TODO (IM-766) Variant groups scalability: limit number of axes or set a limit?