• v2.0
    • Versions
    • master
    • v3.2
    • v3.1
    • v3.0
    • v2.3
    • v2.2
    • v2.1
    • v2.0
    • v1.7
    • v1.6
    • v1.5
    • v1.4
    • v1.3
    • v1.2
    • v1.1
    • v1.0
Caution! You are browsing the documentation for Akeneo in version 2.0, which is not maintained anymore.
Consider upgrading to the latest version.

 

  • Install Akeneo PIM
    • Install Akeneo PIM with Docker
    • Install Akeneo PIM manually
      • System Requirements
      • System installation on Debian 9 (Stretch)
      • System installation on Ubuntu 16.04 (Xenial Xerus)
      • Installing Akeneo PIM Community Edition (CE) with the Archive
      • Installing Akeneo PIM Enterprise Edition (EE) with the Archive
      • Setting up the job queue daemon
    • How to customize the Dataset
    • How to Add Translation Packs
  • Migrate Akeneo PIM projects
  • Import and Export data
    • How import works
    • Understanding the Product Import
    • Understanding the Product Export
    • Formats
      • Localized labels
      • Scopable labels
      • Association types data structure
      • Attribute data structure
      • Category data structure
      • Family data structure
      • Family variant data structure
      • Group data structure
      • Options data structure
      • Product data structure
      • Product model data structure
    • Akeneo Connectors
    • How to Customize Import / Export
      • How to create a new Connector
      • How to import Products from a XML file
      • How to clean a CSV file during a Product import
      • How to automate imports/exports
  • Manipulate the Akeneo PIM data
    • How to Customize Mass Edit Operations
      • How to register a new bulk action
      • How to Register a New Mass Edit Action on Products
      • How to Avoid Rules Execution on Mass Edit Actions
    • How to Manipulate Products
      • How to Query Products
      • How to Create Products
      • How to Update Products
      • How to Validate Products
      • How to Save Products
      • How to Remove Products
    • How to Manipulate Non-Product Objects
      • How to Query Non-Product Objects
      • How to Create Non-Product Objects
      • How to Update Non-Product Objects
      • How to Validate Non-Product Objects
      • How to Save Non-Product Objects
      • How to Remove Non-Product Objects
    • How to add a custom action rule
      • General information about rule format
      • How to add a custom action in the rule engine
    • How to Define Access Control List
    • How to Customize the Catalog Structure
      • How to Create a Reference Data
      • How to add a custom unit of measure
      • How to Add New Properties to a Category
    • How to Customize Product Assets
      • How to Add a New Transformation
      • How to Add a Default Thumbnail For Unknown File Types
      • How to connect to an external server for storage
      • How to change the validation rule to match a reference file to an asset
      • How to Mass Import Assets
    • How To Customize Teamwork Assistant (Enterprise Edition)
      • Customize notifications
      • Add a calculation step
      • How to log calculation step
      • Remove projects impacted by a custom catalog update
  • Design the user interfaces
    • How to customize any frontend part of the application
    • How to add an action button or meta data to the product edit form
    • How to add a tab to a form
    • How to add a new tab in System / Configuration
    • How to add custom information to a field
    • How to add a new field type
    • Create a custom product export builder filter
    • How to create the UI to manage a Reference Data
    • How to add a new page
    • How to customize the main menu
    • Styleguide
  • Maintain Akeneo PIM projects
    • First aid kit
    • Bug qualification
    • Common issues
    • Scalability Guide
      • Audit with 3 Representative Catalogs
      • More than 10k attributes?
      • More than 10k families?
      • More than 10k categories?
      • More than 500 attributes usable in the product grids?
      • More than 100k products to export?
      • More than 1GB of product media to export?
    • How to purge history
      • How to Purge jobs executions
      • How to adapt the version purger to your needs
  • Contribute to Akeneo PIM
    • How to report an issue?
    • How to translate the user interface?
    • How to enhance the documentation?
    • How to contribute to a Connector?
    • How to submit a patch to the PIM?
    • How behavior tests are architectured in the PIM?
      • Establishing Decorator Pattern
      • Using Spin In Behat
  • Technical overview
    • Product Information
    • Teamwork Assistant (Enterprise Edition)
      • Project creation
      • Project completeness
      • Project Completeness widget
      • Catalog update impact
      • Scalability guide
      • Users permission summary for Behat tests
    • Collaborative workflow
      • Simple workflow
      • Partial workflow
  • Technical architecture
    • Best Practices
      • Create a project
      • Create a reusable bundle
      • Setup Behat
      • Code Conventions
      • Coding Standards
    • Bundles
    • Components
    • How to implement your business logic using the event system
    • Events
      • Storage events
      • Workflow events (Enterprise Edition only)
    • How to Localize your data
      • How to change the PIM locale
      • How to Use Localizers
      • How to use Presenters
    • How to Add a Notification
    • Performances Guide
      • Memory usage of rules execution (Enterprise Edition)
      • Memory leak fix in Rules Engine (ORM)
      • More than 100 WYSIWYG editors in a page
      • PHP7 and HHVM Compatibility?
    • How to Use the Web REST API
    • Standard format
      • Products
      • Other entities
      • Usage
    • Application Technical Information
      • Application Technical Dependencies
      • Server side set up for hosting
      • System Requirements
      • Recommended configuration
      • Client side configuration and compatibilities
      • Operation processes
      • Flow Matrix
  • Akeneo Cloud Edition
    • Flexibility Mode
      • Overview
      • Partners Starterkit
      • Environments Access
      • Composer settings
      • Periodic tasks / Crontab settings
      • PIM Application
      • PIM Updates and Migrations
      • Data File Transfer
      • Backups management
      • Partners
      • Queue management
      • System Components
      • Disk Usage
    • Serenity Mode
      • Overview
      • PIM Updates and Migrations

Disk Usage¶

Introduction¶

Each Flexibility instance has got its own dedicated disk.

Storage space depends on the contract, which usually starts as follow:

  • 350GB for a production instance
  • 150GB for a sandbox instance

Because each disk is dedicated to only one instance, the storage space of one instance cannot be shared with another instance.

Usages Impacting Disk Usage¶

The disk usage will inscrease following the lifecycle of the instance. For example, each time:

  • a new product / family / … is created
    • entries are created in database
  • a product / family / … is modified
    • previous version is stored in versioning table
    • by default, old versions are kept forever (as of v3.2.7)
  • an asset is uploaded
    • it is stored on the file system
    • if the asset is replaced with a new one, the old version stays on disk
  • an import / export runs
    • it creates an archive
    • by default, those archived are retained during 180 days (as of v3.2.7)
  • a file is uploaded using SFTP / scp / …

Impact of Disk Usage on the PIM¶

When the disk is full, the PIM cannot work anymore: it cannot save modifications in database, upload new asset, …

It can even prevent ElasticSearch from updating its indexes, resulting in divergence with the database. To avoid those issues, ensuring a healthy disk usage is necessary.

Improving Disk Usage¶

To reduce disk usage, some temporary files are deleted automatically on a regular basis. For example:

  • old temporary import/export files
  • old temporary file storage
  • akeneo_batch directories
  • ansible temporary directories

Moreover, the integrator can:

  • activate the purge of old versions of products
  • remove assets that are not linked to any product anymore
mkdir -p /home/akeneo/purge

echo "Track assets that will be deleted in a csv file"
mysql akeneo_pim -e "SELECT fi.id, fi.original_filename, fi.file_key, r.file_info_id, r.asset_id, v.* FROM akeneo_file_storage_file_info fi LEFT JOIN pimee_product_asset_reference r ON fi.id = r.file_info_id LEFT JOIN pimee_product_asset_variation v ON fi.id = v.file_info_id WHERE storage = 'assetStorage' AND r.file_info_id IS NULL AND r.asset_id IS NULL AND v.source_file_info_id IS NULL" > /home/akeneo/purge/asset_rows_to_delete.csv

echo "List files to be deleted in a txt file"
echo "Path of files to be deleted is calculated by concatenating "/home/akeneo/pim/app/file_storage/asset/" with value of "fi.file_key" from the MySQL resquest."
mysql akeneo_pim -se "SELECT concat('/home/akeneo/pim/app/file_storage/asset/',fi.file_key) FROM akeneo_file_storage_file_info fi LEFT JOIN pimee_product_asset_reference r ON fi.id = r.file_info_id LEFT JOIN pimee_product_asset_variation v ON fi.id = v.file_info_id WHERE storage = 'assetStorage' AND r.file_info_id IS NULL AND r.asset_id IS NULL AND v.source_file_info_id IS NULL" > /home/akeneo/purge/asset_files_to_delete.txt

echo "Delete rows in database"
mysql akeneo_pim -se "SET FOREIGN_KEY_CHECKS=0;DELETE v, r, fi FROM akeneo_file_storage_file_info fi LEFT JOIN pimee_product_asset_reference r ON fi.id = r.file_info_id LEFT JOIN pimee_product_asset_variation v ON fi.id = v.file_info_id WHERE storage = 'assetStorage' AND r.file_info_id IS NULL AND r.asset_id IS NULL AND v.source_file_info_id IS NULL;SET FOREIGN_KEY_CHECKS=1;" > /home/akeneo/purge/asset_rows_deleted.csv

echo "Delete assets files based on previous list"
cat /home/akeneo/purge/asset_files_to_delete.txt | xargs rm -f
  • purge old versions from versioning table
echo "Check pim_versioning_version table size"
mysql akeneo_pim -e 'SELECT table_name AS `Table`, round(((data_length + index_length) / 1024 / 1024 / 1024), 2) `Size in GB` FROM information_schema.TABLES WHERE table_schema = "akeneo_pim" AND table_name = "pim_versioning_version";'

Warning

Warning: mysqlcheck –optimize will duplicate the table(s) before optimizing it (them). Which means that, before running the command, one must make sure that the remaining free space is at least equals to the biggest table size. To avoid any data loss, backing the tables up before running mysqlcheck is prefered. For more information: https://dev.mysql.com/doc/refman/5.7/en/mysqlcheck.html

Warning

Warning: mysqlcheck –optimize will lock the table during the operation. Hence the table will be unavailable for the PIM. For more information: https://dev.mysql.com/doc/refman/5.7/en/mysqlcheck.html

mkdir -p /home/akeneo/purge

echo "Cleansing versions older than 90 days"
nohup php bin/console --env=prod pim:versioning:purge --more-than-days 90 --force -n &

echo "Shrink MySQL tables"
nohup mysqlcheck --optimize akeneo_pim pim_versioning_version &

Moreover, the customer and the integrator can:

  • open a ticket to ask the Cloud Team to set the duration of retention of archives of import / export
  • contact the Customer Success Manager to upscale the disk

Found a typo or a hole in the documentation and feel like contributing?
Join us on Github!