Magento Multi Store shop designed for high load
This case study describes a project we delivered in 2018. The shop ran several stores on Magento 1.9 and Magento 2.0, and the diagram below shows the infrastructure we built for it.
Goals, needs, and wants
The goal of the solution was to create redundant, stable, and lightweight infrastructure for a Magento multi-store shop at the same time. The client had a development team working on the shop's updates and functionality improvement, so we suggested implementing Continuous Integration and Continuous Deployment into their workflow to ease the development process and prevent various errors and issues that may arise during direct deployment to production.
Containerized and automated solution
We divided the whole infrastructure into independent parts, which were packed with Docker. For the containers' orchestration, we used Rancher. For deployment automation, Jenkins was chosen.
Ansible playbooks were used to automate the configuration of the environments and the interaction between infrastructure components. Jenkins interacted with the client's GitHub, where the application's code was stored.
Proofs
Load tests showed that the infrastructure successfully handled ~86,000 visitors at one time. Load balancing was implemented with HAProxy, which distributed the load between 2 production environments.
CI & CD allows us to deploy the code to the Development env and test new functionality there. After a successful review, the code goes to the Staging env, where it's kept until the final review by the QA lead. After their final approval, the QA lead pushes the changes to the Production env.
Results and more
The result of our work is shown in the diagram at the top of this case study. The goals, needs, and wants of our client were fully covered by the provided solution. Besides Dockerization, we also performed backend optimization and gave the development team detailed information about the application's performance bottlenecks and security weaknesses, as a quality product should be tuned and secured from all sides.
How we would build it today
The approach still holds: separate environments, automated deployments, load balancing and caching in front of the application. Most of the 2018 stack has changed, though. Magento 1 reached end of life in June 2020, and Rancher 1.6, the Docker orchestration we used, reached end of life the same month.
Today we would build the same shop on Magento Open Source or Adobe Commerce 2.4.x, currently 2.4.9, with the services Adobe lists in its system requirements for that release: PHP 8.5, MySQL 8.4 or MariaDB, OpenSearch instead of Elasticsearch, and Valkey instead of Redis. Docker Compose is enough for development environments and smaller setups. For production at this scale, Kubernetes with autoscaling and a managed load balancer replaces manual scaling, and a pipeline in GitHub Actions or GitLab CI/CD does the job of Jenkins. Read more about how such pipelines work in our post on continuous integration.
If you're starting with a single server, our step-by-step Magento 2.4.9 installation guide with Nginx covers the whole setup from a clean Ubuntu server to a working store.