Adobe Commerce Magento 2.4 vs. Sylius a Symfony based eCommerce Framework
Magento is a popular legacy CMS platform for running an online store. It is chosen by people who want to create a large online store with many functionalities that facilitate the management of a small number of products. Unfortunately, this is done by quite high requirements of server price and bad out-of-the-box performance.
When you are using Magento CMS version 2, you have to be aware that its core and 3-d party modules optimizations (thanks to the broken M2 ecosystem) will cost more money than initial development using outsourced services. Magento 2 is a product of sales technology, not a modern software project. We can check what the main achievements of the Magento 2 are:
Magento Commerce was acquired by the Permira funds from eBay Inc. in 2015 and received an investment from Hillhouse Capital in 2017. Now it is Acquired by Adobe. Over the past three years, Magento has undergone a period of rapid sales growth and transformation, which included:
- Creating a cloud-first platform for businesses of all sizes, enabling the digital transformation from on-premise to cloud commerce systems, including advanced omnichannel, order management, and business intelligence capabilities;
- Accelerating Magento’s global growth by enhancing operational execution and improving the integration of Solution and Technology Partners, which included open-source ecosystem monetization, strategic deployment of capital to acquire core functionality and adjacent businesses; and
- Acquiring PageBuilder 3-d party extension and BI startup extension.
- Generate thousands of new bugs
The Magento 2 team's biggest achievement is a sale of this half-backed “prototype” to the folks from Adobe as Cloud First Platform, scalable and user friendly. However, Adobe has realized that Magento 2 may not be so cloudy as was advertised originally.
Let’s check how Adobe improved Magento and its performance last years versus the modern Sylius Symfony based e-commerce platform.
Sylius Ecommerce Paltform
Sylius is a next-generation European Open Source eCommerce Platform. Being built on top of the enterprise-grade Symfony framework versus Magento's self-designed bicycle on top of the legacy Zend 1. It is the most modern and stable foundation for professional eCommerce projects. It has 160 open issues vs. 2000+ in the Magento 2 repository.
This Platform solves the challenge of selling online through various channels. Magento’s CMS legacy “Websites” and “store views,” EAV are the solutions of the past. Silius focus on providing the foundation for the professional development of omnichannel eCommerce for SMB and Enterprise Businesses.
Sylius is not CMS like Magento. It is an e-Commerce framework focusing on great developer experience and the flexibility that allows implementing even the most challenging ideas.
Also, Sylius is not just a business which just selling licenses and reselling broken cloud solutions. Sylius provides training, consulting, and development.
Let's Go.
We will measure the Time To First Byte(TTFB) metric of these two incredible eCommerce platforms.
TTFB is a measurement of how long it takes the first bits to reach the browser from the AWS Cloud server's initial request.
TTFB is a very key metric because the browser cannot begin rendering web page content until it has the initial markup. Google recommending to have TTFB around 300 ms.
What is a good back-end TTFB time?
A good target for just the processing time for back-end requests is on the order of 100ms (0.1 seconds). That doesn’t mean you should expect a TTFB of 100ms, just that the back-end processing time shouldn’t take longer than that. It is important to remember that the user can’t see ANYTHING at all before the TTFB, so any improvements there go directly to the user experience.
96% percentile
In my previous tests, I have found one critical issue. I have measured the average or median response time. However, for the best evaluation, we should measure the 95% percentile.
Average response time and measurement of a Varnish speed were designed to mislead about real platform performance. To build a modern website, you need something more than just static generated HTML for all visitors.
Why use the 95% percentile for eCommerce?
The 95th percentile encompasses the experience of almost all of your eCommerce users, with only the most severe outliers excluded. This makes it perfect for spotting short-term trends and anomalies.
We are having 2 optimized for PHP C5.large instances with default modules and sample data for the test.
A big thanks to the BitBag (https://bitbag.io/) company for the test environment setup and additional consulting.
We will do testing using the K6 Magento performance toolkit and analytic tool.
Each test has 20 iterations with 1 simultaneous user. Each server will have its own K6 software installed, and tests will be made from the same server to avoid network overhead.
Install K6 performance tool on Ubuntu:
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 379CE192D401AB61echo "deb https://dl.bintray.com/loadimpact/deb stable main" | sudo tee -a /etc/apt/sources.listsudo apt-get updatesudo apt-get install k6
We will test the main e-commerce scenarios :
- Simple Product Page
- Configurable Page
- Category Page
- CMS page
- Add to cart AJAX
- Cart Page
- API Performance
- Frontend Performance
- Backend Throughput Performance
Simple Product Performance:
Magento 2
data_received..............: 1.5 MB 96 kB/s
data_sent..................: 2.6 kB 166 B/s
http_req_blocked...........: min=3.58µs med=3.97µs avg=40.81µs max=725.35µs p(95)=51.81µs
http_req_connecting........: min=0s med=0s avg=34.05µs max=681.03µs p(95)=34.05µs
http_req_duration..........: min=256.1ms med=258.56ms avg=261.41ms max=301.36ms p(95)=266.07ms
http_req_receiving.........: min=1.18ms med=1.25ms avg=1.65ms max=3.88ms p(95)=3.37ms
http_req_sending...........: min=17.8µs med=21.21µs avg=23.88µs max=52.48µs p(95)=35.74µs
http_req_tls_handshaking...: min=0s med=0s avg=0s max=0s p(95)=0s
http_req_waiting...........: min=254.89ms med=256.52ms avg=259.73ms max=299.59ms p(95)=264.35ms
http_reqs..................: 20 1.293711/s
iteration_duration.........: min=91.01µs med=768.37ms avg=701.89ms max=812.87ms p(95)=774.94ms
iterations.................: 20 1.293711/sResults:
Min: 254.89ms
Max: 299.59ms
95%: 264.35ms
Average: 259.73ms
Sylius
data_received..............: 308 kB 28 kB/s
data_sent..................: 2.9 kB 267 B/s
http_req_blocked...........: min=3.85µs med=4.46µs avg=22.12µs max=357.15µs p(95)=22.91µs
http_req_connecting........: min=0s med=0s avg=15.25µs max=305.11µs p(95)=15.25µs
http_req_duration..........: min=34.5ms med=35.18ms avg=35.22ms max=36.77ms p(95)=35.98ms
http_req_receiving.........: min=208.43µs med=250.7µs avg=251.95µs max=303.04µs p(95)=301.33µs
http_req_sending...........: min=14.45µs med=20.61µs avg=21.79µs max=54.6µs p(95)=24.98µs
http_req_tls_handshaking...: min=0s med=0s avg=0s max=0s p(95)=0s
http_req_waiting...........: min=34.25ms med=34.88ms avg=34.95ms max=36.52ms p(95)=35.71ms
http_reqs..................: 20 1.824021/s
iteration_duration.........: min=104.52µs med=545.72ms avg=496.21ms max=547.38ms p(95)=546.58ms
iterations.................: 20 1.824021/sResults:
Min: 34.25ms
Max: 34.25ms
95%: 35.71.ms
Average: 34.96
Configurable Product Performance
Magento 2
data_received..............: 2.0 MB 126 kB/s
data_sent..................: 2.4 kB 149 B/s
http_req_blocked...........: min=3.35µs med=4.12µs avg=40.38µs max=728.48µs p(95)=41.29µs
http_req_connecting........: min=0s med=0s avg=33.43µs max=668.6µs p(95)=33.43µs
http_req_duration..........: min=289.23ms med=292.9ms avg=296.08ms max=328.83ms p(95)=316.37ms
http_req_receiving.........: min=1.75ms med=1.79ms avg=2.02ms max=3.95ms p(95)=3.03ms
http_req_sending...........: min=15.73µs med=20.48µs avg=22.46µs max=58.44µs p(95)=27.21µs
http_req_tls_handshaking...: min=0s med=0s avg=0s max=0s p(95)=0s
http_req_waiting...........: min=287.46ms med=291.02ms avg=294.04ms max=326.99ms p(95)=314.52ms
http_reqs..................: 20 1.238122/s
iteration_duration.........: min=97.55µs med=803.06ms avg=733.41ms max=839.41ms p(95)=826.3ms
iterations.................: 20 1.238122/sResults:
Min: 287.46ms
Max: 326.99ms
95%: 314.52ms
Average: 294.04ms
Sylius
http_req_blocked...........: min=3.59µs med=4.18µs avg=20.01µs max=297.79µs p(95)=35.58µs
http_req_connecting........: min=0s med=0s avg=12.43µs max=248.61µs p(95)=12.43µs
http_req_duration..........: min=42.08ms med=42.79ms avg=42.82ms max=43.45ms p(95)=43.34ms
http_req_receiving.........: min=148.74µs med=205.89µs avg=214.5µs max=295.17µs p(95)=292.59µs
http_req_sending...........: min=13.87µs med=21.09µs avg=22.83µs max=54.71µs p(95)=30.84µs
http_req_tls_handshaking...: min=0s med=0s avg=0s max=0s p(95)=0s
http_req_waiting...........: min=41.86ms med=42.53ms avg=42.58ms max=43.22ms p(95)=43.13ms
http_reqs..................: 20 1.803815/s
iteration_duration.........: min=99.2µs med=553.32ms avg=503.16ms max=554.54ms p(95)=554.04ms
iterations.................: 20 1.803815/sResults:
Min: 41.86ms
Max: 43.22ms
95%: 43.13ms
Average: 42.58ms
Category Page performance :
Magento 2
data_received..............: 4.5 MB 276 kB/s
data_sent..................: 2.6 kB 159 B/s
http_req_blocked...........: min=3.68µs med=4.18µs avg=41.48µs max=749.03µs p(95)=43.83µs
http_req_connecting........: min=0s med=0s avg=34.48µs max=689.61µs p(95)=34.48µs
http_req_duration..........: min=286.62ms med=290.68ms avg=300.36ms max=432.63ms p(95)=322.98ms
http_req_receiving.........: min=2.55ms med=2.72ms avg=3.32ms max=5.96ms p(95)=5.84ms
http_req_sending...........: min=14.96µs med=20.52µs avg=24.18µs max=54.73µs p(95)=50.93µs
http_req_tls_handshaking...: min=0s med=0s avg=0s max=0s p(95)=0s
http_req_waiting...........: min=283.93ms med=287.27ms avg=297.01ms max=429.88ms p(95)=317.24ms
http_reqs..................: 20 1.230701/s
iteration_duration.........: min=94.81µs med=800.45ms avg=737.32ms max=943.21ms p(95)=828.51ms
iterations.................: 20 1.230701/sResults:
Min: 283.93ms
Max: 429.88ms
95%: 317.24ms
Average: 297.01ms
Sylius
http_req_blocked...........: min=3.71µs med=4.36µs avg=20.16µs max=313.03µs p(95)=26.74µs
http_req_connecting........: min=0s med=0s avg=13.21µs max=264.36µs p(95)=13.21µs
http_req_duration..........: min=34.18ms med=35.04ms avg=35.07ms max=36.15ms p(95)=36.08ms
http_req_receiving.........: min=160.24µs med=187.16µs avg=197.86µs max=308.15µs p(95)=277.06µs
http_req_sending...........: min=15.64µs med=20.75µs avg=26.67µs max=120.14µs p(95)=46.27µs
http_req_tls_handshaking...: min=0s med=0s avg=0s max=0s p(95)=0s
http_req_waiting...........: min=33.89ms med=34.8ms avg=34.84ms max=35.95ms p(95)=35.83ms
http_reqs..................: 20 1.829379/s
iteration_duration.........: min=106.61µs med=545.62ms avg=496.1ms max=546.73ms p(95)=546.62ms
iterations.................: 20 1.829379/sResults:
Min: 33.89ms
Max: 35.95ms
95%: 35.83ms
Average: 34.84ms
CMS Page Performance:
CMS page is almost “Hellow World” performance test
Magento 2
http_req_blocked...........: min=3.75µs med=4.59µs avg=41.31µs max=734.21µs p(95)=47.12µs
http_req_connecting........: min=0s med=0s avg=33.96µs max=679.2µs p(95)=33.96µs
http_req_duration..........: min=160.82ms med=162.08ms avg=163.41ms max=178.67ms p(95)=168.22ms
http_req_receiving.........: min=1.23ms med=1.27ms avg=1.91ms max=4.43ms p(95)=3.59ms
http_req_sending...........: min=18.83µs med=20.57µs avg=22.34µs max=53.29µs p(95)=24.37µs
http_req_tls_handshaking...: min=0s med=0s avg=0s max=0s p(95)=0s
http_req_waiting...........: min=159.54ms med=160.34ms avg=161.47ms max=177.33ms p(95)=163.92ms
http_reqs..................: 20 1.47991/s
iteration_duration.........: min=125.37µs med=672.39ms avg=612.85ms max=690.14ms p(95)=678.12ms
iterations.................: 20 1.47991/sResults:
Min: 159.54ms
Max: 177.33ms
95%: 163.92ms
Average: 161.47ms
Sylius
http_req_blocked...........: min=3.78µs med=4.24µs avg=20.48µs max=324.25µs p(95)=22.94µs
http_req_connecting........: min=0s med=0s avg=13.93µs max=278.66µs p(95)=13.93µs
http_req_duration..........: min=21ms med=21.51ms avg=21.63ms max=22.95ms p(95)=22.35ms
http_req_receiving.........: min=72.3µs med=103.61µs avg=112.48µs max=256.24µs p(95)=214.19µs
http_req_sending...........: min=18.75µs med=20.88µs avg=23.12µs max=59.99µs p(95)=27.36µs
http_req_tls_handshaking...: min=0s med=0s avg=0s max=0s p(95)=0s
http_req_waiting...........: min=20.91ms med=21.39ms avg=21.49ms max=22.68ms p(95)=22.23ms
http_reqs..................: 20 1.872535/s
iteration_duration.........: min=125.18µs med=532.11ms avg=483.91ms max=534.13ms p(95)=533ms
iterations.................: 20 1.872535/sResults:
Min: 20.91ms
Max: 22.68ms
95%: 22.23ms
Average: 21.49ms
Add to Cart AJAX
I will send add to cart request from the browser:
Magento 2:
Result: 497ms
Sylius
Result: 69.44ms
Cart Page :
Magento 2
Result: 365ms
Sylius:
Result:90 ms
Rest API Performance Authentication :
Rest API regularly faster because it doesn't need to generate HTML content. It has just simple JSON or XML.
Magento 2
http_req_blocked...........: min=3.5µs med=4.1µs avg=42.26µs max=758.93µs p(95)=50.52µs
http_req_connecting........: min=0s med=0s avg=35.44µs max=708.93µs p(95)=35.44µs
http_req_duration..........: min=71.45ms med=72.21ms avg=73.18ms max=83.79ms p(95)=75.61ms
http_req_receiving.........: min=49.53µs med=53.71µs avg=55.1µs max=62.9µs p(95)=61.52µs
http_req_sending...........: min=19.09µs med=21.88µs avg=23.89µs max=55.19µs p(95)=35.01µs
http_req_tls_handshaking...: min=0s med=0s avg=0s max=0s p(95)=0s
http_req_waiting...........: min=71.38ms med=72.13ms avg=73.1ms max=83.67ms p(95)=75.52ms
http_reqs..................: 20 1.710087/s
iteration_duration.........: min=120.96µs med=582.8ms avg=530.78ms max=595.43ms p(95)=585.83ms
iterations.................: 20 1.710087/sResults:
Min: 71.38ms
Max: 83.67ms
95%: 75.52ms
Average: 73.1ms
Sylius:
http_req_blocked...........: min=3.62µs med=4.18µs avg=20.8µs max=335.95µs p(95)=21.47µs
http_req_connecting........: min=0s med=0s avg=14.26µs max=285.33µs p(95)=14.26µs
http_req_duration..........: min=8.87ms med=9.13ms avg=9.14ms max=9.51ms p(95)=9.47ms
http_req_receiving.........: min=56.73µs med=75.62µs avg=75.96µs max=92.53µs p(95)=89.38µs
http_req_sending...........: min=16.58µs med=20.24µs avg=22.55µs max=55.75µs p(95)=28.71µs
http_req_tls_handshaking...: min=0s med=0s avg=0s max=0s p(95)=0s
http_req_waiting...........: min=8.78ms med=9.03ms avg=9.04ms max=9.38ms p(95)=9.37ms
http_reqs..................: 20 1.92064/s
iteration_duration.........: min=95.44µs med=519.67ms avg=472.5ms max=520.64ms p(95)=520.02ms
iterations.................: 20 1.92064/sResults:
Min: 8.78ms
Max: 9.38ms
95%: 9.37ms
Average: 9.04ms
So we can see that even such a huge company as Adobe Cant improves Magento's performance and make it a modern fast platform for eCommerce.
Pivot Table and Chart
+----------------------+-------------+--------+
| | Magento 2.4 | Sylius |
+----------------------+-------------+--------+
| Simple Product | 264.35 | 35.71 |
+----------------------+-------------+--------+
| Configurable Product | 314.52 | 43.13 |
+----------------------+-------------+--------+
| Category | 317.24 | 35.83 |
+----------------------+-------------+--------+
| CMS Page | 163.92 | 22.23 |
+----------------------+-------------+--------+
| Add to Cart AJAX | 497 | 69.44 |
+----------------------+-------------+--------+
| Cart Page | 365 | 90 |
+----------------------+-------------+--------+
| API Get Category | 75.52 | 9.37 |
+----------------------+-------------+--------+
Frontend Performance Product Page
Magento Bundling is disabled to achieve better performance.
First Contentful Paint: 2.2 s
First Contentful Paint marks the time at which the first text or image is painted.
Speed Index: 16.1 s
Speed Index shows how quickly the contents of a page are visibly populated. Learn more.
Largest Contentful Paint: 3.4 s
Largest Contentful Paint marks the time at which the largest text or image is painted. Learn More
Time to Interactive: 28.2 s
Time to interactive is the amount of time it takes for the page to become fully interactive. Learn more.
Total Blocking Time: 240 ms
Sum of all time periods between FCP and Time to Interactive, when task length exceeded 50ms, expressed in milliseconds. Learn more.
Cumulative Layout Shift: 0.441
Cumulative Layout Shift measures the movement of visible elements within the viewport. Learn more.
Content Breakdown:
+-----------+---------+--------------+
| MIME Type | Bytes | Uncompressed |
+-----------+---------+--------------+
| js | 659,494 | 2,625,687 |
+-----------+---------+--------------+
| image | 90,571 | 92,565 |
+-----------+---------+--------------+
| font | 74,204 | 74,204 |
+-----------+---------+--------------+
| css | 70,494 | 465,877 |
+-----------+---------+--------------+
| html | 23,654 | 109,305 |
+-----------+---------+--------------+
| other | 249 | 249 |
+-----------+---------+--------------+
| flash | 0 | 0 |
+-----------+---------+--------------+
| video | 0 | 0 |
+-----------+---------+--------------+Sylius:
Content Breakdown:
+-----------+---------+--------------+
| MIME Type | Bytes | Uncompressed |
+-----------+---------+--------------+
| image | 332,737 | 332,737 |
+-----------+---------+--------------+
| js | 144,416 | 507,928 |
+-----------+---------+--------------+
| css | 106,771 | 643,109 |
+-----------+---------+--------------+
| font | 95,344 | 95,344 |
+-----------+---------+--------------+
| html | 4,342 | 18,058 |
+-----------+---------+--------------+
| flash | 0 | 0 |
+-----------+---------+--------------+
| other | 0 | 0 |
+-----------+---------+--------------+
| video | 0 | 0 |
+-----------+---------+--------------+Sylius needs some improvements like deffer JS. Magento has it occasionally because of Require JS async script loading.
Magento Adobe Commerce also has no bad performance; however, only in the default configuration after 1-st party and3-d party customizations can make merchants dream about 1s TTFB. And under the load, Magento is unpredictable. It can abuse MySQL or Redis server or even Elastic Search.
Let's check the TTFB of the Helly Hansen Magento implementation:
1549 ms time to first-bite. By the way, I see Helly Hansen moved out from the Magento cloud and has better performance. My previous test result was:
2. Product Page rendering
Magento Performance Times (ms)
min mean[+/-sd] median max
Connect: 38 38 0.0 38 38
Processing: 4004 4004 0.0 4004 4004
Waiting: 3580 3580 0.0 3580 3580
Total: 4042 4042 0.0 4042 4042It is the subject of a separate investigation…
Details of the Magento Adobe Commerce Cloud performance you can see there:
A real Magento Commerce projects are always slow, both Backend and Frontend.
A real Sylius Project has TTFB: 253.5ms
https://smathersandbranson.com/product/stars-and-stripes-needlepoint-pillow
http_req_blocked...........: min=132.34ms med=132.34ms avg=132.34ms max=132.34ms p(95)=132.34ms
http_req_connecting........: min=64.95ms med=64.95ms avg=64.95ms max=64.95ms p(95)=64.95ms
http_req_duration..........: min=448.19ms med=448.19ms avg=448.19ms max=448.19ms p(95)=448.19ms
http_req_receiving.........: min=194.63ms med=194.63ms avg=194.63ms max=194.63ms p(95)=194.63ms
http_req_sending...........: min=61.54µs med=61.54µs avg=61.54µs max=61.54µs p(95)=61.54µs
http_req_tls_handshaking...: min=66.77ms med=66.77ms avg=66.77ms max=66.77ms p(95)=66.77ms
http_req_waiting...........: min=253.5ms med=253.5ms avg=253.5ms max=253.5ms p(95)=253.5ms
http_reqs..................: 1 0.900963/s
iteration_duration.........: min=114.27µs med=197.35µs avg=363.9ms max=1.09s p(95)=982.27ms
iterations.................: 1 0.900963/sThroughput performance.
We are having etalon instance C5.large 2vCPU 77$/months with everything installed on it.
To understand how many CPU and instance sizes we need, we should understand how many users a single instance can handle.
The basic formula will be.
Physical CPU count = TTFB(seconds) * Users per second. In our case, we have minimal network latency TTFB = CPU time spent to generate the page.
However, Magento also has many AJAX requests and heavy cron jobs, but we will not count this. Intel's CPUs (Xeon, I9, I7, etc.) have virtualization 2 threads share one physical CPU, so 1 physical CPU = 2vCPU.
Category page — 5,20,40 users
Magento 2, 5 customers.
data_received..............: 4.8 MB 559 kB/s
data_sent..................: 4.8 kB 560 B/s
http_req_blocked...........: min=2.01µs med=3.08µs avg=85.45µs max=752.65µs p(95)=637.12µs
http_req_connecting........: min=0s med=0s avg=78.23µs max=709.66µs p(95)=605.61µs
http_req_duration..........: min=482.52ms med=1.05s avg=1.06s max=1.63s p(95)=1.3s
http_req_receiving.........: min=1.86ms med=6.62ms avg=10.01ms max=34.82ms p(95)=23.26ms
http_req_sending...........: min=10.48µs med=17.95µs avg=20.68µs max=77.18µs p(95)=39.81µs
http_req_tls_handshaking...: min=0s med=0s avg=0s max=0s p(95)=0s
http_req_waiting...........: min=479.93ms med=1.03s avg=1.05s max=1.62s p(95)=1.29s
http_reqs..................: 40 4.629349/s
iteration_duration.........: min=97.55µs med=1.05s avg=1.01s max=1.64s p(95)=1.28s
iterations.................: 40 4.629349/sResults:
Max: 1.62s
95%: 1.29s
Average: 1.05s
Sylius, 5 customers
data_received..............: 661 kB 648 kB/s
data_sent..................: 5.3 kB 5.2 kB/s
http_req_blocked...........: min=2.04µs med=3.08µs avg=87.85µs max=751.98µs p(95)=667.82µs
http_req_connecting........: min=0s med=0s avg=80.53µs max=709.87µs p(95)=637.78µs
http_req_duration..........: min=50.81ms med=123.76ms avg=119.91ms max=157.64ms p(95)=154.39ms
http_req_receiving.........: min=39.08µs med=82.99µs avg=188µs max=1.02ms p(95)=753.47µs
http_req_sending...........: min=8.99µs med=17.38µs avg=20.46µs max=71.93µs p(95)=43.75µs
http_req_tls_handshaking...: min=0s med=0s avg=0s max=0s p(95)=0s
http_req_waiting...........: min=50.15ms med=123.66ms avg=119.7ms max=157.53ms p(95)=154.31ms
http_reqs..................: 40 39.255346/s
iteration_duration.........: min=138.78µs med=123.22ms avg=114.78ms max=158.3ms p(95)=154.63ms
iterations.................: 40 39.255346/sResults:
Max: 157.53ms
95%: 154.63ms
Average: 114.78ms
Magento 2, 20 users
http_req_blocked...........: min=1.99µs med=3.08µs avg=227.87µs max=1.58ms p(95)=1.08ms
http_req_connecting........: min=0s med=0s avg=215.37µs max=1.51ms p(95)=1.05ms
http_req_duration..........: min=876.29ms med=4.54s avg=4.35s max=8.07s p(95)=6.9s
http_req_receiving.........: min=1.97ms med=26.82ms avg=33.6ms max=144.99ms p(95)=77.43ms
http_req_sending...........: min=8.32µs med=17.02µs avg=42.77µs max=287.63µs p(95)=218.93µs
http_req_tls_handshaking...: min=0s med=0s avg=0s max=0s p(95)=0s
http_req_waiting...........: min=873.48ms med=4.5s avg=4.31s max=8s p(95)=6.87s
http_reqs..................: 80 4.340769/s
iteration_duration.........: min=111.46µs med=4.51s avg=4.24s max=8.07s p(95)=6.85s
iterations.................: 80 4.340769/sResults:
Max: 8s
95%: 6.87s
Average: 4.31s
Sylius, 20 users
data_received..............: 1.3 MB 642 kB/s
data_sent..................: 11 kB 5.2 kB/s
http_req_blocked...........: min=1.7µs med=2.98µs avg=426.01µs max=3.24ms p(95)=2.56ms
http_req_connecting........: min=0s med=0s avg=407.88µs max=3.17ms p(95)=2.51ms
http_req_duration..........: min=54.12ms med=500.59ms avg=459.25ms max=571.1ms p(95)=553.49ms
http_req_receiving.........: min=526.52µs med=581.77µs avg=707.27µs max=3.59ms p(95)=1.39ms
http_req_sending...........: min=9.04µs med=17.25µs avg=67.61µs max=350.61µs p(95)=305.51µs
http_req_tls_handshaking...: min=0s med=0s avg=0s max=0s p(95)=0s
http_req_waiting...........: min=53.33ms med=499.52ms avg=458.48ms max=570.49ms p(95)=552.88ms
http_reqs..................: 80 38.833384/s
iteration_duration.........: min=140.09µs med=500.27ms avg=448.94ms max=571.54ms p(95)=553.59ms
iterations.................: 80 38.833384/sResults:
Max: 571.54ms
95%: 553.59ms
Average: 458.48ms
Magento 2, 40 exited shoppers on Black Friday
data_received..............: 12 MB 526 kB/s
data_sent..................: 12 kB 529 B/s
http_req_blocked...........: min=1.95µs med=3.68µs avg=801.62µs max=4.95ms p(95)=3.92ms
http_req_connecting........: min=0s med=0s avg=784.98µs max=4.89ms p(95)=3.89ms
http_req_duration..........: min=822.24ms med=8.5s avg=8.38s max=15.48s p(95)=14.61s
http_req_receiving.........: min=1.86ms med=36.21ms avg=44.3ms max=198.8ms p(95)=100.5ms
http_req_sending...........: min=8.43µs med=19µs avg=162.29µs max=750.12µs p(95)=664.62µs
http_req_tls_handshaking...: min=0s med=0s avg=0s max=0s p(95)=0s
http_req_waiting...........: min=816.52ms med=8.46s avg=8.33s max=15.45s p(95)=14.51s
http_reqs..................: 100 4.346488/s
iteration_duration.........: min=94.95µs med=8.49s avg=8.22s max=15.48s p(95)=14.6s
iterations.................: 100 4.346488/sResults:
Max: 15.45s
95%: 14.51s
Average: 8.33s
Sylius, 40 shoppers
http_req_blocked...........: min=1.97µs med=3.75µs avg=1.66ms max=8.33ms p(95)=6.78ms
http_req_connecting........: min=0s med=0s avg=1.6ms max=8.28ms p(95)=6.73ms
http_req_duration..........: min=58.02ms med=986.18ms avg=824.05ms max=1.08s p(95)=1.05s
http_req_receiving.........: min=156.17µs med=570.94µs avg=590.48µs max=1.27ms p(95)=756.95µs
http_req_sending...........: min=9.14µs med=19.29µs avg=118.4µs max=547.09µs p(95)=391.87µs
http_req_tls_handshaking...: min=0s med=0s avg=0s max=0s p(95)=0s
http_req_waiting...........: min=57.56ms med=985.58ms avg=823.34ms max=1.08s p(95)=1.05s
http_reqs..................: 100 39.019238/s
iteration_duration.........: min=136.71µs med=986.19ms avg=810ms max=1.08s p(95)=1.05s
iterations.................: 100 39.019238/sResults:
Max: 1.08s
95%: 1.05s
Average: 823.34ms
As we can see using C5.Large(3 GHz Intel Xeon Platinum 8124M) AWS instance, Sylius can handle 40 users per second with the hosting price of 77$* when for Magento 2 to handle the same traffic requires 770$. Per 77$ M2 can only manage 4 simultaneous users. And it’s without any additional modules Ajax and Cron jobs and Admin users. After customization, to handle 40 users per second, you will need a 7700$ cheap server. Trafic 2 requests per second can create different search and spambots. Magento 2 is the most popular system for Spam User registration, carding attack, and vulnerability scanners. eCommerce, in general, popular for different price and marketing parsers. Even without users at all, you can have a high load on your server. Also, competitors like to generate traffic to terrorize your Magento 2.
*price in us-west-1 AWS region (San Jose North California), in the Oregon us-west-2 region the same instance price 62$
To handle 40 requests/sec by Magento 2 according to my formula, we need:
0.270 * 40 = 10.8 physical CPU or 20 vCPU (x2)2 vCPUs cost on AWS is 77$, 20 vCPUs = 770$
The result is absolutely the same as in our test.
To handle 40 requests per second by Sylis, we need:
0.035 * 40 = 1.4 physical CPU or 2.8 vCPU (x2)2 vCPUs cost on AWS = 77$
For sure, real results can vary and not in Magento’s favor. AJAX, Crones, heavy Admin, EAV, Cache Invalidation, and broken 1st and 3-d party extensions slow down Magento dramatically. It is difficult to find a complex web store with TTFB less than 1540ms.
Read why you should use AWS Graviton 2 processor and why it will kill Magento Cloud:
Basically, because for the same price, you are having physical processors, not a virtual one.
Pivot Table and Chart:
+--------------+-------------+--------+
| | Magento 2.4 | Sylius |
+--------------+-------------+--------+
| 5 Customers | 1290 | 154.63 |
+--------------+-------------+--------+
| 20 Customers | 6870 | 553.59 |
+--------------+-------------+--------+
| 40 Customers | 14510 | 1050 |
+--------------+-------------+--------+
The end.
