Installing and optimizing Magento 2 on AWS Linux 2

How To Install Magento to AWS Linux.

Why use Amazon Linux?

AWS Linux has an Extras mechanism in Amazon Linux 2 to enable the consumption of new versions of application software. Extras help alleviate the compromise between the stability of the OS and the freshness of available software. For example, now you can install newer versions of MariaDB on a stable operating system supported for five years. Examples of extras include Ansible 2.4.2, Memcached 1.5, PHP.7+, Nginx 1.12, Postgresql 9.6, MariaDB 10.2, Go 1.9, Redis 4.0, R 3.4, Rust 1.22.1

Avalable AWS linux packages:

0 ansible2 available \
[ =2.4.2 =2.4.6 =2.8 =stable ]
2 httpd_modules available [ =1.0 =stable ]
3 memcached1.5 available \
[ =1.5.1 =1.5.16 =1.5.17 ]
5 postgresql9.6 available [ =9.6.6 =9.6.8 ]
6 postgresql10 available [ =10 ]
8 redis4.0 available [ =4.0.5 =4.0.10 ]
9 R3.4 available [ =3.4.3 =stable ]
10 rust1 available \
[ =1.22.1 =1.26.0 =1.26.1 =1.27.2 =1.31.0 =1.38.0 ]
11 vim available [ =8.0 ]
13 ruby2.4 available \
[ =2.4.2 =2.4.4 =2.4.7 =stable ]
_
14 php7.2 available \
[ =7.2.0 =7.2.4 =7.2.5 =7.2.8 =7.2.11 =7.2.13 =7.2.14
=7.2.16 =7.2.17 =7.2.19 =7.2.21 =7.2.22 =7.2.23
=7.2.24 =7.2.26 =stable ]
_ lamp-mariadb10.2-php7.2 available \
[ =10.2.10_7.2.0 =10.2.10_7.2.4 =10.2.10_7.2.5
=10.2.10_7.2.8 =10.2.10_7.2.11 =10.2.10_7.2.13
=10.2.10_7.2.14 =10.2.10_7.2.16 =10.2.10_7.2.17
=10.2.10_7.2.19 =10.2.10_7.2.22 =10.2.10_7.2.23
=10.2.10_7.2.24 =stable ]
18 libreoffice available \
[ =5.0.6.2_15 =5.3.6.1 =stable ]
19 gimp available [ =2.8.22 ]
20 docker=latest enabled \
[ =17.12.1 =18.03.1 =18.06.1 =18.09.9 =stable ]
21 mate-desktop1.x available [ =1.19.0 =1.20.0 ]
22 GraphicsMagick1.3 available \
[ =1.3.29 =1.3.32 =1.3.34 ]
23 tomcat8.5 available \
[ =8.5.31 =8.5.32 =8.5.38 =8.5.40 =8.5.42 =8.5.50
=stable ]
24 epel available [ =7.11 ]
25 testing available [ =1.0 ]
26 ecs available [ =stable ]
27 corretto8 available \
[ =1.8.0_192 =1.8.0_202 =1.8.0_212 =1.8.0_222 =1.8.0_232
=1.8.0_242 ]
28 firecracker available [ =0.11 =stable ]
29 golang1.11 available \
[ =1.11.3 =1.11.11 =1.11.13 =stable ]
30 squid4 available [ =4 ]
31 php7.3=latest enabled \
[ =7.3.2 =7.3.3 =7.3.4 =7.3.6 =7.3.8 =7.3.9 =7.3.10
=7.3.11 =7.3.13 =stable ]
32 lustre2.10 available \
[ =2.10.5 =2.10.8 =stable ]
33 java-openjdk11 available [ =11 =stable ]
34 lynis available [ =stable ]
35 kernel-ng available [ =stable ]
36 BCC available [ =0.x ]
37 mono available [ =5.x ]
38 nginx1=latest enabled [ =stable ]
39 ruby2.6 available [ =2.6 =stable ]
40 mock available [ =stable ]
41 postgresql11 available [ =11 =stable ]
42 php7.4 available [ =stable ]

Connect to our Instance

sudo ssh -i ~/.ssh/{pem.file} ec2-user@{ IP addres}

Install PHP 7.2, 7.3, 7.4, 7.5, 8 … 12 … using Amazon Linux Extras

## Instaling PHPsudo yum remove php*sudo amazon-linux-extras install -y php7.3sudo yum install -y php php-common php-mysqlnd php-opcache php-xml php-mcrypt php-gd php-soap php-redis php-bcmath php-intl php-mbstring php-json php-iconv php-fpm php-zipphp -vsudo service php-fpm restart

Installing Composer PHP

sudo yum install wget unzipphp -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"sudo php composer-setup.php --install-dir=/usr/local/bin --filename=composercomposer -v

Install Nginx

sudo amazon-linux-extras install -y nginx1nginx -v
sudo service nginx restart

Install MySQL cli tool not the server

sudo yum install mysql

mysql is a simple SQL shell with input line editing capabilities. It supports interactive and noninteractive use. When used interactively, query results are presented in an ASCII-table format. When used noninteractively (for example, as a filter), the result is presented in tab-separated format. The output format can be changed using command options.

Instal Mysql Maria DB for Magento AWS linux 2

Install Magento 2

# Installing Magento 2composer create-project --repository=https://repo.magento.com/ magento/project-community-edition magento2CE
Warning from repo.magento.com: You haven't provided your Magento authentication keys. For instructions, visit
https://devdocs.magento.com/guides/v2.3/install-gde/prereq/connect-auth.html
Authentication required (repo.magento.com):
Username:
Password:
## Installing Git + Magento Sample Data Modulessudo yum install gitgit clone https://github.com/magento/magento2-sample-data.git
cd magento2-sample-data/
git statusgit checkout 2.3
cd ..
cp -r magento2-sample-data/* magento2CE/

Routing Domain DNS to our server IP

Configure Nginx for Magento:

Examples from Ansible AWS Magento Cloud Deploy Script Repo:

https://github.com/Genaker/AWS_Magento2_Ansible/blob/master/templates/nginx/etc/sites-available/magento2.j2

sudo nano /etc/nginx/conf.d/magento2CE.confserver {
server_name {domain past here}; # or just _
listen 80;set $MAGE_ROOT /var/www/htm/magento2CE/;include /var/www/htm/magento2CE/nginx.conf.sample;
}

Check if our Magento root path is /var/www/html/magento2CE/

Replace fascgi_proxy_pass fastcgi_backend with /var/run/php-fpm/www.sock

fastcgi_pass unix:/var/run/php5-fpm/DOMAINNAME.socket;

Or use this Amazing Repo to install Nginx :

wget https://raw.githubusercontent.com/magenx/Magento-nginx-config/master/m2_config_install.shsudo bash m2_config_install.sh — -> CREATING NGINX CONFIGURATION FILES NOW — -> Enter your domain name (without www.): _
— -> Enter your web root path: /var/www/html/magento
— -> Enter your web user usually www-data (nginx for Centos): apache

If you will use sockets;

sudo service nginx restart

Fix issues if you have

Now can open Magento Url and install it using a web interface or

You can also set up it using Magento CLI

php bin/magento setup:install --backend-frontname="admin" --key="admin" --session-save="files" --db-host="localhost" --db-name="magento_2_db" --db-user="root" --db-password="root" --base-url="http://local.magento.com/" --base-url-secure="https://local.magento2.com/" --admin-user="admin" --admin-password="admin123" --admin-email="pramod.kharade@example.com" --admin-firstname="Pramod" --admin-lastname="Kharade"

First, we should create RDS: best size m5.large 140$ (North California location — San Jose):

*2.5 GHz Intel Xeon Platinum 8175

*2 vCPU

*8 GiB memory

*Up to 10 Gigabit networking performance

After creation, RDS check security group port 3306Mysql should be open to your instance. You’ll need to add instance IP address to the appropriate database security group via the AWS management console. Node IP address can be added to an existing group or you can create a new one. Instruction: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_WorkingWithSecurityGroups.html#USER_WorkingWithSecurityGroups.Authorizing

When adding the Magento BI IP address, make sure you add a “/32” to the end of the address to indicate to Amazon that it is an exact IP address. Don’t worry; the AWS interface will make it clear that this is required.

Sorry I already have one. I also have Redis ElacticCahe AWS instance but it is better to use local Redis. Faster and cheaper. For MySQL, my recommendation is to use RDS because you have nice features like backup, instance resizing, performance monitoring.

create database magento2CE;

MySql has an error during installation Magento

Module ‘Magento_GoogleShoppingAds’:
Upgrading data…
[ERROR] PDOException: SQLSTATE[HY000]: General error: 1419 You do not have the SUPER privilege and binary logging is enabled (you *might* want to use the less safe log_bin_trust_function_creators variable) in /var/www/html/magento2CE/vendor/magento/framework/DB/Statement/Pdo/Mysql.php:91

Fixing:

Image for post
Image for post
Image for post
Image for post

Better to do this before Magento installation:

Also, I have enabled Aurora Performance Shema for Performance insight monitoring.

The parameter group name is changed immediately, but the parameter group changes aren’t applied until you reboot the instance without failover. For more information, see Rebooting a DB Instance.

Reboot your instance.

OK testing our instance

Image for post
Image for post

Response time is 100ms cached:

Image for post
Image for post

Uncached: 360

This one, not an optimized configuration. Let’s configure PHP-opcache

Image for post
Image for post
Image for post
Image for post
sudo service php-fpm restart

Testing again:

Image for post
Image for post

Result: 75ms

Not cached page

Image for post
Image for post

Result: 223ms

Not bad 143ms improvement

I saw a lot of issues when Magento PHP-opcache works without optimized values. Default PHP 7 op-cache values don’t work well with Magento monster codebase.

OK, let add Redis:

sudo bin/magento setup:config:set — cache-backend=redis — cache-backend-redis-server=redis2.apgcjn.0001.usw1.cache.amazonaws.com — cache-backend-redis-db=0

sudo bin/magento setup:config:set — page-cache=redis — page-cache-redis-server=redis2.apgcjn.0001.usw1.cache.amazonaws.com — page-cache-redis-db=1

Now always after PHP files changes, we should restart PHP-FPM

Test:

Image for post
Image for post

Result:110ms

Image for post
Image for post

Result:315

Remote Redis slow down Magento almost twice:

Install Redis locally:

sudo amazon-linux-extras install redis4.0

[ec2-user@ip-172–31–7–147 magento2CE]$ redis-cli -h redis2.apgcjn.0001.usw1.cache.amazonaws.com — latency
min: 0, max: 1, avg: 0.27 (317 samples)

Image for post
Image for post
remote redis
Image for post
Image for post
local redis
[ec2-user@ip-172–31–7–147 magento2CE]$ redis-cli -h localhost — latency
min: 0, max: 1, avg: 0.03 (401 samples)

Remote Redis is 10 times slower than local.

EC2 instance to connect to Redis using a TCP/IP connection or a Unix domain connection. The typical latency of a 1 Gbit/s network is about 200 us, while the latency with a Unix domain socket can be as low as 30 us. It depends on your network and system hardware. On top of the communication itself, the system adds some more latency (due to thread scheduling, CPU caches, NUMA placement, etc …). System induced latencies are significantly higher in a virtualized environment than on a physical machine.

Image for post
Image for post

If to store cache in files it will be the same fast if not faster as local Redis with a socket. Redis it is for scaling.

Lets Setup Redis as socket

Go to: sudo nano /etc/redis.conf

Uncomment:

# Unix socket.
#
# Specify the path for the Unix socket that will be used to listen for
# incoming connections. There is no default, so Redis will not listen
# on a unix socket when not specified.
#
unixsocket /tmp/redis.sock
unixsocketperm 775

sudo chmod -R 777 /tmp/redis.sock

sudo service Redis restart

Image for post
Image for post
Redis socket?

The same rule applies to MySQL. Local + Socket communication will be faster.

Magento has 60–100 Redis requests

on remote Redis, you will have latency 0.3 and you will have 30ms delay

on local TCP 127.0.0.1 latency is 0.03 you will have 3ms

with Redis on a socket, latency is 0.01 you will have a 1ms delay per page load.

And when developers checking a cached value in the loop everything and per one page has 600 cache requests performance degradation is significant.

Also, you can improve Resids performance switching to an instance with faster network throughput. But the price will be more x2–5 more. 0.3 it is for T3.small instance. M5 R5 instances have better network performance

If you need swap

https://aws.amazon.com/premiumsupport/knowledge-center/ec2-memory-swap-file/

Magento with Auto Scaling

About Magento AWS Auto Scaling infrastructure you can read in my nex posts or check this GitHub repo:

https://github.com/Genaker/TerraformMagentoCloud

Image for post
Image for post
Magento 2 AWSCloud With Autoscaling

Open Source EC2 AWS Linux Magento 2 installation script:

Written by

Magento/APP Cloud Architect. Melting metal server infrastructure into cloud solutions.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store