Shared storage for Magento. FSx vs EFS

Amazon FSx makes it easy and cost effective to launch and run popular file systems. With Amazon FSx, you can leverage the rich feature sets and fast performance of widely-used open source and commercially-licensed file systems, while avoiding time-consuming administrative tasks like hardware provisioning, software configuration, patching, and backups. It provides cost-efficient capacity and high levels of reliability, and it integrates with other AWS services so that you can manage and use the file systems in cloud-native ways.

Amazon FSx provides you with two file systems to choose from: Amazon FSx for Windows File Server for enterprise workloads and Amazon FSx for Lustre for high-performance workloads.

Image for post
Image for post
FSX Windows File Server
Image for post
Image for post
FSx Lustre

Amazon Elastic File System (Amazon EFS) provides a simple, scalable, fully managed elastic NFS file system for use with AWS Cloud services and on-premises resources. It is built to scale on demand to petabytes without disrupting applications, growing and shrinking automatically as you add and remove files, eliminating the need to provision and manage capacity to accommodate growth.

We are interested in first on — Amazon FSx for Windows File Server with SMB protocol. Because NFS based Lusture provides high performance but starts from 160$/month (184$ u- west-1) with minimum storage capacity 1.2Tb

Windows File storage starts from 32Gb and 8Mb/s speed for 26.12$/month

EFS for such configuration 8Mb/s provisioned throughput will cost 63$/month. EFS provides 100% durability because of multiple availability zones deployment. But if FXs SMB Fits you needs — storage around 32GB you can save a little bit.

Also if you can have a lot of data and need performance Lustre will be faster than EFS. Similar EFS configuration (1200Gb, 200MB/s speed) will costs 1716$/month Lustre cost 185$. Even 1Gb EFS with provisioned speed 100Mb costs 660.33$. Basic EFS price is $6 per 1Mb/s filesystem speed + $0.3 Gb

The open source Lustre file system is designed for applications that require fast storage — where you want your storage to keep up with your compute. Lustre was built to quickly and cost effectively process the fastest-growing data sets in the world, and it’s the most widely used file system for the 500 fastest computers in the world. It provides sub-millisecond latencies, up to hundreds of gigabytes per second of throughput, and millions of IOPS.

With AWS you have verity of Managed File systems to run Magneto with.

Nice Drawing Differences of AWS File Systems:

Source: https://www.awsgeek.com/

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

To Mount FSx Lustre and EFS just use official AWS Guide:

LUSTRE: https://docs.aws.amazon.com/fsx/latest/LustreGuide/mounting-ec2-instance.html

EFS: https://docs.aws.amazon.com/efs/latest/ug/mounting-fs.html

Mount FSx Windows File Server to Magento:

To use FXs SAMBA you need install Samba client for Ubuntu and mount your Windows shares as NFS

Installing all the pieces for our Samba client. We are going to execute:
sudo apt-get install cifs-utils samba-common

System Settings

Time to edit our configuration file. Open a console and edit the ‘/etc/nsswitch.conf’ file and be sure the contents look like this:

passwd: compat
group: compat
shadow: compat
hosts: files mdns4_minimal [NOTFOUND=return] dns wins mdns4
networks: files
protocols: db files
services: db files
ethers: db files
rpc: db files
netgroup: nis

Mount

The only difference in this configuration versus the default install is the ‘wins’ parameter added in the middle of the ‘hosts:’ line. Be sure that exists, save your file and reboot your system.

First, let’s create a directory to attach our Windows share to:
sudo mkdir -p /mnt/tmpshare
…and now connect our share to that new directory:
sudo mount -t cifs //shareserver/sharename /mnt/tmpshare -o username=user
One of two things will happen now – (1) You will be prompted for a password so that the share will allow you access to mount it or (2) the mount command will simply succeed (if you do not have the share restricted by password). A simple ‘df -h’ command will check to see if the share now shows as locally mounted. At this point, let’s do a ‘sudo umount /mnt/tmpshare’ and get this ready to become a permanent share on our system.

A Permanent Commitment
So we have our mount location but we want that share to come up every time we reboot the desktop, and we do not want to have to type our password during every connection attempt (which would cause the reboot mount to fail), nor do we want to include our password for that share in plain text in the fstab file itself. First thing is to create a file in our home directory called ‘.smbcredentials’, which should look like this:

username=user
password=pwd

Follow that up with:
chmod 700 .smbcredentials
Finally, the ‘/etc/fstab’ file should look like this for our example:

//shareserver/sharename /mnt/tmpshare cifs credentials=/home/user/.smbcredentials,rw,noauto,user 0 0

Once we save that file, we can test that everything works as expected by executing:
sudo mount -a
Which will mount filesystems that are referred to in the ‘/etc/fstab’ configuration file that are not currently mounted. A quick ‘df -h’ will check the mount command succeeded.

Written by

Magento/APP Cloud Architect. Melting metal server infrastructure into cloud solutions.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store