Deploy a Flask / MySQL application on AWS - Part 2b: setting up a multiple, load balanced EC2 instance

aws linux flask python nginx

November 10, 2020

Even though AWS documentation is pretty comprehensive I found myself running into a few problems while creating an environment for my flask application.
In this article I'll guide you through all the required steps to create a load balanced EC2 instance that will act as a web server for an Elastic Beanstalk environment.
Prerequisites
  • An AWS account within the 12 months trial period (not mandatory)
  • A local Python 3 environment (preferably a virtual environment but that's up to you)
  • pip package manager
  • A running local Flask application ready to be deployed
  • Familiarities with the command line
  • macOS or Linux if you want to use the command line interface like I did (I guess this should also work for Windows but the commands are probably slightly different).
Creating a load balanced EC2 instance
If you come from the preceding article you should have a running local flask application and an Elastic Beanstalk environment ready to be deployed. Let's now create a t2.micro EC2 instance running Amazon Linux 2 and Python 3.7. The following command will create by default a load balancer associated with the instance:
(env) $ eb create your-instance -i t2.micro --platform "64bit Amazon Linux 2 v3.1.0 running Python 3.7"
Once the instance has been created eb utility will zip our source folder and upload it on the instance. At the same time it will try to install all the dependencies listed in "requirements.txt". Since MySQLdb (our Python mysql client library) requires mysql to be installed first eb should return an error. By looking at the logs of the newly created instance using the following command:
(env) $ eb logs
notice that pip returned the following error:
Collecting mysqlclient ... raise EnvironmentError("%s not found" % (mysql_config.path,)) OSError: mysql_config not found
MySQLdb requires mysql librairies to be installed first so that's what we'll do next.
Installing mysql libraries on the EC2 instance
SSH into your EC2 instance with the key pair you created during environment creation. Using the command:
(env) $ eb ssh
from your application source folder will automatically ssh into your instance without having to manually enter your key pair location.
Note: SSH key pairs are stored in ~/.ssh by default.
You should now be logged into your instance:
INFO: Attempting to open port 22. INFO: SSH port 22 open. INFO: Running ssh -i /Users/guimauve/.ssh/someKey.pem ec2-user@instance-ip _____ _ _ _ ____ _ _ _ | ____| | __ ___| |_(_) ___| __ ) ___ __ _ _ __ ___| |_ __ _| | | __ | _| | |/ _ \/ __| __| |/ __| _ \ / _ \/ _\ | '_ \/ __| __/ _\ | | |/ / | |___| | (_| \__ \ |_| | (__| |_) | __/ (_| | | | \__ \ || (_| | | < |_____|_|\__,_|___/\__|_|\___|____/ \___|\__,_|_| |_|___/\__\__,_|_|_|\_\ Amazon Linux 2 AMI This EC2 instance is managed by AWS Elastic Beanstalk. Changes made via SSH WILL BE LOST if the instance is replaced by auto-scaling. For more information on customizing your Elastic Beanstalk environment, see our documentation here: http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/customize-containers-ec2.html [ec2-user@instance-ip ~]$
mysql librairies require gcc to be installed:
[ec2-user@instance-ip ~]$ sudo yum install gcc [ec2-user@instance-ip ~]$ sudo yum install -y mysql-devel
If the installation was successful we can log out from the instance and try to deploy our application again.
(env) $ eb deploy
If everything went fine eb deploy succeeded and you should get the following output:
Creating application version archive "app-7daa-200908_121806". Uploading: [##################################################] 100% Done... 2020-09-08 10:18:13 INFO Environment update is starting. 2020-09-08 10:18:16 INFO Deploying new version to instance(s). 2020-09-08 10:18:22 INFO Instance deployment successfully generated a 'Procfile'. 2020-09-08 10:18:30 INFO Instance deployment completed successfully. 2020-09-08 10:18:34 INFO New application version was deployed to running EC2 instances. 2020-09-08 10:18:34 INFO Environment update completed successfully.
Run the following command to get the private URL to your environment:
(env) $ eb status | grep CNAME CNAME: your-env-url.elasticbeanstalk.com
Take some time to browse your application from there and make sure all pages are working properly.
(Optional) Disabling EC2 instance auto scaling
To avoid automatically creating new instances and risking going above the free-tier zone we should set the maximum capacity of the default auto scaling group to 1.
From the EC2 panel go to
  • "Auto Scaling Groups" and select the only group you should have.
  • Click on "Edit" and set the "Maximum capcity" to "1".
  • Save the changes by clicking on "Update".
Obtaining an SSL certificate from Amazon Certificate Manager
If you don't already have a SSL certificate for your domain you can get one for free from AWS certificate manager. This will allow your end-users to securely connect to your website via HTTPS and is pretty much mandatory nowadays.
  • Click on "Request a certificate"
  • "Request a public certificate"
  • Enter your domain names: for instance *.your-domain.com and your-domain.com so that users can access your website under its different names.
  • Select the appropriate validation method. If you bought your domain name from AWS or OVH you can chose "DNS validation".
  • Add a tag if needed then proceed to the next page
  • For AWS domain names you have the option to directly create a record in Route 53.
  • For other registrars simply add a new CNAME entry to your DNS zone with the given name and value.
Allowing HTTPS traffic to your application

Adding HTTPS inbound rules to your Elastic Beanstalk environment security groups

Go to the EC2 panel and select "Security Groups"
From the security group list select the elastic load balancer security group and write down its ID. It should have a name similar to 'awseb...-AWSEBLoadBalancerSecurityGroup-...':
Click on "Actions", "Edit inbound rules" and add an HTTPS rule with the source being CIDR block "0.0.0.0/0". Save it and go back to the security group list.
Now select the one associated to your Elastic Beanstalk environment:
Click on "Actions", "Edit inboud rules" and add an HTTPS rule with the source being the ELB security group of which you should have the ID written down somewhere. Save the modifications.

Adding HTTPS listeners to the load balancers

For the users to access our application we still need to make both the application load balancer AND the elastic load balancer listen on the HTTPS port.
From the EC2 console go to "Load Balancers" and select the corresponding load balancer (you should only have one at this point).
  • Click on "Listeners"
  • "Edit"
  • "Add" and select HTTPS protocol
  • Click on "Change" under the SSL certificate column
  • Select "Choose a certificate from ACM" and select the certificate you created earlier
Click on "Save".
Now from the Elastic beanstalk console:
Select your environment and chose "Configuration"
  • "Edit" the Load Balancer settings
  • "Add listener"
  • "Listener port" : 443
  • "Listener protocol" : HTTPS
  • "Instance port" : 80
  • "Instance protocol" : HTTP
  • "SSL certificate": the same one you created earlier
Forcing HTTPS redirection from the web server
Note: HTTPS redirection can now be made via the new generation of Elastic Load Balancer. At the time of writing I was doing fine with the "Classic" load balancer and decided to stick with it.
Most browsers nowadays will automatically redirect to https for security reasons, but it doesn't mean that accessing a website using classic http is not possible. In fact it is. To make sure that the users are always connected to your website through https we need to force https redirection.
To redirect http requests to https we need to modify nginx configuration by replacing the original file located in /etc/nginx/nginx.conf by a custom one. I had a hard time finding a way to do this but finally managed to replace nginx.conf after deployment. The following paragraph is a copy and paste of my answer on stackoverflow about this issue.
Using an Amazon Linux 2 envrionment:
Source: AWS documentation
Scripts are now executed from subfolders in .platform that you have to place at the root of your project, like so:
~/flask_app/ |-- application.py |-- static | |-- styles.css |-- templates | |-- index.html |-- readme.md |-- .ebextensions/ | |-- 01_write_custom_ng_conf.config |-- .platform/ |-- hooks |-- postdeploy |-- 01_replace_ng_conf.sh # Executed post deployment
Create a .config file at the root of .ebextensions.
01_write_custom_ng_conf.config
To automatically redirect any http request to https we simply need to add the following lines to nginx.conf server block:
34 if ($http_x_forwarded_proto = 'http') { 35 return 301 https://$host$request_uri; 36 }
Create your modified nginx.conf file and place it in /tmp.
1 files: 2 "/tmp/custom_nginx.conf": 3 mode: "000755" 4 owner: root 5 group: root 6 content: | 7 user nginx; 8 error_log /var/log/nginx/error.log warn; 9 pid /var/run/nginx.pid; 10 worker_processes auto; 11 worker_rlimit_nofile 32136; 12 13 events { 14 worker_connections 1024; 15 } 16 17 http { 18 include /etc/nginx/mime.types; 19 default_type application/octet-stream; 20 21 log_format main '$remote_addr - $remote_user [$time_local] "$request" ' 22 '$status $body_bytes_sent "$http_referer" ' 23 '"$http_user_agent" "$http_x_forwarded_for"'; 24 25 include conf.d/*.conf; 26 27 map $http_upgrade $connection_upgrade { 28 default "upgrade"; 29 } 30 31 server { 32 listen 80; 33 server_name _; 34 # The server will redirect any http request made on port 80 to its https counterpart 35 if ($http_x_forwarded_proto = 'http') { 36 return 301 https://$host$request_uri; 37 } 38 39 access_log /var/log/nginx/access.log main; 40 41 client_header_timeout 60; 42 client_body_timeout 60; 43 keepalive_timeout 60; 44 gzip off; 45 gzip_comp_level 4; 46 gzip_min_length 1000; 47 gzip_types text/plain text/css application/json application/javascript application/x-javascript text/xml application/xml application/xml+rss text/javascript; 48 49 # Include the Elastic Beanstalk generated locations 50 include conf.d/elasticbeanstalk/*.conf; 51 } 52 }
01_replace_ng_conf.sh
Create a small script in .platform/hooks/postdeploy and change permissions to 755.
(env) $ touch .platform/hooks/postdeploy/01_replace_ng_conf.sh (env) $ chmod 755 .platform/hooks/postdeploy/01_replace_ng_conf.sh
Note: Both creation and permission setting can be done at the same time using GNU core utility install.
Add the following lines:
1 #!/usr/bin/bash 2 3 # Replace the original nginx.conf by our custom one 4 sudo mv /tmp/custom_nginx.conf /etc/nginx/nginx.conf 5 6 # Restart nginx to apply modifications 7 sudo service nginx restart
This method should work for any environment running on a Amazon Linux 2 instance.
Make sure that your .config files indentation is perfect as errors are not always detected by the parser and the code won't be executed."
Our application is now ready to be accessed by users. We only have to route our domain name to the application.
Routing your domain name to your application
Routing a domain name bought from Route 53 to a Elastic Beanstalk application is pretty straightforward.
Go to Route 53 panel and select the domain name you want to use
In "Hosted Zones":
  • Click on "Create record"
  • "Simple routing" then "Next"
  • "Define simple record"
  • "Define simple record"
  • Add "www" to your domain name
  • "Value/Route traffic to" -> "Allias to Application and Classic Load Balancer"
  • Select your region
  • Select your load balancer
  • Leave "Record type" on "A - Routes traffic to an IPv4 address and some AWS ressources"
  • Set "Evaluate target health" to "No"
  • Then click on "Define simple record"
Wait a few minutes and try to access to your website via your domain name. If you religiously followed this tutorial (and assuming I didn't forget anything) it should work. You'll also notice that the connection to your web application is secured meaning that we properly configured https traffic.
Special procedure with OVH (and probably other registrars)
OVH DNS zone doesn't allow to add an A record under the format of a DNS name but only under a proper IPv4 address format. To overcome this restriction we simply need to resolve the dns name of the load balancer and get its IPv4 adresses.
Go to the EC2 console and select "Load Balancers". Chose the one used by your application and copy its DNS name:
Using nslookup:
$ nslookup your-elb-address.amazonaws.com
we get the following output:
Server: ################################# Address: #################################:######## Non-authoritative answer: Name: your-elb-address.amazonaws.com Address: X.X.X.X Name: your-elb-address.amazonaws.com Address: X.X.X.X
where "Address" is one of the IPv4 address behind the ELB dns name.
Simply add each address as an A record to your dns zone as shown below:
Wait a few minutes to let the information propagate and you should be able to acess your website via your domain name.