This article provide details on using AWS to run FDTD parameter and optimization sweeps. This guide will be using Lumerical-AWS Python integration module, to easily configure and launch EC2 instances and run FDTD parameter sweeps and optimization jobs.
In this guide we will be using the term 'compute nodes' or 'nodes' for the EC2 instances that's created and used to run our simulation jobs.
Requisites
- Additional FDTD engine licenses for each instance used to run simulation jobs.
e.g. 4 instances = 4 FDTD_engine licenses - Amazon Web Services account with permission to access the following services:
- Identity and Access Management (IAM)
- Simple Storage Service (S3)
- Virtual Private Cloud (VPC)
- Elastic Compute Cloud (EC2)
- CloudFormation
- Auto Scaling
- Systems Manager Agent (SSM)
- Amazon Web Services Command Line Interface (AWS - CLI)
- The AWS region must be "us-east-1" to be able to access the Lumerical AMIs used in this process.
- Python 2.7 and Python 3.6 with pip and Cryptography packages.
- FDTD version 8.19.1522 or higher
Configuration
Users and permissions
- A new user account can be created or for an existing user account, it should have access to the appropriate services listed above.
- Multi Factor Authentication should be Disabled for this account. To check the user's MFA status, see: AWS Documentation: Checking MFA Status
- Adding or modifying permissions:
- Choose: Attach existing policies directly
- Create a new policy using this example, aws-admin-permissions list
- Assign the policy to the user
- Creating and adding SSM roles:
- Login to AWS management console
- Go to Services > IAM > Roles
- “Create role”
- Select “EC2 - Allows EC2 instances to call AWS services on your behalf.” > "Next"
- Search and select role: “AmazonEC2RoleforSSM” > "Next"
- Role Name: “SSM_Role” > "Create role"
S3
Amazon Simple Storage Service (S3) is designed to make web-scale computing easier for users. S3 has a simple web services interface that you can use to store and retrieve any amount of data, at any time, from anywhere on the web. S3 is used to store simulation and script files to run FDTD simulation jobs on AWS.
Your simulation files has to be stored in S3 when using the Lumerical-AWS integration process.
To create your S3 bucket, access the S3 service from the AWS management console and create your S3 bucket. A unique bucket name must be provided for each bucket.
The full S3 bucket URI is required to manage objects in your bucket. Take note of your full S3 bucket URI. eg. "s3://bucketname/folder_name/"
More information from AWS: Creating an S3 bucket
Key pair
- The EC2 Key Pair is required to be able to connect and manage your EC2 instances.
- The key pair should be created in region, "us-east-1"
- Follow Amazon's guide on Creating a Key Pair for details.
- Save the key pair on a secure location in your local computer.
- This key pair is required to configure AWS-CLI.
Linux systems
- Change the permission on your Key Pair so that only the owner can read this file.
chmod 400 aws-demo-key.pem
Command Line Interface
The AWS Command Line Interface (AWS CLI) is an open source tool that enables you to interact with Amazon's services using commands in your command-line shell. With minimal configuration, you can start using functionality equivalent to that provided by the AWS Management Console from the command prompt or terminal program.
AWS-CLI installation
We recommend using Python 3.6 and install AWS CLI using the Python+pip method as shown from these guide from Amazon Web Services.
Windows
Linux
macOS
Configure AWS CLI
Once the AWS CLI is installed, we will have to configure this with our user credentials so that it can interact with AWS.
- Obtain an AWS User Access Key ID and a Secret Access Key, by accessing the Security Credentials tab for the user created above step, Users and permissions. Both keys that will be needed to configure your AWS CLI.
- Open a command prompt or terminal window,
aws configure - Enter your User Access Key ID, and Secret Access Key when prompted.
- Select default region = "us-east-1"
- And default output format = "json"
AWS Access Key ID [None]: AKL3M2E9R4I2A6LEXAMPLE
AWS Secret Access Key [None]: kajshdiuasdiourewru12232EXAMPLEKEY
Default region name [None]: us-east-1
Default output format [None]: json - Check your AWS CLI configuration,
aws ec2 describe-account-attributes --attribute-names supported-platforms
- This should return your supported AWS services.
- Open a command prompt or terminal window,
See Amazon guide, Configuring AWS CLI for more information.
Check AWS cloud service integration status
- Open any Lumerical simulation tool.
- Navigate to the Help menu and Select "Cloud integration status"
- A separate window with "Green" status icon indicates that you are connected to AWS and will show your AWS account details.