Scenario
As a feature service in AWS, Simple Storage Service (S3) stores data as object comprising object data and metadata. S3 organizes objects with bucket, simply put, a bucket is a container where you store objects. A bucket has a unique name in the Region, and an object is identified by an object key. It may be a bit surprising that you can use SQLs to query contents of objects. S3 itself supports S3 Select, and you can submit standard SQL queries through Amazon Athena, an analytical service. You may think S3 is not a simple container. Right, it in effect can store snapshots for EBS and Amazon RDS, can be used as backup storages for AWS Backup, data stores for AWS Kinesis and AWS Glue, and it can also host a static website. S3 comes with a range of storage classes such as S3 Standard, S3 Intelligent-Tiering, S3 Standard-IA, S3 One Zone-IA, S3 Glacier Instant Retrieval, S3 Glacier Flexible Retrieval, S3 Glacier Deep Archive, etc.
By default, the access to S3 goes through the Internet. In the meanwhile, S3 also supports private accesses via a Gateway VPC endpoint or an Interface VPC endpoint in the same Region. The traffic is not leaving AWS network, as illustrated by the diagram below. So, the services in the VPC can work with it in a more secure way.
In this demo, we’ll create an interface VPC endpoint and test the access to a specific S3 bucket through it.
Examples
Create an Interface Endpoint
Sign in the AWS Management Console, choose VPC service, click Endpoints in the left pane. The home page of the Endpoints is presented where you can view the list of currently available endpoints in the VPC.
Click Create endpoint.
In the Name tag field, enter if-endpoint.
You can leave it blank as it is optional.
For Service category, choose AWS
services.
For Services, select S3 Interface.
Under VPC, choose the destination VPC from
the dropdown list where you would like to create the endpoint, for example, the
default VPC in the AWS Region.
For Subnets, we choose a public subnet
and a private for the demo.
For Security groups, choose the one named by sg_public which allows all traffic.
For Policy associated with this endpoint,
choose Full access.
Click Create endpoint.
The Console navigates back to the home page
of the Endpoints. You can find the newly created interface endpoint on the
list. An interface endpoint comes with a Regional DNS name and one or multiple
Zonal DNS names depending on how many Availability Zones it is attached to.
Moving to Subnets tab, you may
notice that AWS has created a network interface for each subnet and assigned it
a private IP address. As designed, the endpoint connects to S3 via the AWS
PrivateLink.
Test Connectivity Using AWS CLI
We are about to carry out a few tasks on the test bucket tempfortest via the interface endpoint that we just created, vpce-05c5c15c114775198.
Log in the public instance and list up the contents
in the bucket tempfortest.
aws s3 ls s3://tempfortest/ --region ap-northeast-1 --endpoint-url https://bucket.vpce-05c5c15c114775198-1zb58c4z.s3.ap-northeast-1.vpce.amazonaws.com
The text in bold is the endpoint’s regional DNS name.
Log into the private instance and view the bucket as well. To connect to the private instance, we log in the public instance first and use it a transit server to log in the private one via SSH.
We upload another file, anotherfile.txt,
to the test bucket.
aws s3 cp --endpoint-url https://bucket.vpce-05c5c15c114775198-1zb58c4z.s3.ap-northeast-1.vpce.amazonaws.com anotherfile.txt s3://tempfortest
We go back to the public instance and check
the bucket again. The uploaded file appears there.
Endpoint Policy
We didn’t touch on more details about the
Endpoint policy here. In nature, it is a JSON format document in the IAM policy
language. You can restrict access to a specific bucket or a specific account. For
example, the Full access we granted to the demo endpoint is stated as below.
{
"Statement": [
{
"Effect": "Allow",
"Principal": "*",
"Action": "*",
"Resource": "*"
}
]
}
Moreover, you can set up a bucket policy on
S3 side to further strength access control to directory level, object level. If
we look more broadly, we can also design user/role-based access control according to the use cases. You can find plenty of examples in AWS help documentation.
No comments:
Post a Comment