Getting to know the S3 endpoints has been one of the worst parts of S3 hosting. In fact, it’s been one of the worst parts of all of AWS since I started my cloud hosting journey years ago. Understanding the Amazon Web Services S3 Website and REST API endpoints has been endlessly confusing and has caused me hours of frustration. Now I have somewhat of handle on it and would like to relay my distilled learnings.
In case you haven’t heard the term before, and S3 endpoints is basically a URL that use use with your S3 bucket. It can be used internally in Route 53 or CloudFront, or can be used externally, if you don’t want to bother buying a domain name and (this is important!) your S3 bucket is publicly accessible.
S3 endpoints are simple in concept, but not so much in execution (at least for the novice). Amazon states it clearly:
When you configure a bucket for website hosting, the website is available via the region-specific website endpoint. Website endpoints are different from the endpoints where you send REST API requests.
We already know that Amazon S3 can host websites. In case you didn’t know S3 has an application programming interface (API) that you can use to interact with S3, for such tasks as:
APIs are not the topic of today, but you should know that AWS S3 has a robust API that can do lots of powerful things.
Don’t worry, it’s not a whole lot. Just get down the absolute basics and you’ll be in a good place to move forward. Here goes:
A website endpoint will like look this:
bucket-name.s3-website-region.amazonaws.com
In practice this might look like:
http://aws-website-mysitecom-fbuor.s3-website-us-east-1.amazonaws.com
An S3 API endpoint will look something like:
s3.website-region.amazonaws.com
One thing that makes the REST API endpoints extra confusing is that they can be used either with or without the website name in the URL. The following is the outline for the same request done in each style.
Virtual-hosted-style S3 REST endpoint:
Target file: /index.html
Host: MyBucket.s3-us-east-1.amazonaws.com
Path-style S3 REST endpoint:
Target file: /MyBucket/index.html
Host: s3-us-east-1.amazonaws.com
Note that you can choose the bucket by using a target file or the sub-sub-domain of the host URI, it makes no difference to S3.
The main time to use the website endpoint is when you are doing your DNS entry (such as a CNAME) to an S3 hosted site. In this scenario MySite.com, should have a DNS record that points to mysite.com.s3-website-.amazonaws.com. (You could use CloudFront as well, but a story for another day.)
Another good time to use the website endpoint is in the CloudFront > Origin Setting > Origin Domain Name field. By putting the website endpoint here, you can use all the S3 functions that make the service so powerful for static hosting, like routing rules and bucket policies. These are important since S3 can’t do otherwise simple things, like 404 a specific URL or 301 a file or path on its own. Instead you need to use S3-specific tools.
Long story short, if you S3 bucket is configured to be a website, you need to use the website endpoint! Seems obvious when stated in plain English, but it’s not immediately obvious when you have an S3-hosted site that works perfectly under almost all circumstances, but for some reason your routing rules don’t work!
As with most of the other posts on CloudConfusing.com, this is an oversimplification of the process and tools designed to get you from point A to point B so if you have any tips or feedback, please let me know.
Sal December 24th, 2017
Posted In: AWS
Tags: API, Endpoint, Hosting, REST, S3