AWS Storage
Amazon
Simple Storage Service (S3)
Definition
- S3 is an Object Store
- Secure, durable, and highly-scalable cloud storage
- Optimized for reads and intentionally light weight
- Accessible from anywhere on the web
- One of the AWS Foundational Services
Features
- Storage Classes
- Lifecycle Policies
- Rich set of Access Controls
- Replication is Automatic:
- Scalability - Automatically partitions buckets
Amazon
Glacier
Definition
- S3 optimized for long-term backup and archival.
- 3-5 hour retrieval time
-
Dual Product Offering
- An S3 Storage Class
- Archival Storage Service
Amazon S3
Background
Types of Storage
-
Block - Storage Device Level
-
Organizes data into numbered, fixed size blocks
-
Organizes data into numbered, fixed size blocks
-
File - Server and Operating System Level
-
Organizes data into named hierarchy of folders and files
-
Organizes data into named hierarchy of folders and files
-
Object
- Independent of Servers, Operating Systems
- Accessed over a network
- The native interface for S3 is a ReST API.
S3 Basics
- S3 Object Characteristics
- Each S3 object contains BOTH data and metadata
- Each S3 object is uniquely identifed by:
<bucket><key>[<versionId>] - Max Size = 5 terabytes
- Operations (GET, PUT) are on whole objects
-
Data
- S3 treats all objects as a stream of bytes.
- S3 is completely format agnostic
-
Metadata
- A set of name/value pairs
- System metadata with object characteristics.
- Optional User metadata
Bucket Characteristics
- A bucket is a container (web folder) for objects (files) stored in S3.
- Each account may define 100 buckets
- Buckets are created and stored within specific AWS regions
-
Buckets are the top-level, global namespace in S3
- Must be globally unique across all AWS
- Naming Conventions
- Can hold an unlimited number of objects
-
A simple flat folder with no hierarchy
Note: Console Folder Hierarchy
Accessing S3 Objects
-
Operations
- Intentionally simple
- Based on a ReST implementation of CRUD operations
- Bucket Operations
- Create, Delete, List
- Object Operations
- Write, Read, Delete,
- Note: the absence of an Update. Why?
- Bucket Operations
Accessing S3 Objects
- Direct Interface
- Representationale State Transfer (ReST)
- Create, Read, Update, Delete (CRUD) operations mapped to HTTP methods
Ref: POST Object- Create -> HTTP PUT (or POST to accomadate use of HTML forms)
- Read -> HTTP GET
- Update -> HTTP POST ( or PUT)
- Delete -> HTTP DELETE
Accessing S3 Objects
- High Level Interface
- AWS Software Developement Kit (SDK)
- Wrapper Libraries
- AWS Command Line Interface (CLI)
- AWS Management Console
Accessing S3 Objects
-
Durability
-
Will my data still be there ?
-
S3 is 99.999999999% durable
-
-
Availability
- Can I access my data ?
- S3 is 99.99% available
-
Reduced Redundancy Storage (RRS)
- Reduced Cost Alternative
- RRS is 99.99% durable
-
Best Practice
- Protect against user mistakes
- Versioning
- Cross-Region Replication
- MFA Delete
- Protect against user mistakes
AWS Storage
By Daniel R Creager
AWS Storage
- 531