AWS Solutions Architect Associate Exam Cheatsheets [SAA-C03]

This guide was prepared by me during my studies for the AWS Solutions Architect exam. Below is a comprehensive collection of key topics and a cheat sheet for quick reference.

Analytics

Amazon Athena

Type: Serverless Interactive Query Service Description: Amazon Athena is a serverless query service that allows you to analyze data stored in Amazon S3 using standard SQL. With Athena, there is no need to manage infrastructure or perform complex ETL processes. It is ideal for running ad-hoc queries and gaining insights from large datasets quickly and cost-effectively.

Key Features

1. Serverless Architecture:
   • No infrastructure to manage; simply define the data and execute SQL queries.
2. SQL Query Support:
   • Use standard ANSI SQL for querying structured, semi-structured, and unstructured data.
3. Supports Various Data Formats:
   • Compatible with CSV, JSON, Parquet, ORC, Avro, and other popular data formats.
4. Data Partitioning:
   • Query specific partitions of data to improve query performance and reduce costs.
5. Schema on Read:
   • Define schemas when querying the data rather than during data ingestion, making it flexible for analyzing raw data.
6. Integration with AWS Glue:
   • Uses AWS Glue Data Catalog for metadata management and schema definitions.
7. Scalable Query Processing:
   • Automatically scales to handle complex and large queries.
8. Pay-Per-Query Pricing:
   • Billed based on the amount of data scanned by queries, encouraging optimization.

Subtypes and Components

1. Query Execution Engine:
   • Powered by Presto, an open-source distributed SQL query engine optimized for large-scale data analysis.
2. Data Catalog Integration:
   • Leverages AWS Glue Data Catalog to store metadata about tables and databases.
3. Federated Query Support:
   • Query data across various sources, including RDS, Redshift, or on-premises databases, without moving data.
4. Partitioning and Compression:
   • Supports partitioned and compressed datasets for enhanced performance.

Use Cases

1. Ad-Hoc Data Analysis:
   • Quickly analyze data in Amazon S3 without complex data pipelines.
2. Log and Event Analysis:
   • Query server logs, clickstream data, or application logs stored in S3.
3. Data Exploration:
   • Explore raw or semi-structured data for insights before loading into a data warehouse.
4. Business Intelligence (BI):
   • Integrate with tools like Amazon QuickSight for interactive reporting and dashboards.
5. ETL Replacement:
   • Simplify ETL workflows by directly querying data in its native format.
6. Machine Learning Data Preparation:
   • Prepare datasets for machine learning workflows by filtering and transforming data using SQL. Integration with AWS Services Integrated With:

• Amazon S3: Primary data storage for querying with Athena.
• AWS Glue: Provides the Data Catalog to manage metadata and table definitions.
• Amazon QuickSight: Enables visualization of Athena query results in interactive dashboards.
• Amazon CloudWatch Logs: Analyze logs stored in S3.
• AWS Lake Formation: Simplifies managing and securing data lakes queried by Athena. Not Directly Integrated With:
• RDS or Aurora: While Athena cannot natively query these databases, you can export their data to S3 for analysis.
• DynamoDB: Data must be exported to S3 or queried via federated queries using AWS Glue connectors.

Governance and Security

1. Access Control:
   • Manages access through IAM policies, ensuring users have the appropriate permissions for querying and managing resources.
2. Encryption:
   • Supports data encryption at rest (using S3 encryption) and in transit (using SSL/TLS).
   • Query results can also be encrypted and stored in S3.
3. Auditing and Logging:
   • Integrated with AWS CloudTrail for logging query and access events.
4. Data Privacy:
   • Access control integrated with AWS Lake Formation to enforce granular security policies. Benefits
5. No Infrastructure Management:
   • Focus on querying data without worrying about provisioning or maintaining servers.
6. Cost Efficiency:
   • Pay only for the data scanned, and optimize costs by compressing, partitioning, and filtering data.
7. Fast Query Execution:
   • Presto-powered execution ensures low-latency querying for large datasets.
8. Flexibility:
   • Analyze various data formats, including structured and semi-structured, without transformation.
9. Scalable:
   • Automatically scales resources to match query demands, enabling high availability.

AWS Data Exchange

• Type: Data Sharing Service
• Description: Facilitates secure and efficient exchange of third-party data sets, enabling customers to subscribe to and use data products in the cloud.
• Use Cases:
  • Accessing financial market data
  • Integrating healthcare datasets
  • Utilizing demographic and consumer data for analytics Additional Features:
• Subscriptions: Automated updates to subscribed datasets.
• Data API Access: Seamlessly retrieve data directly into analytics tools. Governance and Security:
• Control data access with IAM roles.
• Ensure compliance by subscribing only to verified providers. Examples:
• Leverage third-party weather data for logistics optimization.

AWS Data Pipeline

• Type: Data Workflow Orchestration
• Description: Automates the movement and transformation of data between different AWS services and on-premises data sources at specified intervals.
• Use Cases:
• Periodic ETL processes
• Data replication across services
• Data processing workflows

Amazon EMR (Elastic MapReduce)

• Type: Managed Big Data Framework
• Description: Provides a managed Hadoop framework to process vast amounts of data across resizable clusters of Amazon EC2 instances.
• Subtypes:
• Hadoop: Batch processing and distributed storage
• Spark: Real-time data analytics and machine learning
• Presto: Interactive SQL queries on large datasets
• Hive: Data warehousing and SQL-like querying
• HBase: NoSQL database for big data applications
• Use Cases:
• Large-scale data processing
• Log analysis
• Machine learning workloads Additional Features:
• Auto Scaling Clusters: Dynamically add or remove nodes based on workload.
• Custom AMIs: Use pre-configured images for specific analytics needs. Governance and Security:
• Encrypt data in transit with SSL and at rest using S3 encryption options.
• Monitor job execution via CloudWatch logs. Examples:
• Process petabytes of clickstream data for behavioral analysis.

AWS Glue

Type: Serverless ETL (Extract, Transform, Load) Service Description: AWS Glue is a fully managed ETL service that simplifies the process of extracting, transforming, and loading data for analytics. It automatically discovers and catalogs data, generates ETL code, and provides tools to clean and transform datasets for use in data lakes, warehouses, and machine learning workflows.

Key Features

1. Serverless Architecture:

• No infrastructure management; AWS Glue scales automatically to meet workload demands.

2. Data Catalog:

• Centralized metadata repository to store table definitions, schemas, and job runtime metrics.
• Compatible with AWS Lake Formation for enhanced governance and security.

3. Built-In ETL Engine:

• Generates Python or Scala ETL scripts, which can be customized using an integrated development environment (IDE).

4. Schema Discovery:

• Automatically detects and catalogs schema and format of data in various sources (e.g., S3, RDS, DynamoDB).

5. Developer-Friendly:

• Supports PySpark for building ETL workflows and integrates with Jupyter Notebooks for interactive development.

6. Job Scheduling:

• Allows scheduling and orchestrating ETL workflows, with dependency tracking and retries for failed jobs.

7. Streaming ETL:

• Supports real-time data transformation for streaming data sources like Amazon Kinesis and Kafka.

8. Integration with AWS Ecosystem:

• Works seamlessly with services like Amazon S3, Redshift, Athena, and more for end-to-end data processing.

9. Data Preparation:

• AWS Glue DataBrew provides a visual interface for preparing and transforming data without writing code.

10. Data Quality and Lineage:

• Track and audit the lineage of data transformations for compliance and debugging.

Subtypes and Components

1. AWS Glue Data Catalog:

• Central repository for metadata that integrates with Amazon Athena, Redshift, and Lake Formation.

2. ETL Jobs:

• Fully managed Spark-based jobs for transforming and moving data between sources and destinations.

3. Glue Crawlers:

• Automatically crawl data sources to detect schema and store metadata in the Glue Data Catalog.

4. AWS Glue DataBrew:

• Visual data preparation tool for cleaning and enriching data without code.

5. AWS Glue Studio:

• Low-code interface to visually design and manage ETL workflows.

6. Triggers and Workflows:

• Orchestrate complex ETL pipelines with event-based triggers and multi-step workflows.

7. Glue Connectors:

• Pre-built connectors to integrate with on-premises and cloud-based data sources, including SaaS applications.

Use Cases

1. Data Lakes:

• Create and manage data lakes by ingesting, cataloging, and transforming raw data for analysis.

2. Data Warehousing:

• Load transformed data into Amazon Redshift for BI and reporting use cases.

3. Real-Time Analytics:

• Stream and transform data from sources like Kinesis or Kafka for immediate insights.

4. Machine Learning Preparation:

• Prepare and clean datasets for machine learning models in Amazon SageMaker or other platforms.

5. Data Discovery:

• Automatically identify and catalog data from diverse sources for easy querying and exploration.

6. Regulatory Compliance:

• Ensure data quality, lineage, and transformations for regulatory reporting and audits.

7. Application Data Integration:

• Transform and integrate application data from databases, APIs, and other sources into a unified format.

Integration with AWS Services Integrated With:

• Amazon S3: Primary storage for raw and transformed data.
• Amazon Redshift: Load data into Redshift for data warehousing and analytics.
• Amazon Athena: Query cataloged data directly from the Glue Data Catalog.
• AWS Lake Formation: Enhanced security and governance for data lakes.
• Amazon DynamoDB: Transform and process DynamoDB tables.
• Amazon RDS and Aurora: Extract and transform data from relational databases.
• Amazon SageMaker: Prepare data for machine learning workflows.
• Amazon Kinesis and Kafka: Stream real-time data for transformation and storage. Not Integrated With:
• Standalone EC2 instances unless used with custom configurations to pull data into Glue-supported formats.
• On-premises systems without using Glue connectors or custom scripts.

Governance and Security

1. IAM Policies:

• Use IAM roles to manage access to Glue resources and associated services like S3, DynamoDB, and Redshift.

2. Encryption:

• Data can be encrypted at rest using AWS KMS and in transit using SSL/TLS.

3. Auditing and Logging:

• Monitor Glue activities through AWS CloudTrail and detailed logs in CloudWatch.

4. Data Governance:

• Integrated with AWS Lake Formation for managing fine-grained access control.

5. Data Lineage:

• Trace the flow of data and transformations for compliance and debugging.

Benefits

1. Ease of Use:

• Automates the tedious aspects of data preparation, freeing up developer time.

2. Scalable:

• Handles large datasets with automatic scaling of resources.

3. Cost-Effective:

• Pay-as-you-go pricing ensures costs scale with usage, making it affordable for varying workloads.

4. Flexibility:

• Supports diverse data formats and integrates with a wide range of AWS services.

5. Developer Productivity:

• Provides tools like Glue Studio and DataBrew to simplify ETL and data preparation.

6. Real-Time Processing:

• Supports streaming data transformations for immediate use cases.

Amazon Kinesis

• Type: Real-Time Data Streaming
• Description: Enables real-time processing of streaming data at scale.
• Subtypes:
• Kinesis Data Streams: Real-time data ingestion
• Kinesis Data Firehose: Data delivery to destinations like S3, Redshift, and Elasticsearch
• Kinesis Data Analytics: Real-time data processing using SQL
• Use Cases:
• Real-time analytics
• Log and event data collection
• Streaming data pipelines

Additional Features:

• Data Retention: Keep data for up to 7 days for replay.
• Integration: Works seamlessly with Lambda, S3, and Redshift. Governance and Security:
• Control access to streams using IAM roles and resource policies.
• Encrypt data streams with KMS-managed keys. Examples:
• Real-time monitoring of social media trends.

AWS Lake Formation

• Type: Data Lake Management
• Description: Simplifies the process of building, securing, and managing data lakes, allowing for centralized storage and analysis of diverse data.
• Use Cases:
• Centralized data repository
• Data governance and security
• Simplified data ingestion and cataloging

Amazon Managed Streaming for Apache Kafka (Amazon MSK)

• Type: Managed Kafka Service
• Description: Fully managed service that makes it easy to build and run applications that use Apache Kafka for streaming data.
• Use Cases:
• Real-time data streaming
• Event sourcing
• Log aggregation

Amazon OpenSearch Service (formerly Elasticsearch Service)

• Type: Managed Search and Analytics
• Description: Provides a managed service for deploying, operating, and scaling OpenSearch (and legacy Elasticsearch) clusters in the AWS Cloud.
• Use Cases:
• Log and event data analysis
• Full-text search
• Real-time application monitoring

Amazon QuickSight

• Type: Business Intelligence Tool
• Description: Cloud-powered business analytics service that makes it easy to deliver insights to everyone in your organization.
• Use Cases:
• Interactive dashboards
• Ad-hoc analysis
• Embedded analytics

Amazon Redshift

• Type: Data Warehousing
• Description: Fully managed, petabyte-scale data warehouse service in the cloud.
• Use Cases:
• Complex query processing
• Business intelligence
• Data warehousing Additional Features:
• Materialized Views: Speed up queries by precomputing results.
• Spectrum: Query S3 data without loading it into Redshift. Governance and Security:
• Use role-based access controls (RBAC) for granular permissions.
• Audit access with CloudTrail integration. Examples:
• Perform complex OLAP queries for e-commerce reporting.

Application Integration

Amazon AppFlow

• Type: SaaS Integration
• Description: Enables secure data transfer between AWS services and SaaS applications like Salesforce, ServiceNow, and others without writing custom code.
• Use Cases:
• Data synchronization between SaaS and AWS
• Automated data workflows
• Integrating third-party applications with AWS services

Additional Features:

• Data Filtering: Transfer only necessary records using filters.
• Custom Mappings: Map source fields to destination fields. Governance and Security:
• Encrypt data in transit and at rest.
• Use private links to transfer data securely without internet exposure. Examples:
• Sync Salesforce contacts to S3 for advanced analytics.

AWS AppSync

• Type: Managed GraphQL Service
• Description: Simplifies application development by providing a flexible API to securely access, manipulate, and combine data from multiple sources.
• Use Cases:
• Real-time applications
• Offline data access
• Mobile and web application backends Additional Features:
• Express Workflows: Ideal for high-frequency, short-duration processes.
• Visual Workflow Designer: Drag-and-drop UI to simplify workflow creation. Governance and Security:
• Secure workflows with IAM role-based access.
• Monitor executions with CloudWatch Logs. Examples:
• Automate multi-step approval processes.

Amazon EventBridge

• Type: Serverless Event Bus
• Description: Amazon EventBridge is a serverless event bus service that allows you to connect your applications with event data from various AWS services, third-party SaaS applications, or custom sources. EventBridge enables event-driven architectures by facilitating real-time data transfer and triggering actions across systems without the need for manual integration or polling mechanisms. Key Features

1. Event Bus:

• Centralized event routing system for handling events from AWS services, third-party SaaS applications, and custom event sources.

2. Event Filtering:

• Apply rules to filter and process events based on defined patterns, enabling targeted actions.

3. Event Archiving and Replay:

• Automatically archives events for later reprocessing or troubleshooting.

4. Schema Registry:

• Central repository for managing and discovering event schemas, supporting schema versioning and validation.

5. Cross-Account Event Sharing:

• Share events across multiple AWS accounts securely using resource policies.

6. Third-Party Integration:

• Native integration with SaaS applications like Zendesk, Shopify, and Datadog.

7. Custom Event Sources:

• Publish custom application events directly to EventBridge for further processing.

8. Scalability:

• Automatically scales to handle millions of events per second.

9. Low Latency:

• Processes and delivers events in near real-time, ensuring timely execution of workflows.

Components

• Event Bus:

• Default Bus: Captures events from AWS services by default.

• Custom Bus: Allows for segregating event sources for different applications or services.

• Partner Bus: Integrates with supported SaaS applications.

• Rules:

• Define how events are processed and routed to one or more targets.

• Targets:

• Services or resources that receive the events, such as Lambda, SQS, SNS, Step Functions, or Kinesis.

• Schema Registry:

• Stores event schemas and enables event schema discovery for producers and consumers.

• Event Archive:

• Retains events for replay or debugging with configurable retention policies.

• Use Cases:

• Application integration

• Real-time data processing

• Event-driven workflows Integration with AWS Services Integrated With:

• AWS Lambda: Trigger serverless functions for event-driven processing.

• Amazon SQS: Queue events for asynchronous processing.

• Amazon SNS: Distribute notifications or trigger downstream actions.

• AWS Step Functions: Orchestrate workflows triggered by events.

• Amazon Kinesis: Process real-time data streams for analytics.

• Amazon ECS/EKS: Launch tasks or pods in response to events.

• AWS CloudWatch Logs: Generate events based on log patterns for monitoring and automation.

• Amazon API Gateway: Route external events to EventBridge for processing.

• AWS Glue: Trigger ETL workflows upon data updates. Not Directly Integrated With:

• Standalone databases like RDS or DynamoDB unless events are routed through another trigger like Lambda or DynamoDB Streams.

Amazon MQ

• Type: Managed Message Broker Service
• Description: Provides a managed message broker service for Apache ActiveMQ and RabbitMQ, facilitating the migration of messaging workloads to AWS.
• Use Cases:
• Messaging between distributed systems
• Application decoupling
• Legacy application integration

Amazon Simple Notification Service (SNS)

• Type: Pub/Sub Messaging Service
• Description: Fully managed messaging service for both application-to-application (A2A) and application-to-person (A2P) communication.
• Use Cases:
• Sending notifications
• Fan-out message delivery
• Event-driven computing

Amazon Simple Queue Service (SQS)

Type: Managed Message Queue Service Description: Amazon SQS is a fully managed message queuing service that enables decoupling and scaling of distributed systems, microservices, and serverless applications. It allows asynchronous communication between application components by securely transmitting messages via queues, ensuring reliable and scalable operations.

Key Features

1. Message Queues:

• Store messages in a queue for processing by consumers, enabling asynchronous communication.

2. Two Queue Types:

• Standard Queue: Provides at-least-once message delivery and best-effort ordering.
• FIFO Queue (First-In-First-Out): Ensures exactly-once processing and message order consistency.

3. Scalability:

• Scales automatically to handle high-throughput workloads.

4. Serverless Architecture:

• Fully managed with no infrastructure to provision or manage.

5. Message Retention:

• Messages can be retained for up to 14 days.

6. Dead-Letter Queues (DLQs):

• Store undeliverable messages for troubleshooting.

7. Message Visibility Timeout:

• Prevents messages from being processed multiple times by hiding them from other consumers during processing.

8. Delay Queues:

• Delay delivery of messages by a specified duration.

9. Long Polling:

• Reduce empty responses by waiting for messages to be available in the queue.

10. Encryption:

• Supports encryption at rest using AWS Key Management Service (AWS KMS).

Components

1. Producers:

• Applications or services that send messages to SQS queues.

2. Queue:

• Stores messages temporarily until consumed.

3. Consumers:

• Applications or services that retrieve and process messages from the queue.

4. Dead-Letter Queue (DLQ):

• Configurable queue for storing failed messages.

5. Attributes:

• Metadata associated with each queue or message, such as retention period, visibility timeout, or delay duration.

Queue Types

1. Standard Queue:

• High throughput and low latency.
• At-least-once delivery with potential duplication of messages.
• Best-effort ordering (not guaranteed).

2. FIFO Queue:

• Exactly-once delivery.
• Preserves the order of message processing.
• Limited throughput compared to standard queues (300 transactions per second with batching, or 3000 with high-throughput mode).

Use Cases

1. Decoupling Microservices:

• Ensure independent scalability and failure isolation between application components.

2. Load Leveling:

• Buffer requests during high traffic to ensure backend systems process them at a steady pace.

3. Serverless Applications:

• Trigger AWS Lambda functions to process messages in SQS.

4. Job Queues:

• Manage long-running tasks or background jobs asynchronously.

5. Message Offloading:

• Offload temporary or bulk messages for later processing.

6. Batch Processing:

• Combine multiple messages for efficient consumption and processing.

7. Event-Driven Architectures:

• Trigger downstream workflows based on event-driven messages.

8. Dead-Letter Handling:

• Troubleshoot and manage failed message deliveries using DLQs.

Integration with AWS Services Integrated With:

• AWS Lambda: Trigger Lambda functions for event-driven processing.
• Amazon SNS: Fan-out messages to multiple SQS queues for parallel processing.
• Amazon EC2: Process messages in distributed systems running on EC2 instances.
• AWS Step Functions: Orchestrate workflows that include SQS as a task.
• AWS KMS: Encrypt messages at rest for enhanced security.
• Amazon CloudWatch: Monitor queue metrics and set alarms for visibility and debugging.
• AWS IAM: Control access and permissions for producers and consumers. Not Directly Integrated With:
• Direct database triggers without intermediary services like Lambda or custom scripts.

Governance and Security

1. IAM Policies:

• Manage fine-grained access to queues for producers and consumers.

2. Encryption:

• Use AWS KMS for encryption at rest.
• Messages in transit are encrypted using SSL/TLS.

3. Auditing:

• Log message activity and access through AWS CloudTrail.

4. Visibility Timeout:

• Prevent multiple consumers from processing the same message.

5. Dead-Letter Queues (DLQs):

• Isolate and debug failed messages for improved reliability.

Benefits

1. Fully Managed:

• Offload operational overhead of managing message queues.

2. Scalability:

• Automatically scales to accommodate any workload size.

3. Reliability:

• High availability and durability with at-least-once message delivery.

4. Cost-Effective:

• Pay-per-use pricing ensures costs scale with usage.

5. Flexibility:

• Supports both standard and FIFO queues for diverse use cases.

6. Ease of Integration:

• Seamless integration with AWS services and APIs for custom workflows.

7. Operational Resilience:

• Buffering mechanisms help absorb traffic spikes without service disruption.

AWS Step Functions

• Type: Serverless Orchestration Service
• Description: Enables the coordination of multiple AWS services into serverless workflows, simplifying the development and execution of multi-step applications.
• Use Cases:
• Building data processing pipelines
• Orchestrating microservices
• Automating IT and business processes

AWS Cost Management

AWS Cost and Usage Report

• Type: Billing and Usage Analytics
• Description: Provides the most detailed information about AWS usage and costs, delivered to an S3 bucket.
• Use Cases:
• Deep cost analysis
• Custom billing reports
• Integration with analytics tools like Amazon Athena or Amazon Redshift

AWS Cost Explorer

• Type: Cost Visualization Tool
• Description: Allows users to visualize, understand, and manage AWS costs and usage over time.
• Use Cases:
• Cost trend analysis
• Budget forecasting
• Cost optimization insights Additional Features:
• Savings Plans Recommendations: Optimize compute costs with custom recommendations.
• Tag-Based Filtering: Analyze costs by department or project. Governance and Security:
• Restrict access to billing data with IAM policies. Examples:
• Forecast costs for the next billing cycle to ensure budget adherence.

Savings Plans

• Type: Flexible Pricing Model
• Description: Offers significant savings over On-Demand pricing in exchange for a commitment to use a specific amount of compute power over one or three years.
• Use Cases:
• Cost savings for predictable workloads
• Optimizing compute costs
• Running EC2, Fargate, and Lambda workloads

Compute

AWS Batch

• Type: Batch Processing Service
• Description: Efficiently runs hundreds to thousands of batch computing jobs by dynamically provisioning the optimal quantity and type of compute resources.
• Use Cases:
• High-throughput data processing
• Scientific simulations
• Media transcoding

Amazon EC2 (Elastic Compute Cloud)

• Type: Scalable Virtual Servers
• Subtypes:
• On-Demand Instances: Pay-as-you-go pricing for short-term needs.
• Spot Instances: Cost-effective for non-critical workloads.
• Reserved Instances: Lower pricing for committed usage.
• Savings Plan Instances: Flexible commitment-based pricing.
• Use Cases:
• Hosting applications and websites
• Running large-scale distributed systems
• Machine learning model training

Amazon EC2 Auto Scaling

• Type: Dynamic Scaling
• Description: Automatically adjusts the number of EC2 instances based on demand.
• Use Cases:
• Maintaining application availability
• Cost optimization by scaling down during low usage

AWS Elastic Beanstalk

• Type: PaaS (Platform as a Service)
• Description: Simplifies deployment and management of applications by automatically handling capacity provisioning, load balancing, and scaling.
• Use Cases:
• Deploying web applications
• Auto-scaling applications
• Rapid environment setup for developers

AWS Outposts

• Type: Hybrid Cloud Service
• Description: Extends AWS services to on-premises data centers.
• Use Cases:
• Low-latency applications
• Hybrid cloud architectures
• On-premises data processing

AWS Serverless Application Repository

• Type: Repository for Serverless Applications
• Description: Enables users to discover, deploy, and share serverless applications.
• Use Cases:
• Quickly deploying prebuilt serverless applications
• Sharing serverless solutions across teams
• Accelerating development

VMware Cloud on AWS

• Type: Hybrid Cloud Integration
• Description: Enables migration and extension of on-premises VMware environments to AWS.
• Use Cases:
• Seamless migration to the cloud
• Hybrid cloud deployments
• Disaster recovery for VMware workloads

AWS Wavelength

• Type: Edge Computing Service
• Description: Enables developers to build applications that deliver ultra-low latency to 5G devices and edge computing workloads.
• Use Cases:
• Real-time gaming
• IoT applications
• AR/VR experiences

Containers

Amazon Elastic Container Service (ECS)

• Type: Managed Container Orchestration
• Subtypes:
• ECS Anywhere: Extends ECS to on-premises environments.
• Use Cases:
• Running containerized microservices
• Deploying applications across hybrid environments
• Managing containers without Kubernetes

Amazon Elastic Kubernetes Service (EKS)

• Type: Managed Kubernetes Service
• Subtypes:
• EKS Anywhere: Kubernetes on-premises with AWS management.
• EKS Distro: Open-source distribution of Kubernetes.
• Use Cases:
• Deploying and managing Kubernetes clusters
• Running highly scalable containerized workloads
• Hybrid Kubernetes deployments

Amazon Elastic Container Registry (ECR)

• Type: Container Registry
• Description: Provides a secure, scalable, and reliable registry for storing Docker images.
• Use Cases:
• Storing and managing container images
• Integrating with ECS and EKS
• Simplified deployment of containerized applications

Database Services Cheat Sheet

Amazon Aurora

• Type: Managed Relational Database
• Description: High-performance, fully managed relational database compatible with MySQL and PostgreSQL.
• Features:
• Distributed, fault-tolerant storage.
• Automatic backups and failover.
• Global Database for low-latency reads across regions.
• Use Cases:
• High-throughput online transaction processing (OLTP).
• Enterprise applications requiring high availability.
• E-commerce platforms and SaaS applications.

Amazon Aurora Serverless

• Type: Auto-Scaling Relational Database
• Description: Serverless version of Aurora that scales automatically based on demand.
• Features:
• Pay-per-use pricing.
• Automatic scaling to handle workload spikes.
• Compatible with MySQL and PostgreSQL.
• Use Cases:
• Applications with unpredictable traffic patterns.
• Development and testing environments.
• Cost-optimized, low-maintenance workloads.

Amazon DocumentDB (with MongoDB compatibility)

• Type: NoSQL Document Database
• Description: Managed database service designed to run MongoDB workloads.
• Features:
• Scalability with replicas and sharding.
• Fully managed backups and patching.
• High availability with multi-AZ deployments.
• Use Cases:
• Content management systems.
• Cataloging and inventory applications.
• Storing hierarchical or semi-structured data.

Amazon DynamoDB

• Type: NoSQL Key-Value and Document Database
• Description: Fully managed, serverless NoSQL database for low-latency applications.
• Features:
• Single-digit millisecond response times.
• On-demand or provisioned capacity modes.
• Global Tables for multi-region replication.
• Use Cases:
• Real-time applications (e.g., gaming leaderboards).
• IoT data storage.
• Shopping cart and session management.

Additional Features:

• Streams: Capture item-level changes for event-driven applications.
• Global Tables: Multi-region replication for low-latency global access. Governance and Security:
• Define fine-grained access controls with IAM policies.
• Monitor table activity using CloudWatch Metrics. Examples:
• Power real-time leaderboards for online games.

Amazon ElastiCache

• Type: In-Memory Data Store
• Description: Fully managed in-memory data store compatible with Redis and Memcached.
• Subtypes:
• Redis: Advanced in-memory data structure store with support for replication and persistence.
• Memcached: Simple key-value store for caching.
• Use Cases:
• Real-time caching for high-throughput applications.
• Session storage for web applications.
• Gaming leaderboards and real-time analytics.

Amazon Keyspaces (for Apache Cassandra)

• Type: Managed NoSQL Database
• Description: Fully managed database service compatible with Apache Cassandra.
• Features:
• Scalable and highly available.
• Serverless and zero-maintenance.
• Compatible with Cassandra Query Language (CQL).
• Use Cases:
• IoT applications requiring high throughput.
• Time-series data storage.
• Decentralized data models.

Amazon Neptune

• Type: Graph Database
• Description: Fully managed graph database for storing and navigating relationships.
• Features:
• Supports both property graph and RDF graph models.
• Optimized for high-performance graph queries.
• High availability with automated failover.
• Use Cases:
• Social networking applications.
• Fraud detection through relationship analysis.
• Knowledge graphs and recommendation engines.

Amazon Quantum Ledger Database (QLDB)

• Type: Immutable Ledger Database
• Description: Fully managed ledger database providing a transparent, immutable, and cryptographically verifiable transaction log.
• Features:
• Append-only, cryptographically chained data structure.
• Managed service with no server provisioning required.
• ACID-compliant transactions.
• Use Cases:
• Financial transaction tracking.
• Supply chain management.
• Identity verification and compliance systems.

Amazon RDS (Relational Database Service)

• Type: Managed Relational Database
• Description: Simplifies the setup, operation, and scaling of relational databases.
• Supported Engines:
• MySQL
• PostgreSQL
• MariaDB
• Oracle
• SQL Server
• Features:
• Automated backups and software patching.
• Multi-AZ deployments for high availability.
• Read replicas for improved performance.
• Use Cases:
• Hosting transactional databases.
• Enterprise applications requiring relational data storage.
• E-commerce platforms and CMS applications.

Amazon Redshift

• Type: Data Warehousing Service
• Description: Fully managed, petabyte-scale data warehouse for analyzing structured and semi-structured data.
• Features:
• Columnar storage for high-performance analytics.
• Integration with BI tools like QuickSight.
• Redshift Spectrum for querying S3 data without ETL.
• Use Cases:
• Business intelligence and reporting.
• Analyzing large datasets from IoT or transactional systems.
• Preparing data for machine learning.

Comparison of Database Services Service Best For Amazon Aurora High-performance relational workloads. Aurora Serverless Unpredictable or intermittent traffic patterns. Amazon DocumentDB MongoDB-compatible document database. Amazon DynamoDB Low-latency key-value and document storage. Amazon ElastiCache Real-time caching and in-memory data stores. Amazon Keyspaces Cassandra-compatible workloads for time-series or IoT data. Amazon Neptune Graph-based applications for relationship analysis. Amazon QLDB Immutable ledger use cases like transaction tracking. Amazon RDS Relational databases with multiple engine options. Amazon Redshift Analytical queries on large datasets for business intelligence.

Developer Tools

AWS X-Ray

• Type: Distributed Tracing Service
• Description: Helps developers analyze and debug distributed applications, providing insights into performance and identifying bottlenecks.
• Use Cases:
• Debugging serverless and microservices applications.
• Identifying latency issues in distributed systems.
• Monitoring end-to-end application performance.

Front-End Web and Mobile

AWS Amplify

• Type: Front-End Development Framework
• Description: Provides tools and services to build scalable, secure mobile and web applications.
• Use Cases:
• Hosting single-page applications.
• Simplifying backend integrations.
• Rapid prototyping and deployment.

Amazon API Gateway

• Type: Managed API Gateway
• Description: Enables developers to create, publish, maintain, and secure APIs.
• Use Cases:
• Creating RESTful APIs.
• Managing WebSocket APIs.
• Enabling serverless architectures with AWS Lambda. Additional Features:
• Usage Plans: Control API access with rate limiting and quotas.
• WebSocket Support: Enable real-time, two-way communication. Governance and Security:
• Implement authorization with IAM, Lambda authorizers, or Cognito.
• Use WAF for additional protection against threats. Examples:
• Build serverless APIs for mobile applications.

AWS Device Farm

• Type: Mobile App Testing Service
• Description: Tests mobile and web apps on real devices hosted in the AWS cloud.
• Use Cases:
• Cross-platform application testing.
• Identifying bugs in different OS environments.
• Automated testing for mobile applications.

Amazon Pinpoint

• Type: Customer Engagement Service
• Description: Helps businesses engage with customers via targeted messaging campaigns.
• Use Cases:
• Marketing automation.
• User retention campaigns.
• Analyzing customer behavior.

Machine Learning

Amazon Comprehend

• Type: Natural Language Processing (NLP)
• Description: Extracts insights from text, such as sentiment, key phrases, entities, and more.
• Use Cases:
• Analyzing customer feedback.
• Text classification for document processing.
• Social media sentiment analysis.

Amazon Forecast

• Type: Time-Series Forecasting Service
• Description: Uses machine learning to generate accurate forecasts for business metrics.
• Use Cases:
• Inventory planning.
• Financial forecasting.
• Resource demand predictions.

Amazon Fraud Detector

• Type: Fraud Detection Service
• Description: Identifies potentially fraudulent online activities using machine learning.
• Use Cases:
• Preventing payment fraud.
• Reducing fake account registrations.
• Monitoring transaction anomalies.

Amazon Kendra

• Type: Enterprise Search Service
• Description: Provides intelligent search capabilities for internal documents and datasets.
• Use Cases:
• Knowledge management.
• Internal document search.
• Enhancing customer support tools.

Amazon Lex

• Type: Conversational AI Service
• Description: Builds conversational interfaces using automatic speech recognition (ASR) and natural language understanding (NLU).
• Use Cases:
• Building chatbots.
• IVR systems for customer support.
• Real-time conversational applications.

Amazon Polly

• Type: Text-to-Speech Service
• Description: Converts text into lifelike speech using deep learning.
• Use Cases:
• Content accessibility for visually impaired users.
• Automating audio for news and articles.
• Building voice-enabled applications.

Amazon Rekognition

• Type: Image and Video Analysis
• Description: Provides image and video recognition capabilities, including facial recognition and object detection.
• Use Cases:
• Content moderation for images/videos.
• Real-time facial recognition in security systems.
• Image metadata extraction.

Amazon SageMaker

• Type: Machine Learning Development Platform
• Description: Helps build, train, and deploy machine learning models at scale.
• Use Cases:
• Custom machine learning model development.
• Real-time model inference.
• Automated ML pipeline creation. Additional Features:
• Autopilot: Automatically build, train, and tune ML models.
• Studio: Fully integrated development environment for ML workflows. Governance and Security:
• Encrypt ML models and datasets.
• Secure endpoints with VPC configurations. Examples:
• Predict customer churn using structured customer data.

Amazon Textract

• Type: Document Text Extraction
• Description: Automatically extracts text, forms, and tables from scanned documents.
• Use Cases:
• Invoice and receipt processing.
• Document digitization.
• Automating data entry workflows.

Amazon Transcribe

• Type: Speech-to-Text Service
• Description: Converts speech into text using machine learning.
• Use Cases:
• Transcribing customer calls.
• Real-time speech analytics.
• Captioning for videos.

Amazon Translate

• Type: Language Translation Service
• Description: Provides neural machine translation for 75+ languages.
• Use Cases:
• Localizing content for global audiences.
• Real-time translation in communication apps.
• Automating multilingual customer support.

Management and Governance Cheat Sheet

AWS Auto Scaling

• Type: Resource Scaling Service
• Description: Automatically adjusts resource capacity to maintain performance and minimize costs.
• Use Cases:
• Scaling EC2 instances based on demand.
• Scaling DynamoDB read/write capacity.
• Scaling ECS tasks for containerized applications.

AWS CloudFormation

• Type: Infrastructure as Code
• Description: Automates resource provisioning and management using JSON or YAML templates.
• Use Cases:
• Deploying consistent environments across accounts and regions.
• Version-controlling infrastructure changes.
• Managing complex multi-service deployments.

AWS CloudTrail

• Type: Logging Service
• Description: Records API calls and activity made on AWS services.
• Use Cases:
• Security auditing.
• Compliance monitoring.
• Troubleshooting operational issues.

Amazon CloudWatch

• Type: Monitoring and Observability
• Description: Collects and monitors logs, metrics, and events for AWS resources and applications.
• Features:
• Metrics: Monitor resource usage (e.g., CPU, memory).
• Alarms: Trigger actions based on thresholds.
• Logs: Aggregate and analyze application and system logs.
• CloudWatch Events: Automate responses to system changes.
• Use Cases:
• Monitoring EC2 instance performance.
• Setting alerts for resource thresholds.
• Visualizing operational metrics.

AWS Command Line Interface (AWS CLI)

• Type: Command-Line Tool
• Description: Unified tool to manage AWS services from the terminal.
• Use Cases:
• Automating repetitive tasks.
• Scripting deployments and operations.
• Querying AWS resources programmatically.

AWS Compute Optimizer

• Type: Cost and Performance Optimization
• Description: Provides recommendations to optimize AWS resources for cost and performance.
• Use Cases:
• Right-sizing EC2 instances.
• Optimizing Lambda function configurations.
• Balancing cost and performance for resource usage.

AWS Config

• Type: Configuration Management
• Description: Tracks and evaluates AWS resource configurations for compliance.
• Features:
• Configuration History: Records changes in resource configurations.
• Rules: Define compliance policies.
• Remediation: Automatically fixes non-compliant resources.
• Use Cases:
• Ensuring compliance with organizational policies.
• Auditing resource configurations.
• Detecting and fixing misconfigurations.

AWS Control Tower

• Type: Multi-Account Governance
• Description: AWS Control Tower simplifies the setup, governance, and management of multi-account AWS environments by implementing AWS best practices. It provides a pre-configured landing zone with governance controls (guardrails) to ensure security and compliance across accounts.
• Use Cases:
• Setting up multi-account environments.
• Enforcing security and compliance policies.
• Centralized account governance. Key Features:

1. Landing Zone:

• Pre-configured multi-account environment based on AWS best practices.
• Includes logging, security accounts, and predefined network setups.

2. Guardrails:

• Preventive Guardrails: Actively block non-compliant actions, such as creating resources in unauthorized regions.
• Detective Guardrails: Continuously monitor and flag non-compliance, such as unencrypted storage.

3. Service Control Policies (SCPs):

• Apply policies across organizational units (OUs) to restrict specific actions.
• Examples: Prevent IAM policy changes, deny access to certain AWS regions.

4. Account Factory:

• Automates account creation with predefined templates for networking, security, and compliance.
• Supports customization of VPC, IAM roles, and other baseline settings.

5. Audit and Logging:

• Centralized logging through AWS CloudTrail and AWS Config for compliance tracking.
• Includes an audit account for governance and analysis.

6. Dashboard:

• Provides a unified view of all accounts, compliance status, and resources.
• Simplifies operational insights and alerts. Governance and Security:

1. Integrated Services:

• AWS Organizations: Manages multi-account structure and applies SCPs.
• AWS IAM Identity Center (SSO): Manages secure and centralized user access.
• AWS Config: Tracks resource configurations and ensures compliance with guardrails.
• CloudTrail: Provides detailed logging of account activities.

2. Centralized Policy Enforcement:

• Use SCPs to restrict unauthorized actions across accounts.
• Monitor compliance through detective guardrails and Config rules.

3. Custom Guardrails:

• Define additional preventive or detective controls using Config or custom Lambda functions.
• Example: Ensure EC2 instances do not use public IP addresses.

4. Encryption and Access Control:

• Enforce encryption standards across accounts using SCPs and guardrails.
• Use fine-grained access controls to restrict sensitive actions.

Integrations:

• Security Tools:
• Integrates with GuardDuty, AWS Security Hub, and Amazon Macie for enhanced security monitoring.
• Flags anomalies and potential threats across accounts.
• Cost Management Tools:
• AWS Cost Explorer and AWS Budgets integration for tracking and managing account expenditures.
• Networking:
• Integrates with AWS Transit Gateway to streamline inter-account connectivity.

AWS Health Dashboard

• Type: Service Health Monitoring
• Description: Provides personalized information about AWS service disruptions and planned maintenance.
• Use Cases:
• Monitoring AWS service health.
• Proactive incident management.
• Receiving notifications for service events impacting your resources.

AWS License Manager

• Type: License Management Service
• Description: Simplifies the management of software licenses across AWS and on-premises environments.
• Use Cases:
• Tracking license usage for compliance.
• Managing bring-your-own-license (BYOL) workloads.
• Centralized license administration.

Amazon Managed Grafana

• Type: Visualization and Monitoring
• Description: Fully managed service for Grafana dashboards.
• Use Cases:
• Monitoring infrastructure metrics.
• Visualizing operational data.
• Integrating with CloudWatch, Prometheus, and more.

Amazon Managed Service for Prometheus

• Type: Managed Monitoring Service
• Description: Fully managed service for monitoring and alerting using Prometheus.
• Use Cases:
• Kubernetes metrics monitoring.
• Application performance monitoring.
• Centralizing metrics for large-scale systems.

AWS Management Console

• Type: Web-Based User Interface
• Description: A GUI for accessing and managing AWS services.
• Use Cases:
• Resource creation and management.
• Monitoring and troubleshooting.
• User-friendly interface for non-technical users.

AWS Organizations

• Type: Multi-Account Management
• Description: AWS Organizations simplifies the management of multiple AWS accounts by providing centralized control over policies, billing, and resource sharing. It enables organizations to enforce governance, streamline account creation, and optimize costs. Key Features:

1. Service Control Policies (SCPs):

• Enforce fine-grained permissions across accounts or organizational units (OUs).
• Examples: Restrict access to specific regions or services.

2. Organizational Units (OUs):

• Group accounts by function, project, or environment for better management.
• Apply SCPs at the OU level for consistent governance.

3. Consolidated Billing:

• Combine all accounts under a single payment method for simplified billing and cost tracking.
• Enable cost allocation tags for detailed expense analysis.

4. Resource Sharing:

• Share resources like VPCs, Transit Gateways, and license configurations across accounts securely.

5. Account Management:

• Simplify the creation and management of AWS accounts with predefined configurations. Governance and Security:
• Enforce least privilege access across accounts with SCPs.
• Monitor account activity and compliance through CloudTrail and AWS Config.
• Secure inter-account communication with AWS PrivateLink and Resource Access Manager (RAM).

Use Cases:

1. Centralized Policy Management:

• Enforce compliance by applying SCPs to OUs for consistent governance across accounts.

2. Cost Optimization:

• Use consolidated billing to track and reduce costs across the organization.

3. Resource Sharing:

• Share VPCs and other resources efficiently across accounts for centralized management.

4. Scaling Operations:

• Create and manage new accounts easily while inheriting organizational policies.

Benefits:

• Centralized Governance: Manage accounts and policies from a single location.
• Cost Efficiency: Consolidate billing and streamline cost tracking.
• Simplified Resource Sharing: Securely share resources without duplicating efforts.
• Scalability: Scale cloud operations while maintaining consistent control.

AWS Proton

• Type: Application Delivery Automation
• Description: Automates the deployment and management of container and serverless applications.
• Use Cases:
• Standardizing infrastructure and deployment.
• Managing microservices at scale.
• Simplifying developer workflows.

AWS Service Catalog

• Type: Service Deployment Management
• Description: AWS Service Catalog enables organizations to centrally manage and distribute approved IT services and applications. It simplifies governance by creating a catalog of pre-approved resources, ensuring consistency and compliance across deployments.

Key Features:

1. Portfolio Management:

• Organize and manage a collection of approved resources, including EC2 instances, RDS databases, and S3 buckets.
• Define permissions for users and groups to access specific portfolios.

2. Product Management:

• Create products using AWS CloudFormation templates to standardize resource provisioning.
• Version control allows easy updates and rollbacks.

3. Tagging and Tracking:

• Enforce tagging policies for cost allocation and resource management.
• Track usage of provisioned products across accounts.

4. Access Control:

• Granular permissions to restrict who can view, deploy, and manage resources.

5. Self-Service Portal:

• Empower users to deploy resources themselves from the catalog while adhering to governance policies.

Governance and Security:

• Compliance: Enforce organizational policies by distributing pre-approved configurations only.
• Audit and Tracking: Monitor deployments with CloudTrail and AWS Config.
• IAM Integration: Manage permissions to portfolios and products securely.
• Cost Control: Prevent over-provisioning by restricting resource specifications in templates.

Use Cases:

1. Standardized Resource Deployment:

• Ensure consistent configurations for commonly used resources like VMs or databases.

2. Cost Management:

• Restrict access to resource types or sizes that exceed budget constraints.

3. Governed Self-Service:

• Allow teams to provision resources independently while adhering to security and compliance standards.

4. Multi-Account Resource Sharing:

• Distribute resources across accounts in AWS Organizations. Benefits:
• Streamlined Management: Simplify resource deployment and updates through centralized catalogs.
• Enhanced Compliance: Ensure only approved configurations are deployed.
• Operational Efficiency: Empower teams with self-service capabilities while maintaining governance.
• Cost Optimization: Reduce waste through controlled provisioning.

AWS Systems Manager

• Type: Operations Management
• Description: AWS Systems Manager provides a unified interface for managing AWS resources and on-premises infrastructure. It simplifies operational tasks such as automation, patch management, and monitoring, while enhancing security and compliance across environments.

Key Features:

1. Session Manager:

• Provides secure shell access to EC2 instances without needing bastion hosts or SSH keys.
• Integrated logging for audit and compliance.

2. Automation:

• Automates repetitive tasks like instance provisioning and configuration management using pre-built or custom runbooks.

3. Patch Manager:

• Automates patching for operating systems and applications across instances.

4. State Manager:

• Ensures desired configurations are applied and maintained on instances.

5. Parameter Store:

• Securely store, manage, and retrieve configuration data and secrets for your applications.

6. Run Command:

• Execute commands on multiple instances without needing to log in.

7. Inventory:

• Collect and view metadata about instances and software installed across environments.

8. OpsCenter:

• Centralized dashboard for tracking and managing operational issues.

Governance and Security:

• Access Control: Manage permissions with AWS Identity and Access Management (IAM).
• Audit Trails: Use AWS CloudTrail to log all actions taken via Systems Manager.
• Encryption: Securely store sensitive data in Parameter Store using AWS Key Management Service (KMS).
• Compliance Tracking: Integrates with AWS Config to ensure resources meet compliance requirements.

Use Cases:

1. Unified Operations Management:

• Centralize operational tasks across AWS and on-premises infrastructure.

2. Patch Automation:

• Ensure all instances remain up-to-date with minimal manual intervention.

3. Configuration Management:

• Automatically apply and enforce desired instance configurations.

4. Secure Access Management:

• Provide auditable access to instances without requiring open ports or bastion hosts.

5. Incident Response:

• Use Automation and OpsCenter for faster diagnosis and resolution of operational issues. Benefits:
• Enhanced Efficiency: Streamline routine operational tasks through automation.
• Improved Security: Secure access to instances and sensitive configurations.
• Cost Savings: Reduce operational overhead with centralized tools and automation.
• Compliance Simplification: Easily monitor and enforce compliance across environments.

AWS Trusted Advisor

• Type: Resource Optimization Tool
• Description: Provides recommendations for cost optimization, security, fault tolerance, performance, and service limits.
• Use Cases:
• Identifying cost-saving opportunities.
• Enhancing security configurations.
• Optimizing AWS resource usage.

AWS Well-Architected Tool

• Type: Architecture Assessment Tool
• Description: Helps assess workloads and implement best practices using the AWS Well-Architected Framework.
• Use Cases:
• Reviewing workloads for compliance with best practices.
• Identifying potential architectural risks.
• Improving operational efficiency.

Comparison of Management and Governance Services Service Best For AWS Auto Scaling Scaling resources dynamically. AWS CloudFormation Automating resource provisioning. AWS CloudTrail Tracking API activity and compliance. Amazon CloudWatch Monitoring resources and setting alerts. AWS Config Ensuring compliance and tracking changes. AWS Control Tower Managing multi-account environments. AWS Organizations Centralizing account management. AWS Systems Manager Unified operational insights and tasks. AWS Trusted Advisor Resource optimization and best practices.

Media Services

Amazon Elastic Transcoder

• Type: Media Transcoding Service
• Description: Converts media files into formats optimized for playback on devices.
• Use Cases:
• Video format conversion.
• Streaming optimization.
• Content delivery for mobile users.

Amazon Kinesis Video Streams

• Type: Video Streaming
• Description: Processes and analyzes streaming video data in real time.
• Use Cases:
• Real-time video analytics.
• Video archiving and playback.
• Machine learning for video analysis.

Migration and Transfer

AWS DataSync

Type: Data Transfer Automation Service Description: AWS DataSync simplifies, automates, and accelerates the process of transferring and replicating large volumes of data between on-premises storage systems, edge locations, and AWS storage services over the internet or AWS Direct Connect. It supports file data and associated file system metadata such as ownership, timestamps, and access permissions, ensuring seamless data migration and synchronization.

Key Features:

1. High-Performance Data Transfer:

• Transfers large datasets at speeds significantly faster than open-source tools.

2. Metadata Preservation:

• Maintains file system metadata such as ownership, timestamps, and permissions during transfers.

3. Supported Locations:

• Amazon S3: Transfer data to/from S3 buckets, supporting all storage classes.
• Amazon EFS (Elastic File System): Seamless integration with EFS for NFS-based file systems.
• Amazon FSx (File Systems): Supports FSx for Windows File Server, FSx for Lustre, and FSx for NetApp ONTAP.
• On-Premises Storage: Includes Network File System (NFS) and Server Message Block (SMB) storage systems.

4. Protocol Support:

• Uses NFS or SMB protocols for accessing and transferring data from on-premises systems.

5. Task Automation:

• Automates data synchronization tasks, including scheduling recurring transfers.

6. Data Validation:

• Automatically verifies data integrity during transfers, ensuring accuracy.

7. AWS Management Console and API Integration:

• Configure, monitor, and manage transfers easily via the AWS Management Console or programmatically with APIs.

Subtypes and Concepts:

1. Locations:

• Define the source and destination endpoints for data transfers, such as S3 buckets, EFS file systems, or FSx systems.
• Example: A location for Amazon FSx for Windows File Server serves as an endpoint for transferring data using the Server Message Block (SMB) protocol.

2. Agents:

• Software agents deployed on-premises to facilitate secure communication between local storage systems and AWS.

3. Tasks:

• A task specifies the source, destination, filters, and configurations for a data transfer job.

4. Filters:

• Allow users to include or exclude specific files or directories during a transfer.

Use Cases:

1. Cloud Migration:

• Efficiently move on-premises data to AWS storage services such as S3, EFS, or FSx.

2. Hybrid Workloads:

• Synchronize data between on-premises storage and AWS for hybrid application scenarios.

3. Backup and Disaster Recovery:

• Automate the replication of critical data to AWS for backup and recovery purposes.

4. Data Lake Creation:

• Ingest large datasets into Amazon S3 for analytics and machine learning applications.

5. Metadata Preservation:

• Retain access permissions, ownership, and timestamps for compliance and operational needs.

Governance and Security:

• Encryption:
• Encrypts data in transit using Transport Layer Security (TLS).
• Supports encryption at rest using AWS KMS for S3 and FSx destinations.
• Access Control:
• Uses IAM roles and policies to control access to DataSync resources and target AWS services.
• Monitoring and Logging:
• Integrates with CloudWatch to provide visibility into data transfer metrics and task statuses.

Benefits:

• Simplified Management:
• Automates and schedules repetitive data transfer tasks.
• Faster Transfers:
• Moves data at high speeds, significantly reducing transfer times compared to traditional methods.
• Seamless Integration:
• Works natively with AWS storage services for smooth data migration and synchronization.
• Reliable and Secure:
• Ensures data integrity and security during transfers, with automatic verification and encryption.
• Cost Efficiency:
• Pay only for the data transferred, avoiding additional licensing or hardware costs.

AWS Migration Hub

Type: Migration Tracking and Management Service Description: AWS Migration Hub provides a centralized platform to plan, track, and manage the migration of applications and resources to AWS. It integrates with various AWS and partner migration tools, offering a unified view of migration progress and outcomes across multiple projects and services.

Key Features:

1. Centralized Migration Tracking:

• Offers a single dashboard to monitor migration progress for applications and servers across multiple AWS services and partner tools.

2. Application Grouping:

• Organize servers and resources into logical application groups for better tracking and dependency management.

3. Integration with Migration Tools:

• Supports AWS-native tools like AWS Application Migration Service, AWS Database Migration Service (DMS), and third-party tools such as CloudEndure.

4. Customizable Metrics:

• Tracks migration status using metrics like server discovery, data replication, and cutover completion.

5. Dependency Visualization:

• Automatically identifies application dependencies to ensure complete and efficient migration planning.

6. Flexible Reporting:

• Provides detailed migration reports for individual applications or the entire portfolio.

7. Multi-Region Support:

• Manage and track migrations across multiple AWS Regions for global projects.

Subtypes and Components:

1. Discovery Tools:

• Integrates with tools like AWS Application Discovery Service to collect data about on-premises infrastructure and application dependencies.

2. Migration Tools:

• Works seamlessly with AWS services like Application Migration Service, DMS, and partner tools to perform the actual migration.

3. Application Portfolio:

• Helps group and manage applications, enabling a structured approach to migration.

4. Progress Status:

• Tracks and categorizes migration phases (e.g., data replication, cutover, testing) for each application or server.

Use Cases:

1. End-to-End Migration Management:

• Plan, execute, and monitor migrations of applications, servers, and databases from on-premises or other clouds to AWS.

2. Application Dependency Mapping:

• Discover and visualize interdependencies between applications to reduce migration risks and ensure smooth transitions.

3. Migration Tool Consolidation:

• Use a unified dashboard to manage migrations from multiple tools, reducing complexity and improving visibility.

4. Progress Monitoring:

• Track real-time migration progress and completion status for individual applications or portfolios.

5. Compliance and Reporting:

• Generate detailed reports for auditing and compliance purposes.

Governance and Security:

• Role-Based Access:
• Use IAM policies to control access to Migration Hub resources and data.
• Integration with CloudTrail:
• Logs all user activities and API calls for monitoring and compliance.
• Encryption:
• Ensures secure data handling by integrating with AWS KMS for encryption.

Benefits:

• Centralized Visibility:
• Consolidates migration progress and metrics into a single view, reducing management overhead.
• Simplified Planning:
• Facilitates detailed planning and execution by identifying dependencies and tracking tools.
• Enhanced Collaboration:
• Teams can work collaboratively with shared insights and consistent progress updates.
• Tool Agnostic:
• Supports a variety of migration tools, making it versatile for different workloads and environments.
• Scalable Management:
• Handles small-scale to enterprise-level migrations seamlessly.

AWS Snow Family

Type: Offline Data Transfer and Edge Computing Services Description: The AWS Snow Family consists of physical devices designed for offline data transfer and edge computing. It enables the movement of large volumes of data between on-premises locations and AWS or facilitates compute capabilities in disconnected or edge environments. The family includes Snowcone, Snowball, and Snowmobile, each tailored for specific data transfer and edge computing needs.

Key Services in AWS Snow Family

1. AWS Snowcone

   • Description:
   • The smallest member of the Snow Family, ideal for portable edge computing and data transfer.
   • Lightweight and rugged, with onboard compute for running applications in remote environments.
   • Capacity:
   • Storage: 8 TB of usable storage.
   • Use Cases:
   • Portable data collection in remote or mobile environments.
   • Edge computing for IoT devices.
   • Data transfer from constrained locations.

2. AWS Snowball

   • Description:
   • Available in two variants: Snowball Edge Storage Optimized and Snowball Edge Compute Optimized.
   • Supports large-scale data transfer and edge computing.
   • Capacity:
   • Storage Optimized: Up to 80 TB of usable storage.
   • Compute Optimized: Supports 42 TB of usable storage with additional compute capabilities.
   • Features:
   • Edge Computing: Run applications on Snowball devices using AWS IoT Greengrass and EC2 instances.
   • Encryption: Data is encrypted with AWS KMS during transfer.
   • Use Cases:
   • Large-scale data migrations to AWS.
   • Processing and analyzing data in disconnected environments.
   • Disaster recovery and backup.

3. AWS Snowmobile

   • Description:
   • A 45-foot shipping container designed for petabyte- and exabyte-scale data migrations.
   • Moves massive datasets securely and efficiently.
   • Capacity:
   • Up to 100 PB per Snowmobile.
   • Use Cases:
   • Migrating entire data centers to AWS.
   • High-volume archival and regulatory data transfers.
   • Large-scale digital media migration.

Common Features Across the Snow Family

1. Secure Data Handling:

• End-to-end encryption using AWS KMS.
• Tamper-evident devices ensure data security during transit.

2. Integration with AWS Services:

• Data is seamlessly ingested into Amazon S3, EBS, or Glacier.

3. Data Validation:

• Automatically validates data upon transfer to ensure integrity.

4. Durability:

• Designed to withstand harsh environments for remote or edge use cases.

5. Edge Computing Capabilities:

• Run applications locally on Snow devices for preprocessing, analytics, or AI/ML inference.

Use Cases

1. Data Transfer and Migration:

• Efficiently move large volumes of data from on-premises to AWS without relying on the internet.

2. Edge Computing:

• Process and analyze data in disconnected or latency-sensitive environments, such as oil rigs, ships, or military operations.

3. Disaster Recovery:

• Rapidly back up or restore large datasets for disaster recovery planning.

4. Media and Entertainment:

• Transfer massive video libraries or high-resolution content for processing and archiving.

5. IoT and Analytics:

• Collect and preprocess IoT data at the edge before transferring to AWS for further analysis.

Benefits

• Scalability:
• Suitable for small data volumes (Snowcone) to massive datasets (Snowmobile).
• Cost Efficiency:
• Avoids high network transfer costs by utilizing offline transfer methods.
• Portability:
• Compact and rugged devices like Snowcone and Snowball can operate in remote and challenging environments.
• Enhanced Security:
• Hardware encryption and tamper-resistant designs safeguard sensitive data.
• Reliability:
• Provides consistent data transfer and edge compute capabilities even in disconnected environments.

AWS Transfer Family

Type: Managed File Transfer Service Description: The AWS Transfer Family enables secure and reliable file transfers directly into and out of AWS storage services using industry-standard file transfer protocols such as SFTP (Secure File Transfer Protocol), FTPS (File Transfer Protocol Secure), and FTP (File Transfer Protocol). It integrates seamlessly with Amazon S3 and Amazon EFS, making it easy to modernize and migrate legacy file transfer workflows to the cloud without changing client-side configurations.

Key Features

1. Protocol Support:

• SFTP (SSH File Transfer Protocol): Securely transfers files using encryption and authentication.
• FTPS (File Transfer Protocol Secure): Transfers files over an encrypted SSL/TLS connection.
• FTP (File Transfer Protocol): For legacy workflows requiring unencrypted file transfers.

2. Native AWS Integration:

• Transfers files directly to Amazon S3 or Amazon EFS, enabling further processing or storage in the AWS ecosystem.

3. User Management:

• Supports Amazon Cognito, AWS Directory Service, or custom identity providers for user authentication.

4. High Availability and Scalability:

• Fully managed service with automatic scaling to handle fluctuating workloads without manual intervention.

5. Customizable Workflows:

• Supports custom workflows for post-transfer processing, such as Lambda triggers for file processing or moving files to different S3 buckets.

6. Monitoring and Logging:

• Integrated with AWS CloudWatch and AWS CloudTrail for detailed monitoring, auditing, and troubleshooting.

7. Security and Compliance:

• Provides fine-grained access control using IAM policies.
• Offers encryption for data at rest (S3, EFS) and in transit.

Subtypes and Components

1. Endpoint Types:

• Public endpoints for internet-facing transfers.
• VPC-hosted endpoints for private, secure transfers within a Virtual Private Cloud (VPC).

2. Authentication Methods:

• Service-Managed Users: Store user credentials securely in AWS Transfer Family.
• Custom Identity Providers: Use AWS Lambda to authenticate users via external systems.
• AWS Directory Service: Integrate with Microsoft Active Directory for enterprise authentication.

3. Custom Processing Workflows:

• Automate actions like virus scanning, data parsing, or moving files across buckets using AWS Lambda. Use Cases

1. Legacy File Transfer Modernization:

• Migrate on-premises SFTP/FTPS/FTP workflows to AWS without client-side changes.

2. Secure Data Exchange:

• Enable secure data sharing with business partners, vendors, or clients.

3. Integration with Analytics Pipelines:

• Automate ingestion of files into data lakes or analytics pipelines using Amazon S3.

4. Multi-Region Data Access:

• Transfer data across regions for global applications or disaster recovery.

5. IoT and Edge Data Collection:

• Securely transfer files from edge devices to AWS for processing and analysis. Governance and Security
• Encryption:
• Data is encrypted in transit using SFTP or FTPS and at rest in S3 or EFS.
• Access Control:
• Use IAM roles and policies to define granular access permissions for users.
• Auditing and Monitoring:
• Logs all file transfer activities via CloudTrail and provides performance metrics in CloudWatch.
• Compliance:
• Meets industry standards like GDPR, HIPAA, PCI DSS, and FedRAMP, ensuring regulatory compliance for sensitive workloads. Benefits
• Ease of Use: Fully managed service eliminates the need to set up or maintain file transfer servers.
• Scalability: Automatically scales to handle large volumes of file transfers.
• Cost-Effectiveness: Pay only for the resources used, without additional licensing costs.
• Compatibility: Supports legacy protocols for seamless migration without disrupting existing workflows.
• Enhanced Security: Built-in encryption, IAM-based access control, and compliance certifications.

AWS Application Migration Service (AWS MGN)

Type: Lift-and-Shift Migration Service Description: AWS Application Migration Service (AWS MGN) simplifies the migration of applications from physical, virtual, or cloud-based environments to AWS with minimal downtime. It continuously replicates source servers into AWS and converts them into native AWS resources. This approach allows for efficient and reliable migrations without requiring application re-architecture.

Key Features

1. Continuous Replication:

• Uses block-level replication to copy data from source servers to AWS, ensuring minimal downtime during migration.

2. Automated Conversion:

• Converts replicated servers into Amazon EC2 instances, retaining the original application stack.

3. Testing and Validation:

• Launch test instances in AWS to validate the migration before final cutover.

4. Support for Multiple Platforms:

• Migrates workloads from physical servers, VMware, Hyper-V, KVM, and other cloud providers.

5. Customizable Launch Settings:

• Define EC2 instance types, networking configurations, and security groups for target resources.

6. Post-Migration Modernization:

• Integrate migrated applications with AWS services, such as RDS, Lambda, or S3, for optimization.

7. Integrated Dashboard:

• Provides a centralized view of replication status, server health, and migration progress.

Subtypes and Components

1. Replication Agent:

• A lightweight agent installed on source servers to facilitate data replication to AWS.

2. Launch Templates:

• Define instance types, storage configurations, and networking settings for converted resources.

3. Testing and Cutover:

• Test migrated workloads in isolation before switching production traffic.

4. Orchestration:

• Automated workflows to manage replication, testing, and launch processes.

Use Cases

1. Lift-and-Shift Migrations:

• Quickly move existing workloads to AWS without requiring changes to the application stack.

2. Disaster Recovery:

• Set up AWS as a disaster recovery site with near real-time replication and rapid failover capabilities.

3. Data Center Consolidation:

• Migrate workloads from multiple on-premises data centers to AWS.

4. Cloud-to-Cloud Migration:

• Transition applications from other cloud providers to AWS for consistent operations.

5. Application Modernization:

• Migrate legacy applications to AWS, followed by modernization using AWS services.

Governance and Security

• Encryption:
• Data is encrypted in transit using TLS and optionally encrypted at rest using AWS KMS.
• Access Control:
• Utilize IAM roles to control access to migration resources and operations.
• Monitoring and Logging:
• Integrated with AWS CloudWatch for real-time monitoring and CloudTrail for auditing migration activities.

Benefits

1. Simplified Migrations:

• Eliminates the complexity of manual migration steps, reducing migration effort and risk.

2. Minimal Downtime:

• Continuous replication ensures that workloads remain operational until the final cutover.

3. Cost-Effective:

• Pay only for the resources consumed during replication and testing, with no licensing costs.

4. Scalability:

• Supports large-scale migrations of hundreds or thousands of servers.

5. Flexibility:

• Customize launch configurations to match the performance and security requirements of target environments.

AWS Application Discovery Service

Type: Migration Planning Service Description: AWS Application Discovery Service helps enterprises plan migrations by collecting detailed information about on-premises workloads, including server configurations, utilization, and application dependencies. This data enables organizations to make informed decisions and prioritize migration efforts while reducing risk and downtime during migration.

Key Features

1. Discovery Agents:

• Lightweight software agents installed on on-premises servers to collect detailed system and application metadata, including CPU, memory, storage, and network configurations.

2. Agentless Discovery:

• Uses VMware vCenter to collect information about virtual machines without installing agents.

3. Application Dependency Mapping:

• Automatically identifies dependencies between applications and servers, providing a comprehensive view of workloads.

4. Performance Metrics:

• Gathers resource utilization data (CPU, memory, disk, and network) for workload sizing and optimization in AWS.

5. Data Integration:

• Exports collected data to AWS Migration Hub or other tools for further analysis and migration tracking.

6. Customizable Filters:

• Focus discovery efforts on specific applications, workloads, or servers for granular migration planning.

7. Centralized Dashboard:

• Provides a single view of discovered resources and dependencies, streamlining the assessment process.

Subtypes and Components

1. Discovery Agents:

• Installed on individual servers to capture system-level details, including installed software, running processes, and resource utilization.

2. Agentless Discovery Connector:

• Deployed in VMware environments to gather data without requiring agent installation.

3. Data Export:

• Integrates with AWS Migration Hub for centralized migration tracking and with other analysis tools like Amazon Athena or Amazon QuickSight.

4. Dependency Visualization:

• Graphically displays server and application interdependencies, enabling better migration strategies.

Use Cases

1. Migration Planning:

• Analyze on-premises workloads to determine migration feasibility and resource requirements.

2. Application Dependency Mapping:

• Identify interdependent applications to ensure seamless migrations with minimal downtime.

3. Cloud Readiness Assessment:

• Understand existing workload configurations to determine compatibility and cost optimization in AWS.

4. Cost Estimation:

• Use collected utilization data to right-size AWS resources, reducing migration costs.

5. Risk Mitigation:

• Identify potential bottlenecks or challenges before initiating migrations.

Governance and Security

• Data Encryption:
• Data collected by agents is encrypted in transit and at rest to ensure security.
• Access Control:
• Role-based access through IAM policies to restrict who can view and export discovery data.
• Auditing:
• Integration with AWS CloudTrail to log API activity and provide audit trails.

Benefits

1. Comprehensive Visibility:

• Provides detailed insights into on-premises workloads, reducing blind spots during migration.

2. Simplified Planning:

• Automates the discovery process, saving time and effort in collecting system details.

3. Improved Accuracy:

• Helps avoid over-provisioning or under-provisioning AWS resources by providing accurate utilization data.

4. Seamless Integration:

• Works with AWS Migration Hub and other AWS services for end-to-end migration tracking.

5. Reduced Risk:

• Dependency mapping ensures that all related applications are migrated together, minimizing disruptions.

Networking and Content Delivery Cheat Sheet

AWS Client VPN

• Type: Secure VPN Access
• Description: Provides a secure VPN solution to access AWS and on-premises resources.
• Features:
• Supports OpenVPN-based connections.
• Fully managed and scalable.
• Integrates with Active Directory for authentication.
• Use Cases:
• Remote workforce access to AWS resources.
• Secure access to on-premises and cloud environments.
• Enabling multi-factor authentication (MFA) for remote connections.

Amazon CloudFront

• Type: Content Delivery Network (CDN)
• Description: Distributes content globally with low latency and high transfer speeds using edge locations.
• Features:
• Edge Locations: Points of presence (POPs) for caching content.
• Origin Shield: Extra caching layer for higher cache hit ratios.
• Custom SSL Certificates: Secure HTTPS connections.
• Use Cases:
• Accelerating website and API delivery.
• Streaming media content.
• Secure content distribution with geo-restrictions.

AWS Direct Connect

• Type: Dedicated Network Connection
• Description: AWS Direct Connect establishes a private, high-speed, and low-latency network connection between your on-premises infrastructure and AWS. It provides a reliable alternative to internet-based connectivity, enabling hybrid cloud architectures with consistent performance and enhanced security. Key Features:

1. Dedicated Connectivity:

• Offers a dedicated physical connection to AWS, bypassing the public internet for improved performance and reliability.

2. High Bandwidth:

• Supports bandwidth options from 50 Mbps to 100 Gbps, accommodating diverse workload needs.

3. Reduced Latency:

• Provides consistent and predictable network performance, ideal for latency-sensitive applications.

4. Resilience and Redundancy:

• Supports redundant connections and integration with AWS Transit Gateway for high availability.

5. Private VIF and Public VIF:

• Private Virtual Interface (VIF): Connects directly to VPCs for private communication.
• Public Virtual Interface (VIF): Provides access to public AWS services, such as S3 and DynamoDB.

6. Direct Connect Gateway:

• Allows you to connect to multiple VPCs across regions using a single Direct Connect connection.

Governance and Security:

• Dedicated Links: Isolate traffic from the public internet to improve security and minimize risks.
• Encryption: Use VPN over Direct Connect for additional encryption if required.
• Compliance: Meet strict regulatory and compliance requirements with private connections.
• Access Control: Manage access to resources via IAM and VPC security groups.

Use Cases:

1. Hybrid Cloud Architectures:

• Extend on-premises infrastructure to the cloud with consistent performance.

2. Data-Intensive Workloads:

• Transfer large datasets to AWS efficiently, such as for backups, analytics, or media processing.

3. Latency-Sensitive Applications:

• Host applications requiring low latency, such as gaming, financial trading, or IoT.

4. Regulatory Compliance:

• Ensure secure and compliant connectivity for industries like healthcare and finance.

5. Multi-Region Access:

• Connect to multiple VPCs and regions using Direct Connect Gateway for global workloads. Benefits:
• Improved Performance: Consistent bandwidth and reduced network jitter.
• Cost Savings: Reduce data transfer costs compared to internet-based connections.
• Enhanced Security: Private, dedicated connections enhance data security.
• Scalability: Flexible bandwidth options support growing workloads.

Elastic Load Balancing (ELB)

• Type: Load Balancing Service
• Description: Elastic Load Balancing (ELB) automatically distributes incoming application traffic across multiple targets, such as Amazon EC2 instances, containers, IP addresses, and Lambda functions, to ensure high availability and fault tolerance. ELB supports multiple protocols, including HTTP, HTTPS, TCP, and UDP, and integrates seamlessly with other AWS services. Key Features:

1. Types of Load Balancers:

• Application Load Balancer (ALB): Operates at Layer 7 (HTTP/HTTPS) and supports advanced routing, such as path- or host-based routing and WebSocket connections. ALB cannot have static IP addresses.
• Network Load Balancer (NLB): Operates at Layer 4 (TCP/UDP) for ultra-low latency and high throughput, ideal for real-time applications. NLB can have static ip addresses.
• Gateway Load Balancer (GWLB): Simplifies deployment and scaling of virtual appliances like firewalls and monitoring systems.
• Classic Load Balancer (CLB): Legacy option supporting Layer 4 and Layer 7, primarily for existing applications.

2. Auto Scaling Integration:

• Automatically scales resources behind the load balancer to handle traffic spikes.

3. Health Checks:

• Monitors the health of targets and routes traffic only to healthy instances.

4. SSL/TLS Termination:

• Offloads SSL/TLS encryption to improve application performance.

5. Cross-Zone Load Balancing:

• Distributes traffic evenly across instances in multiple Availability Zones for enhanced reliability.

6. Sticky Sessions:

• Maintains user sessions by directing traffic from the same client to the same target.

7. Logging and Monitoring:

• Integration with CloudWatch for monitoring metrics, and access logging to S3 for auditing and troubleshooting.

Governance and Security:

• Access Control:
• Use security groups and IAM policies to restrict access.
• TLS Encryption:
• Ensure secure communication with SSL/TLS certificates via AWS Certificate Manager (ACM).
• Compliance:
• Meet regulatory requirements with integrated encryption and logging.
• Web Application Firewall (WAF):
• Protect web applications from common threats like SQL injection and cross-site scripting (XSS).

Use Cases:

1. Web Applications:

• Use ALB to manage HTTP/HTTPS traffic with advanced routing for microservices and containerized applications.

2. Low-Latency Applications:

• Use NLB for latency-sensitive workloads, such as gaming or financial transactions.

3. Virtual Appliances:

• Deploy and scale third-party appliances with GWLB for simplified management.

4. Hybrid Architectures:

• Route traffic between on-premises data centers and AWS cloud resources.

5. Session Persistence:

• Maintain sticky sessions for stateful applications like e-commerce websites.

Benefits:

• High Availability: Distributes traffic across multiple targets and Availability Zones.
• Scalability: Automatically adjusts capacity to meet demand.
• Improved Security: Leverages encryption, access controls, and AWS WAF integration.
• Flexibility: Supports multiple protocols and workloads with different load balancer types.

AWS Global Accelerator

• Type: Network Performance Enhancer
• Description: AWS Global Accelerator improves application performance by routing user traffic through the AWS global network to the optimal endpoint based on health, geography, and latency. It provides static IP addresses that remain constant regardless of endpoint changes, simplifying DNS management and enhancing failover capabilities. Key Features:

1. Static Anycast IPs:

• Assigns two static IP addresses for your application, simplifying DNS management.
• Remains unchanged even if backend resources or regions are updated.

2. Global Traffic Management:

• Routes traffic to the optimal endpoint across AWS Regions for low latency and high availability.

3. Health Checks:

• Monitors endpoint health and automatically redirects traffic away from unhealthy targets.

4. Traffic Dials:

• Control the proportion of traffic directed to specific AWS Regions, enabling blue/green deployments or gradual migrations.

5. Integrated Security:

• Leverages AWS Shield and WAF for DDoS protection and web application security.

6. Seamless Integration:

• Works with Elastic Load Balancers, EC2 instances, and AWS Application Load Balancers.

Governance and Security:

• Enhanced Security:
• Traffic is routed over the AWS global network, reducing exposure to public internet risks.
• DDoS Protection:
• Built-in integration with AWS Shield provides robust protection against distributed denial-of-service (DDoS) attacks.
• Access Control:
• Use security groups, IAM roles, and AWS WAF to secure traffic and endpoints.
• Compliance:
• Suitable for applications requiring compliance with strict regulatory standards by providing secure and predictable traffic routing.

Use Cases:

1. Global Application Acceleration:

• Improve the performance of latency-sensitive applications such as gaming, media streaming, or IoT by routing traffic through the AWS backbone.

2. Disaster Recovery and High Availability:

• Automatically failover traffic to healthy endpoints in case of regional outages or endpoint failures.

3. Multi-Region Deployment Optimization:

• Manage traffic across multiple AWS Regions with granular control and reduced latency.

4. Hybrid Workloads:

• Accelerate traffic between on-premises data centers and AWS.

5. Blue/Green Deployments:

• Use traffic dials to route portions of traffic to new environments for testing or staged rollouts.

Benefits:

• Improved Performance: Leverages the AWS global network to reduce latency and jitter.
• High Availability: Ensures seamless failover and reliability across endpoints.
• Ease of Management: Static IPs simplify DNS configurations and reduce operational overhead.
• Enhanced Security: Routes traffic securely, leveraging AWS’s infrastructure and built-in protections.

AWS PrivateLink

• Type: Private Connectivity
• Description: Establishes private connectivity between VPCs and AWS services or third-party applications without exposing traffic to the internet.
• Features:
• Uses interface VPC endpoints.
• Supports multiple VPCs within the same account or across accounts.
• Use Cases:
• Secure access to AWS services from VPCs.
• Connecting SaaS providers to customer VPCs.
• Ensuring compliance with security standards.

Amazon Route 53

• Type: Domain Name System (DNS) Service
• Description: Provides scalable, highly available DNS and domain registration. Key Features:

1. DNS Management:

• Host and manage DNS records for custom domain names.
• Supports all common DNS record types (e.g., A, AAAA, CNAME, MX, TXT).

2. Routing Policies:

• Simple: Directs traffic to a single resource.
• Weighted: Splits traffic across multiple resources based on assigned weights.
• Latency-Based: Routes traffic to the region with the lowest latency.
• Failover: Redirects traffic to healthy resources during failures.
• Geolocation: Routes traffic based on the user’s geographic location.
• Multi-Value Answer: Returns multiple healthy endpoints for high availability.

3. Domain Registration:

• Allows you to register and manage domain names directly through Route 53.

4. Health Checks and Monitoring:

• Performs health checks to ensure resources are available.
• Automatically removes unhealthy resources from DNS routing.

5. DNS Failover:

• Provides seamless failover between primary and secondary resources during downtime.

6. Integration with AWS Services:

• Works seamlessly with services like CloudFront, Elastic Load Balancing (ELB), and S3 for routing traffic.

Governance and Security:

• IAM Permissions: Control access to Route 53 resources.
• Private Hosted Zones: Create private DNS zones for resources in Amazon VPCs.
• DNSSEC Support: Protects against DNS spoofing and ensures data integrity.
• Logging and Auditing: Integrates with CloudTrail to track changes and DNS queries.

Use Cases:

1. Website and Application Hosting:

• Route traffic to web servers, APIs, or other endpoints across AWS or on-premises environments.

2. High Availability and Failover:

• Use health checks and failover policies to ensure application availability during outages.

3. Latency Optimization:

• Direct users to the nearest regional endpoints for reduced latency.

4. Global Traffic Distribution:

• Implement geolocation or weighted routing for tailored user experiences or load distribution.

5. Multi-Region Disaster Recovery:

• Automatically route traffic to backup resources during regional outages.

Benefits:

• Reliability: Ensures consistent and uninterrupted DNS resolution.
• Scalability: Handles billions of queries per day with low latency.
• Flexibility: Offers multiple routing policies to meet diverse requirements.
• Security: Provides advanced features like DNSSEC and private hosted zones for secure DNS management.
• Seamless Integration: Works effortlessly with other AWS services for complete traffic management.

AWS Site-to-Site VPN

• Type: Managed VPN Service
• Description: Establishes a secure connection between AWS VPCs and on-premises data centers.
• Features:
• Supports IPsec tunnels.
• Redundant connections for high availability.
• Compatible with most on-premises VPN devices.
• Use Cases:
• Hybrid cloud connectivity.
• Secure data transfer between on-premises and AWS.
• Disaster recovery setups.

AWS Transit Gateway

• Type: Network Transit Hub
• Description: Simplifies connectivity between multiple VPCs, AWS accounts, and on-premises networks.
• Features:
• Centralized connectivity for large networks.
• Integration with Direct Connect and Site-to-Site VPN.
• Scalable for thousands of VPCs and connections.
• Use Cases:
• Centralized VPC management in large organizations.
• Multi-region application deployment.
• Simplified network architecture.

Amazon VPC (Virtual Private Cloud)

• Type: Virtual Networking
• Description: Provides a logically isolated network environment in the AWS Cloud. Key Features:

1. Subnets:

• Separate resources by Availability Zones (AZs) and designate as public or private for controlled access.

2. Security Groups:

• Act as stateful firewalls for resources, controlling inbound and outbound traffic at the instance level.

3. Network ACLs (Access Control Lists):

• Provide stateless filtering of traffic at the subnet level to enforce network-level access policies.

4. VPC Peering:

• Connect two VPCs across accounts or Regions to enable resource sharing while maintaining isolation.

5. Internet Gateway (IGW):

• Enables internet access for public-facing resources like web servers.

6. NAT Gateway:

• Allows private instances to access the internet without exposing them directly, supporting security and operational needs.

7. Elastic IPs:

• Assign static public IP addresses to resources for consistent access.

8. Route Tables:

• Direct traffic between subnets, VPCs, and external resources using customizable routes.

9. PrivateLink:

• Connect VPCs to AWS services and third-party applications privately, bypassing the internet.

10. Transit Gateway:

• Simplify connectivity between multiple VPCs, AWS accounts, and on-premises networks. Note: Enhanced networking provides higher bandwidth, higher packet per second (PPS) performance, and consistently lower inter-instance latencies. Governance and Security:
• Access Control:
• Use IAM policies, Security Groups, and Network ACLs to restrict access.
• Monitoring:
• Integrates with CloudWatch and VPC Flow Logs for network monitoring and troubleshooting.
• Encryption:
• Use VPN or Direct Connect for encrypted connections between on-premises and AWS.
• Isolation:
• Maintain logical separation between resources in different VPCs using subnets and route tables.

Use Cases:

1. Web Hosting:

• Host web applications in public subnets with secure backend databases in private subnets.

2. Hybrid Networking:

• Connect on-premises data centers with VPCs using VPN or AWS Direct Connect.

3. Microservices Architectures:

• Use separate subnets and security groups to isolate services.

4. Disaster Recovery:

• Set up failover environments with VPC Peering and NAT Gateways.

5. High Availability:

• Distribute resources across multiple subnets in different Availability Zones for fault tolerance.

6. Big Data and Analytics:

• Process sensitive data in private subnets while securely transferring results to public endpoints.

Benefits:

• Customizable Networking: Tailor VPCs to meet specific application and compliance requirements.
• Enhanced Security: Secure resources with fine-grained access controls.
• Scalability: Expand your network with additional subnets, route tables, or VPC peering.
• Cost-Effective: Leverage NAT Gateways and Internet Gateways for optimal resource access and security.
• Integrated Services: Seamlessly connect with other AWS services like S3, EC2, and Lambda.

Comparison of Networking and Content Delivery Services Service Best For AWS Client VPN Secure remote access for workforce. Amazon CloudFront Accelerated content delivery with global caching. AWS Direct Connect High-speed, low-latency hybrid connectivity. Elastic Load Balancing Traffic distribution and fault tolerance. AWS Global Accelerator Improving global application performance. AWS PrivateLink Private access to AWS services and third-party apps. Amazon Route 53 DNS management and traffic routing. AWS Site-to-Site VPN Secure connections between AWS and on-premises. AWS Transit Gateway Centralized multi-VPC and on-premises connectivity. Amazon VPC Isolated, customizable cloud networks.

Security, Identity, and Compliance Cheat Sheet

AWS Artifact

• Type: Compliance Documentation
• Description: Provides access to AWS compliance reports and agreements.
• Features:
• Downloadable audit artifacts like SOC reports, PCI documentation, and ISO certifications.
• On-demand self-service access.
• Use Cases:
• Simplifying compliance audits.
• Demonstrating regulatory compliance to stakeholders.
• Accessing contractual agreements for specific AWS services.

AWS Audit Manager

• Type: Compliance Automation Tool
• Description: Helps continuously audit AWS usage to ensure compliance with industry standards.
• Features:
• Pre-built frameworks for standards like GDPR, HIPAA, and ISO.
• Automated evidence collection.
• Use Cases:
• Preparing for compliance audits.
• Reducing manual effort in audit preparation.
• Monitoring ongoing compliance.

AWS Certificate Manager (ACM)

• Type: SSL/TLS Certificate Management
• Description: Simplifies the provisioning, management, and deployment of SSL/TLS certificates for securing websites and applications.
• Features:
• Automatic certificate renewal.
• Integration with services like CloudFront and ELB.
• Use Cases:
• Enabling HTTPS for web applications.
• Securing API endpoints.
• Managing SSL/TLS certificates without manual intervention.

AWS CloudHSM

• Type: Hardware Security Module (HSM)
• Description: Offers dedicated hardware for secure key storage and cryptographic operations.
• Features:
• FIPS 140-2 Level 3 compliance.
• Full control over cryptographic keys.
• Use Cases:
• Storing and managing encryption keys securely.
• Compliance with stringent cryptographic standards.
• Offloading cryptographic operations.

Amazon Cognito

Type: Identity and Access Management Service Description: Amazon Cognito provides authentication, authorization, and user management for web and mobile applications. It simplifies the process of adding user sign-up, sign-in, and access control to applications while supporting federated identities from social identity providers (Google, Facebook, Apple, etc.) and enterprise identity providers (SAML, OpenID Connect).

Key Features

1. User Pools:

• Manage user directories for app authentication with built-in support for user registration, login, and recovery flows.
• Supports multi-factor authentication (MFA) and advanced security features like adaptive authentication.

2. Identity Pools (Federated Identities):

• Grant temporary access to AWS services based on user identities.
• Combine federated identities from social providers and user pools.

3. Social and Enterprise Identity Providers:

• Integrates with social providers (Google, Facebook, Apple, Amazon).
• Supports enterprise providers via SAML 2.0 and OpenID Connect.

4. Hosted UI:

• Prebuilt, customizable web UI for user sign-up, sign-in, and password management.

5. Security Features:

• Multi-Factor Authentication (MFA): Adds an extra layer of security using SMS or TOTP-based authentication.
• Adaptive Authentication: Detects unusual sign-in activities and enforces additional security challenges.
• Advanced Security Features: Mitigates compromised credential use and identifies risky users.

6. Token-Based Authentication:

• Issues JSON Web Tokens (JWTs) for secure communication and resource access.

7. Custom Authentication Flows:

• Build custom workflows using AWS Lambda triggers for tailored user experiences.

8. User Management:

• Supports user registration, password reset, account recovery, and account deactivation.

9. Global User Access:

• Provides globally accessible endpoints for consistent user experiences across regions.

10. Scalable and Serverless:

• Automatically scales to handle millions of users without infrastructure management.

Components

1. User Pool:

• Central directory for managing user authentication and attributes.

2. Identity Pool:

• Enables temporary access to AWS resources by mapping user identities to IAM roles.

3. Tokens:

• ID Token: Contains user profile information (e.g., name, email).
• Access Token: Used to authorize access to resources.
• Refresh Token: Allows refreshing of ID and Access tokens.

4. Hosted UI:

• Fully managed user interface for authentication flows, which can be customized to match branding.

5. Lambda Triggers:

• Customize workflows like pre-signup validation, post-authentication actions, and token generation.

Use Cases

1. User Authentication:

• Manage sign-up, sign-in, and user verification for web and mobile applications.

2. Access Control for AWS Resources:

• Provide temporary access to S3, DynamoDB, and other AWS services using identity pools.

3. Federated Authentication:

• Enable users to log in using social identity providers like Google or Facebook.

4. Secure Applications:

• Use multi-factor authentication (MFA) and adaptive authentication to enhance app security.

5. Enterprise Authentication:

• Integrate with SAML or OpenID Connect for single sign-on (SSO) and enterprise identity federation.

6. Custom Authentication Workflows:

• Build tailored experiences for applications using Lambda triggers.

7. Cross-Platform Access:

• Provide consistent user experiences and authentication across devices and platforms.

Integration with AWS Services Integrated With:

• Amazon API Gateway: Protect APIs with Cognito user pools as an authentication mechanism.
• AWS Lambda: Extend Cognito workflows with custom triggers.
• Amazon S3: Grant temporary, scoped access to files based on user permissions.
• Amazon DynamoDB: Enable secure access for user-specific data storage.
• AWS AppSync: Authenticate GraphQL APIs using Cognito.
• AWS Amplify: Simplify integration of authentication for front-end applications.
• AWS IAM: Map Cognito users to IAM roles for secure access to AWS resources.
• Amazon CloudWatch: Monitor user authentication activities and metrics. Not Integrated With:
• Direct integration with databases (e.g., RDS, MongoDB) without using a custom API layer.

Governance and Security

1. IAM Policies:

• Control access to Cognito resources and associated AWS services.

2. Encryption:

• Encrypts user data at rest and in transit.
• Uses AWS KMS for encryption of sensitive attributes.

3. Multi-Factor Authentication (MFA):

• Adds an extra layer of protection via SMS or authenticator apps.

4. Advanced Security:

• Detects compromised credentials and unusual sign-in behaviors.

5. Auditing and Logging:

• Track user activity and authentication logs using AWS CloudTrail.

Benefits

1. Ease of Use:

• Simplifies integration of authentication and user management for applications.

2. Scalable:

• Handles millions of users with no operational overhead.

3. Secure:

• Provides industry-standard security practices like token-based authentication and MFA.

4. Cost-Effective:

• Pay only for active users and operations, with a free tier for small-scale applications.

5. Flexible:

• Supports a wide range of authentication providers, workflows, and integration options.

6. Developer-Friendly:

• Offers SDKs, APIs, and CLI for easy integration and management.

Amazon Detective

• Type: Security Investigation Tool
• Description: Analyzes and visualizes security data to identify and investigate potential threats.
• Features:
• Automated threat correlation.
• Integration with GuardDuty, CloudTrail, and VPC Flow Logs.
• Use Cases:
• Investigating unauthorized access attempts.
• Identifying root causes of security incidents.
• Strengthening security posture based on analytics.

AWS Directory Service

Type: Managed Directory Service Description: AWS Directory Service provides fully managed Microsoft Active Directory (AD) and other directory solutions in the AWS Cloud. It helps manage user authentication, access control, and directory-based policies for cloud and on-premises resources. AWS Directory Service enables seamless integration with Windows workloads, AWS services, and applications that require directory-based authentication. Key Features

1. Directory Types:

• AWS Managed Microsoft AD: Fully managed Microsoft Active Directory running on Windows Server.
• AD Connector: Proxy service for redirecting directory requests to on-premises Active Directory.
• Simple AD: Lightweight, standalone directory based on Samba for basic directory needs.

2. Seamless Integration:

• Works natively with Microsoft Windows workloads, including AWS services like WorkSpaces, Amazon RDS, and Amazon EC2.

3. Multi-Region Deployment:

• Supports automatic replication of directories across multiple AWS Regions for global availability.

4. Scalability and High Availability:

• Provides automatic scaling and replication for resilience and fault tolerance.

5. Managed Service:

• Eliminates the need for manual management, including patching, backups, and monitoring.

6. Hybrid Integration:

• Extend on-premises Active Directory to the cloud using AD Connector for hybrid workloads.

7. Compliance:

• Meets industry standards, including HIPAA, GDPR, and SOC compliance.

8. Secure Access:

• Supports multi-factor authentication (MFA) for added security.

9. Integration with AWS Services:

• Compatible with AWS applications like Amazon WorkSpaces, AWS Single Sign-On (SSO), and Amazon RDS.

Directory Types

1. AWS Managed Microsoft AD:

• Full Microsoft AD with built-in replication, high availability, and schema compatibility.
• Enables group policies and trust relationships with on-premises directories.

2. AD Connector:

• Serves as a proxy to on-premises Active Directory without storing directory information in AWS.
• Ideal for hybrid cloud environments.

3. Simple AD:

• Standalone directory based on Samba 4 for lightweight use cases.
• Suitable for small-scale applications without needing full Microsoft AD capabilities.

Use Cases

1. Windows Workloads on AWS:

• Centralized authentication for Windows applications and EC2 instances.

2. Hybrid Environments:

• Extend on-premises AD to AWS for hybrid workloads using AD Connector.

3. Single Sign-On (SSO):

• Enable SSO for AWS and third-party applications with Active Directory authentication.

4. Access Management:

• Manage user and group access to AWS resources and applications.

5. Amazon WorkSpaces and RDS Authentication:

• Use AWS Directory Service for seamless user authentication in Amazon WorkSpaces and RDS databases.

6. Policy Enforcement:

• Apply group policies to manage permissions and compliance across cloud environments.

Integration with AWS Services Integrated With:

• Amazon WorkSpaces: Directory authentication for desktop-as-a-service solutions.
• Amazon RDS: Integrates with RDS for SQL Server, PostgreSQL, and MySQL to manage database user authentication.
• AWS Single Sign-On (SSO): Centralized SSO for AWS accounts and third-party applications.
• Amazon EC2: Provides domain join capabilities for Windows EC2 instances.
• AWS IAM: Extend IAM roles and permissions with directory-based access control.
• Amazon FSx for Windows File Server: Seamless integration for file storage with directory-based access control. Not Integrated With:
• Standalone AWS services that do not rely on directory-based authentication, such as S3 or DynamoDB.

Governance and Security

1. IAM Policies:

• Control access to AWS Directory Service resources and associated AWS services.

2. Encryption:

• Directory data is encrypted at rest using AWS KMS.
• Communication is secured via SSL/TLS encryption.

3. Compliance:

• AWS Directory Service complies with HIPAA, GDPR, SOC, and other regulatory standards.

4. Audit Logs:

• Monitor directory activities using AWS CloudTrail and Amazon CloudWatch.

5. Secure Connectivity:

• Use AWS Direct Connect or VPN for secure communication with on-premises directories.

Benefits

1. Fully Managed Service:

• Eliminates the overhead of managing and maintaining directory infrastructure.

2. Scalability:

• Supports dynamic scaling to accommodate growth in user and resource demands.

3. Hybrid Flexibility:

• Seamlessly integrates with on-premises Active Directory for hybrid cloud environments.

4. Cost-Effective:

• Pay-as-you-go pricing without the need for expensive hardware or licensing.

5. Reliability:

• High availability and fault tolerance with automated backups and failover.

6. Easy Integration:

• Works natively with AWS services and Windows-based workloads.

7. Security and Compliance:

• Industry-standard security practices and regulatory compliance ensure data protection.

AWS Firewall Manager

• Type: Centralized Firewall Management
• Description: Simplifies the management of AWS WAF, Shield, and VPC security groups across multiple accounts.
• Features:
• Centralized policy enforcement.
• Integration with AWS Organizations.
• Use Cases:
• Ensuring consistent firewall rules across accounts.
• Managing DDoS protection policies.
• Compliance with security policies.

Amazon GuardDuty

• Type: Threat Detection Service
• Description: Uses machine learning to detect anomalies, unauthorized access, and potential threats.
• Features:
• Monitors VPC Flow Logs, DNS logs, and CloudTrail events.
• Automatic threat scoring.
• Use Cases:
• Identifying compromised resources.
• Detecting anomalous activity in AWS accounts.
• Enhancing cloud security monitoring.

AWS IAM Identity Center (Single Sign-On)

• Type: Centralized User Access
• Description: Provides single sign-on (SSO) to AWS accounts and business applications.
• Features:
• Integrates with existing identity providers.
• Role-based access control.
• Use Cases:
• Simplifying user access to AWS resources.
• Managing multi-account access centrally.
• Enhancing security with MFA.

AWS Identity and Access Management (IAM)

• Type: Access Control Service
• Description: Enables fine-grained access control for AWS services and resources.
• Features:
• Role-based and policy-based permissions.
• Temporary security credentials.
• Use Cases:
• Defining least-privilege access for users.
• Managing access for applications and services.
• Auditing user actions for compliance.

Amazon Inspector

• Type: Vulnerability Management
• Description: Scans EC2 instances and container images for vulnerabilities and compliance issues.
• Features:
• Automatically scans resources.
• Provides detailed remediation recommendations.
• Use Cases:
• Identifying software vulnerabilities.
• Automating security compliance checks.
• Monitoring container security.

AWS Key Management Service (KMS)

• Type: Managed Encryption Key Service
• Description: Provides encryption and key management for AWS resources and applications.
• Features:
• Integrated with most AWS services.
• Custom key management policies.
• Use Cases:
• Encrypting data at rest and in transit.
• Managing keys for compliance.
• Enabling secure data sharing between services.

Amazon Macie

• Type: Data Security and Privacy Service
• Description: Amazon Macie is a fully managed service that uses machine learning to automatically discover, classify, and protect sensitive data in Amazon S3. It helps organizations comply with privacy regulations and reduce the risk of data exposure by providing insights into where sensitive data resides and how it is being accessed. Key Features:

1. Data Discovery and Classification:

• Automatically scans S3 buckets to identify sensitive data such as Personally Identifiable Information (PII), financial data, or intellectual property.
• Uses machine learning and pattern matching for accurate detection.

2. Automated Alerts:

• Generates alerts when it detects risks such as overly permissive S3 bucket policies or unencrypted sensitive data.

3. Customizable Classification Rules:

• Define custom criteria and keywords to tailor sensitive data discovery for specific organizational needs.

4. Data Visibility:

• Provides a comprehensive inventory of S3 buckets, including details on bucket security, encryption, and shared access.

5. Integration with Security Tools:

• Integrates with AWS Security Hub, CloudWatch, and other tools for centralized monitoring and response.

6. Dashboard and Reporting:

• Intuitive dashboards provide insights into sensitive data findings and security posture, with exportable reports for auditing and compliance.

Governance and Security:

• Data Encryption: Identifies unencrypted sensitive data and recommends remediation.
• Access Control: Highlights buckets with excessive permissions or public access risks.
• Auditing and Compliance: Simplifies auditing processes for regulations like GDPR, CCPA, and HIPAA by providing visibility into sensitive data.
• Monitoring: Continuous monitoring ensures real-time awareness of changes to data security.

Use Cases:

1. Regulatory Compliance:

• Meet privacy regulations by identifying and securing sensitive customer and financial data.

2. Data Governance:

• Maintain control over sensitive data by identifying access risks and ensuring proper security measures.

3. Security Incident Response:

• Detect and respond to potential data breaches involving sensitive information.

4. Data Visibility:

• Gain insights into data security and usage across S3 buckets.

5. Risk Mitigation:

• Prevent accidental exposure of sensitive data by identifying and remediating risky configurations.

Benefits:

• Enhanced Data Security: Automates the detection and classification of sensitive data to prevent exposure.
• Simplified Compliance: Provides actionable insights to comply with regulations and avoid penalties.
• Cost Efficiency: Focuses scanning and monitoring on relevant data, reducing unnecessary costs.
• Scalability: Automatically adjusts to handle data growth without requiring manual intervention.

AWS Network Firewall

• Type: Managed Network Firewall
• Description: Provides stateful inspection, intrusion prevention, and deep packet inspection for VPCs.
• Features:
• Centralized control for traffic filtering.
• Integration with AWS Firewall Manager.
• Use Cases:
• Protecting VPCs from unauthorized traffic.
• Enforcing compliance with organizational rules.
• Securing hybrid cloud networks.

AWS Resource Access Manager (AWS RAM)

Type: Resource Sharing Service Description: AWS Resource Access Manager (AWS RAM) simplifies the process of securely sharing AWS resources across accounts within your organization or with external accounts. This eliminates the need to duplicate resources and helps centralize management while maintaining security and access controls. Key Features

1. Cross-Account Resource Sharing:

• Share resources across multiple AWS accounts within an organization or with external accounts.

2. Wide Range of Supported Resources:

• Share various AWS resources, such as VPCs, Transit Gateways, Subnets, Resource Groups, and more.

3. Integration with AWS Organizations:

• Allows seamless sharing across accounts in an AWS Organization without requiring additional permissions.

4. Fine-Grained Access Control:

• Leverages IAM policies to define who can share, accept, and manage shared resources.

5. Automated Invitations:

• Automatically send resource-sharing invitations to external AWS accounts.

6. Centralized Management:

• Provides a unified dashboard to monitor and manage shared resources.

7. Transparency and Auditing:

• Integration with AWS CloudTrail logs all resource-sharing activities for compliance and troubleshooting. Subtypes and Components

1. Resource Shares:

• Logical entities that define which resources are shared and with whom.

2. Principals:

• The accounts, organizations, or organizational units (OUs) with which resources are shared.

3. Resource Types:

• Supported resources include:
• Amazon VPC subnets
• AWS Transit Gateways
• Amazon Aurora Clusters
• License Manager configurations
• Resource Groups

4. Accepting and Managing Shares:

• Recipients must explicitly accept shared resources unless they are part of an AWS Organization. Use Cases

1. Centralized Networking:

• Share VPCs, subnets, and Transit Gateways across accounts for consistent network architecture.

2. Cost Optimization:

• Avoid duplicating resources by sharing them across accounts, reducing operational costs.

3. Streamlined Licensing:

• Share License Manager configurations across multiple accounts to maintain compliance.

4. Resource Centralization:

• Centralize resource management in a primary account while granting access to others.

5. Multi-Tenant Applications:

• Enable shared access for applications or services that span multiple AWS accounts. Governance and Security
• Access Control:
• Define sharing permissions using IAM roles and policies.
• Activity Logging:
• AWS CloudTrail logs all sharing actions for monitoring and compliance.
• Security Boundaries:
• Shared resources remain secure as the owning account retains control over permissions. Benefits

1. Simplified Resource Management:

• Centralize resource sharing to reduce administrative overhead.

2. Enhanced Security:

• Use fine-grained IAM policies to control access to shared resources.

3. Cost Efficiency:

• Share resources without the need to replicate them across accounts.

4. Scalability:

• Support for sharing resources with multiple accounts, organizations, or OUs.

5. Seamless Integration:

• Works natively with AWS Organizations, simplifying multi-account setups.

AWS Secrets Manager

• Type: Secrets Management Service
• Description: Securely stores and retrieves secrets like API keys, passwords, and database credentials.
• Features:
• Automatic secret rotation.
• Fine-grained access control with IAM.
• Use Cases:
• Securing application secrets.
• Automating secret rotation.
• Enabling centralized secret management.

AWS Security Hub

• Type: Centralized Security Management
• Description: AWS Secrets Manager securely stores, retrieves, and rotates secrets such as database credentials, API keys, and other sensitive information. It helps organizations simplify secret management, enhance security, and maintain compliance by automating secret rotation and access control. Key Features:

1. Secure Storage:

• Stores sensitive information such as credentials, keys, and tokens securely, with encryption using AWS Key Management Service (KMS).

2. Automatic Rotation:

• Automatically rotates secrets for supported services (e.g., Amazon RDS, Redshift) without disrupting applications.

3. Fine-Grained Access Control:

• Uses AWS Identity and Access Management (IAM) policies to control who can access secrets.

4. Integration with AWS Services:

• Seamlessly integrates with AWS services such as Lambda, EC2, and RDS for secure access to secrets.

5. Secret Versioning:

• Maintains multiple versions of secrets to enable smooth rollback during updates.

6. Audit and Monitoring:

• Logs all secret access and changes via AWS CloudTrail, enabling audit and compliance.

7. Custom Rotation Logic:

• Supports custom Lambda functions for secret rotation for custom or third-party applications.

8. Cross-Region Replication:

• Allows secrets to be replicated across AWS Regions for disaster recovery and multi-region access.

Governance and Security:

• Encryption: Secrets are encrypted at rest and in transit using customer-managed KMS keys.
• Access Management: Define precise permissions using IAM policies and resource-based policies.
• Auditing: Monitor and log all secret-related activities with AWS CloudTrail.
• Compliance Support: Simplifies adherence to regulatory requirements by ensuring secure and traceable secret management.

Use Cases:

1. Database Credentials Management:

• Store and rotate database credentials automatically for RDS, Redshift, and custom databases.

2. API Key Storage:

• Securely store and retrieve API keys for third-party services or internal APIs.

3. Application Secrets Management:

• Manage secrets required by serverless or containerized applications using Lambda, ECS, or Kubernetes.

4. Custom Key Rotation:

• Implement custom rotation logic for proprietary systems or third-party tools.

5. Multi-Region Applications:

• Replicate secrets across Regions for high availability and disaster recovery.

Benefits:

• Enhanced Security: Prevents hardcoding of sensitive data in application code or configuration files.
• Operational Efficiency: Automates rotation and retrieval of secrets, reducing manual effort.
• Simplified Compliance: Centralized secret management and logging simplify audits and regulatory adherence.
• Scalability: Seamlessly handles growing numbers of secrets and integrations.
• Cost-Effective: Pay only for secrets managed, without additional overhead for secret storage or rotation tools.

AWS Shield

Type: Distributed Denial of Service (DDoS) Protection Service Description: AWS Shield is a managed DDoS protection service designed to safeguard applications running on AWS from distributed denial-of-service (DDoS) attacks. AWS Shield provides two levels of protection: AWS Shield Standard, which is automatically available at no additional cost, and AWS Shield Advanced, which offers enhanced protection, detailed metrics, and 24/7 support.

Key Features

1. Two-Tier Protection:

• AWS Shield Standard:
• Automatic protection against common network and transport-layer DDoS attacks.
• Included at no additional cost with all AWS services.
• AWS Shield Advanced:
• Includes advanced protections, detailed metrics, and global threat environment insights.
• Requires subscription and includes access to the AWS DDoS Response Team (DRT).

2. Real-Time Attack Mitigation:

• Detects and mitigates DDoS attacks in real time without affecting application performance.

3. Comprehensive Coverage:

• Protects web applications, APIs, and other internet-facing workloads on AWS.

4. Integration with AWS Services:

• Native integration with Amazon CloudFront, Elastic Load Balancing (ELB), AWS Global Accelerator, and Amazon Route 53 for seamless protection.

5. Health-Based Detection:

• Monitors application health to differentiate legitimate traffic from DDoS traffic.

6. Advanced Metrics and Reporting:

• Shield Advanced provides detailed attack diagnostics, traffic trends, and customizable notifications through CloudWatch.

7. Cost Protection (Shield Advanced):

• Covers costs for scaling resources due to a DDoS attack and waives data transfer charges for attacks.

8. Global Threat Environment Dashboard (Shield Advanced):

• Offers insights into current global DDoS threat trends.

Subtypes and Components

1. AWS Shield Standard:

• Automatically enabled across all AWS accounts.
• Protects against common volumetric and state-exhaustion attacks at no cost.

2. AWS Shield Advanced:

• Subscription-based service with enhanced features, including 24/7 access to the AWS DDoS Response Team (DRT), advanced metrics, and cost protection.

3. AWS DDoS Response Team (DRT):

• Experts available 24/7 to assist with attack mitigation strategies and response.

4. Integration with AWS Firewall Services:

• Works in tandem with AWS WAF and AWS Firewall Manager for comprehensive application-layer protection.

Integration with AWS Services Integrated With:

• Amazon CloudFront: Protects applications globally with edge-based DDoS mitigation.
• Elastic Load Balancing (ALB and NLB): Defends backend workloads against DDoS attacks.
• AWS Global Accelerator: Ensures low-latency routing while mitigating attacks.
• Amazon Route 53: Shields DNS services against DDoS attacks.
• AWS WAF: Enhances application-layer protection by filtering malicious traffic. Not Integrated With:
• Direct protection for standalone Amazon EC2 instances not behind ELB, CloudFront, or Global Accelerator.
• Services like Amazon RDS, S3, or Lambda, which rely on upstream integrations for Shield benefits.

Use Cases

1. Protecting Web Applications:

• Safeguard websites and APIs from volumetric attacks, such as UDP floods or SYN floods.

2. Securing DNS Services:

• Shield Route 53 DNS services from DNS-based DDoS attacks.

3. Mitigating API Threats:

• Prevent disruption to API endpoints exposed through CloudFront or ALB.

4. Compliance and Availability:

• Ensure continuous availability and compliance with SLAs during DDoS events.

5. Reducing Cost Impact of Attacks (Shield Advanced):

• Avoid financial impact due to scaling costs during attacks.

Governance and Security

• Monitoring:
• Integrated with Amazon CloudWatch for real-time attack diagnostics and alerts.
• Access Control:
• Uses IAM policies to manage access to Shield configurations and reports.
• Auditing and Logging:
• Integrated with AWS CloudTrail for logging all Shield-related activities.
• Proactive Response:
• AWS DDoS Response Team (DRT) provides tailored guidance and support during attacks.

Benefits

1. Always-On Protection:

• Automatic coverage against common DDoS attacks with Shield Standard.

2. Enhanced Security Posture:

• Advanced protection and detailed insights with Shield Advanced.

3. Seamless Integration:

• Natively integrates with key AWS services for comprehensive defense.

4. Cost Optimization:

• Shield Advanced offers financial protection against attack-related scaling and data transfer costs.

5. Global Coverage:

• Shield leverages AWS’s global infrastructure to mitigate threats at the edge.

6. Expert Support:

• Access to AWS DDoS Response Team for expert guidance during critical events.

AWS WAF (Web Application Firewall)

Type: Application Layer Protection Description: AWS WAF is a managed web application firewall that helps protect your web applications and APIs against common web exploits and vulnerabilities, including SQL injection, cross-site scripting (XSS), and bot attacks. By defining custom rules or using AWS Managed Rules, you can filter and block malicious traffic before it reaches your application.

Key Features

1. Customizable Rules:

• Define rules to allow, block, or monitor (count) web requests based on IP addresses, HTTP headers, HTTP body, URI strings, and more.

2. AWS Managed Rules:

• Pre-configured rule sets to protect against common threats like OWASP Top 10 vulnerabilities, bots, and more.

3. Bot Control:

• Detects and mitigates bot traffic, distinguishing between legitimate bots (e.g., search engines) and malicious ones.

4. Real-Time Visibility:

• Provides logs and metrics for all web traffic, enabling detailed monitoring and analysis.

5. Rule Prioritization:

• Rules are evaluated in order of priority to determine how web requests are handled.

6. Rate-Based Rules:

• Protect against denial-of-service (DoS) attacks by limiting the rate of requests from specific IP addresses.

7. Integration with AWS Services:

• Works seamlessly with services like Amazon CloudFront, Application Load Balancer (ALB), and Amazon API Gateway.

8. Web Traffic Filtering:

• Filters traffic based on geographic location (geo-matching) or request characteristics.

9. Fine-Grained Controls:

• Provides full control over which requests are allowed or blocked, reducing the risk of overblocking.

Subtypes and Components

1. Web ACLs (Access Control Lists):

• Configurations that define how to monitor and filter traffic for a specific resource.

2. Rules and Rule Groups:

• Individual rules or grouped rulesets applied to Web ACLs for filtering requests.

3. Managed Rule Groups:

• AWS-provided or third-party rules for common security threats.

4. Rate-Based Rules:

• Automatically blocks traffic exceeding a specified request rate.

5. Logging and Metrics:

• Integrated with Amazon CloudWatch for monitoring and detailed request logs.

Integration with AWS Services Integrated With:

• Amazon CloudFront:
• Distributes web content globally while integrating with WAF to filter malicious traffic at the edge.
• Application Load Balancer (ALB):
• Applies WAF rules to incoming traffic routed to backend services.
• Amazon API Gateway:
• Protects REST and WebSocket APIs against malicious requests.
• AWS App Runner:
• Enables WAF protection for containerized web applications. Not Integrated With:
• Amazon Route 53:
• WAF does not natively integrate with Route 53 for DNS-level protection.
• AWS Elastic Beanstalk:
• While ALB within Elastic Beanstalk supports WAF indirectly, WAF is not directly integrated into Beanstalk environments.
• AWS EC2 Instances:
• Does not directly protect traffic to EC2 instances unless routed through an ALB, API Gateway, or CloudFront.

Use Cases

1. Application Layer Security:

• Protect web applications from SQL injection, XSS, and other common threats.

2. Bot Management:

• Detect and block bad bot traffic while allowing legitimate bots.

3. Geo-Blocking:

• Restrict traffic from specific geographic regions.

4. DDoS Mitigation:

• Use rate-based rules to throttle high-volume malicious requests.

5. API Protection:

• Secure APIs by filtering malicious requests or blocking unauthorized access.

6. Regulatory Compliance:

• Use WAF to meet compliance requirements for web application security.

Governance and Security

• Encryption:
• Encrypted communication between WAF and AWS services.
• Monitoring and Auditing:
• Logs all traffic data and rule matches through CloudWatch or S3 for auditing and compliance.
• IAM Access Control:
• Uses IAM policies to control who can manage WAF configurations.

Benefits

1. Customizable Protection:

• Fully customizable rules tailored to specific application needs.

2. Scalable:

• Automatically scales to handle high volumes of web traffic.

3. Seamless Integration:

• Native integration with AWS services for simplified setup and management.

4. Cost-Effective:

• Pay-as-you-go pricing ensures cost efficiency.

5. Real-Time Visibility:

• Gain actionable insights into web traffic and rule performance.

6. Global Protection:

• Combined with CloudFront, provides protection at edge locations for minimal latency.

Comparison of Security Services Service Best For AWS Artifact Accessing compliance reports. AWS Audit Manager Automating compliance audits. AWS Certificate Manager Managing SSL/TLS certificates. AWS CloudHSM Dedicated cryptographic hardware. Amazon Cognito Managing user authentication. Amazon Detective Investigating security incidents. AWS Directory Service Centralized Active Directory management. AWS Firewall Manager Centralized firewall policy enforcement. Amazon GuardDuty Threat detection using machine learning. AWS IAM Fine-grained access control for AWS resources. Amazon Inspector Identifying vulnerabilities in EC2 and containers. AWS KMS Managing encryption keys for compliance. Amazon Macie Discovering and securing sensitive data. AWS Network Firewall Stateful network traffic filtering. AWS RAM Sharing resources across AWS accounts. AWS Secrets Manager Securely storing application secrets. AWS Security Hub Centralized security findings. AWS Shield Protecting against DDoS attacks. AWS WAF Protecting web applications from common threats.

Serverless

AWS Fargate

• Type: Serverless Containers
• Description: AWS Fargate is a serverless compute engine for containers that allows users to run and manage containerized applications without provisioning or managing servers. It integrates with Amazon Elastic Container Service (ECS) and Elastic Kubernetes Service (EKS) to provide a seamless and scalable container orchestration experience.

Key Features:

1. Serverless Container Management:

• Removes the need to provision and manage EC2 instances.
• Automatically scales resources to match the application's demand.

2. Resource Isolation:

• Provides task-level isolation for enhanced security by running each task or pod in its own runtime environment.

3. Simplified Networking:

• Supports AWS VPC networking to give containers dedicated ENIs, security groups, and private IPs.

4. Seamless Integration:

• Works with ECS and EKS to manage container deployments.
• Supports integration with AWS services like CloudWatch, Secrets Manager, and IAM.

5. Flexible Resource Configuration:

• Allows specifying CPU and memory requirements for each container, enabling fine-grained resource allocation.

6. Cost Efficiency:

• Pay only for the resources used by running tasks or pods, with billing based on vCPU and memory usage.

Governance and Security:

• IAM Integration:
• Assign IAM roles to tasks for secure access to AWS resources.
• Data Encryption:
• Support for encrypted EFS volumes and secure data transmission.
• Compliance:
• Fargate complies with various industry standards, including GDPR, HIPAA, and PCI DSS.
• Monitoring and Logging:
• Integrates with CloudWatch for detailed performance metrics and logs.

Use Cases:

1. Microservices Architectures:

• Deploy containerized microservices without managing infrastructure.

2. Event-Driven Applications:

• Run serverless workloads triggered by events using Fargate and AWS Lambda.

3. Batch Processing:

• Scale up containers for parallel processing of data-heavy workloads.

4. CI/CD Pipelines:

• Use Fargate in pipelines to build, test, and deploy containerized applications.

5. Hybrid Cloud Applications:

• Combine Fargate with ECS Anywhere for seamless hybrid container orchestration.

6. Machine Learning Workflows:

• Run pre-processing tasks or lightweight machine learning inference tasks in containers.

Benefits:

• Ease of Use: Eliminates the need to manage infrastructure, enabling focus on application development.
• Scalability: Automatically adjusts compute capacity to meet workload demands.
• Cost Efficiency: Pay-as-you-go pricing model ensures no cost for idle resources.
• Enhanced Security: Task-level isolation improves application security.
• Flexibility: Works seamlessly with multiple container orchestration platforms (ECS and EKS).

AWS Lambda

• Type: Event-Driven Compute
• Description: AWS Lambda is a serverless compute service that allows you to run code in response to events without provisioning or managing servers. Lambda automatically scales to handle incoming requests and executes code only when triggered, ensuring cost-effectiveness. You are billed based on the compute time consumed, with no charge when your code is not running.
• Use Cases:
• Event-driven processing.
• Serverless APIs.
• Background jobs. Integration with AWS Services Integrated With:
• Amazon S3: Trigger functions on object uploads, deletions, or modifications.
• Amazon DynamoDB: Invoke Lambda functions on table updates or new records.
• Amazon API Gateway: Build serverless APIs by invoking Lambda functions for HTTP requests.
• Amazon Kinesis: Process streaming data in real-time.
• Amazon SNS and SQS: Use Lambda for asynchronous messaging and event processing.
• AWS Step Functions: Orchestrate workflows using Lambda functions.
• Amazon EventBridge: Schedule or route events to Lambda for handling. Not Directly Integrated With:
• Standalone EC2 instances or RDS, though these can send events to Lambda indirectly via triggers like CloudWatch Events or SNS.

Storage Services Cheat Sheet

Amazon S3 (Simple Storage Service)

• Type: Object Storage
• Description: Scalable, secure, and durable storage for any data type.
• Storage Classes:
• S3 Standard: General-purpose storage for frequently accessed data.
• Use Cases: Content delivery, mobile and gaming apps, data lakes.
• S3 Intelligent-Tiering: Automatically moves data between tiers based on access patterns.
• Use Cases: Cost-optimized storage for unknown or changing access patterns.
• S3 Standard-IA (Infrequent Access): Lower cost for less frequently accessed data.
• Use Cases: Backups, disaster recovery.
• S3 One Zone-IA: Like Standard-IA but stored in a single Availability Zone.
• Use Cases: Secondary backups, easily recreatable data.
• S3 Glacier Instant Retrieval: Archival storage with millisecond access.
• Use Cases: Frequently accessed archive data.
• S3 Glacier Flexible Retrieval: Archive storage with access times of minutes to hours.
• Use Cases: Data requiring occasional access.
• S3 Glacier Deep Archive: Lowest-cost storage with access times of 12+ hours.
• Use Cases: Long-term compliance archives, regulatory requirements.
• Use Cases:
• Storing application data and backups.
• Hosting static websites.
• Creating data lakes for analytics.

Amazon EBS (Elastic Block Store)

• Type: Block Storage
• Description: Persistent block-level storage for EC2 instances.
• Volume Types:
• General Purpose SSD (gp3, gp2):
• Use Cases: Boot volumes, general-purpose workloads.
• Provisioned IOPS SSD (io2, io1):
• Use Cases: Mission-critical applications like databases.
• Throughput Optimized HDD (st1):
• Use Cases: Big data, data warehouses.
• Cold HDD (sc1):
• Use Cases: Infrequent access, lowest-cost storage.
• Features:
• Snapshots for data backup and recovery.
• Encryption at rest and in transit.
• Use Cases:
• Persistent storage for EC2.
• High-performance transactional databases.

Amazon EFS (Elastic File System)

• Type: Managed File Storage
• Description: Fully managed NFS file system for shared access across multiple EC2 instances.
• Performance Modes:
• General Purpose:
• Use Cases: Low-latency requirements like web servers.
• Max I/O:
• Use Cases: High-throughput workloads like media processing.
• Throughput Modes:
• Bursting Throughput: Automatically scales based on demand.
• Provisioned Throughput: For consistent high throughput.
• Use Cases:
• Shared file systems for containerized applications.
• Data analysis, machine learning, and media processing.

Amazon FSx

• Type: Specialized File Storage
• Variants:
• FSx for Windows File Server:
• Description: Fully managed Windows file system.
• Use Cases: Windows applications, Active Directory integrations.
• FSx for Lustre:
• Description: High-performance file storage optimized for HPC workloads.
• Use Cases: Machine learning, big data analytics, HPC.
• FSx for NetApp ONTAP:
• Description: Enterprise-grade file storage with ONTAP features.
• Use Cases: File sharing, hybrid cloud deployments.
• FSx for OpenZFS:
• Description: Open-source file system for high-speed performance.
• Use Cases: DevOps environments, CI/CD pipelines.
• Use Cases:
• High-performance workloads.
• Application-specific file systems.

Amazon S3 Glacier

• Type: Archival Storage
• Variants:
• Glacier Instant Retrieval:
• Description: Archive data with millisecond access.
• Use Cases: Frequently accessed archival workloads.
• Glacier Flexible Retrieval:
• Description: Cost-effective storage with access in minutes to hours.
• Use Cases: Regulatory data, backups.
• Glacier Deep Archive:
• Description: Lowest-cost archival storage.
• Use Cases: Long-term storage for infrequently accessed data.
• Use Cases:
• Regulatory compliance.
• Long-term data retention.
• Disaster recovery archives.

AWS Storage Gateway

• Type: Hybrid Cloud Storage
• Description: Connects on-premises environments to AWS, enabling seamless integration of cloud storage with on-premises workloads.
• Gateway Types:
• File Gateway:
• Use Cases: Storing file-based workloads in S3.
• Volume Gateway:
• Use Cases: Backing up on-premises applications with snapshots in S3.
• Tape Gateway:
• Use Cases: Replacing physical tape backups with virtual tapes in AWS.
• Use Cases:
• Hybrid cloud storage.
• Archiving and disaster recovery.
• Extending on-premises storage to AWS.

AWS Backup

• Type: Centralized Backup Service
• Description: Simplifies the process of backing up data across AWS services and on-premises systems.
• Features:
• Centralized backup policies.
• Automated backup scheduling.
• Cross-region and cross-account backups.
• Use Cases:
• Disaster recovery planning.
• Compliance with data retention policies.
• Protecting databases, EFS, and storage volumes.

Storage Use Cases Comparison

Service Best For Amazon S3 Storing unstructured data, static websites, backups. Amazon EBS Persistent, low-latency storage for EC2. Amazon EFS Shared file systems for multiple instances. Amazon FSx Specialized file systems like Windows, Lustre. Amazon Glacier Archiving infrequently accessed data. Storage Gateway Bridging on-premises workloads to the cloud. AWS Backup Centralized backup for AWS services.

Full Notes in Details with Photos Below

🧔 Accounts - Identity and Access Management (IAM) Basics

Account and IAM Basics

💡 AWS Account: Container for identities (users) and resources.

Account root user has full control over all of the AWS account and any resources created within in. The root user can’t be restricted.

IAM User Groups and Roles can also be created and given full or limited permissions. All identities start with no permissions.

May be good practice to create multiple AWS Accounts for different uses (prod, dev, test).

• Every AWS account comes with its own running copy of IAM, which is a database.
• IAM is a globally resilient service, so any data is always secure across all AWS regions. (Exam Q)
• The IAM in each of your accounts is your own dedicated instance of IAM, separate from other accounts.
• IAM
  • User
    • IDs which represent humans or apps that need access to your account
  • Group
    • Collection of related users, e.g. dev team, finance or HR
  • Role
    • Can be used by AWS Services, or for granting external access to your account
    • Used when the number of things is uncertain.
• IAM Policy
  • Allow or deny access to AWS services when and only when they’re attached to IAM users, groups or roles.
• IAM three main jobs
  • Manage identities - An ID Provider (IDP)
    • Create, modify and delete IDs such as users and roles
  • Authenticate identities
    • Prove you are who you claim to be - generally username and passwords
  • Authorize
    • Allow or deny access to resources
• IAM is provided for free
  • No cost for users, groups and roles. Limits for number of each.
• IAM is global service.
• Allow or deny its ids on its AWS account
• No direct control on external accounts or users - only control local ids in your account
• Identity federation and MFA
  • Use Facebook, Twitter, Google etc. to access AWS resources

IAM Access Keys

• Long-term credentials
  • Don’t update manually
• IAM User don’t need username and password - for CLI access key is enough
• IAM User can have up to two access keys
  • Can be created, deleted, made inactive or made active

Access Keys consist of two parts

• Both are provided when created an access key
• These are only provided once - no ability to get access to the keys again. Need to be stored safely.
• Both parts are used when accessing AWS via CLI
• Access keys need to be deleted and recreated if they are leaked
• Possible to have two sets of keys such that you can create a new one, update all applications using the keys and then delete the old set

Access Key ID: ABABABABABABABA

Secret Access Key: oierWRhoefWORIOF/DFLWAnljef

☁️ Cloud Computing Fundamentals

Essential Characteristics of Cloud Computing

💡 On demand self-service: A consumer can unilaterally provision computing capabilities, such as server time and network storage, as needed automatically without requiring human interaction with each service provider. Can provision capabilities as needed without requiring human interaction.

Provision and terminate using a UI/CLI without human interaction.

Broad network access: Capabilities are available over the network and accessed through standard mechanisms that promote use by heterogeneous thin or thick client platforms (e.g., mobile phones, tablets, laptops, and workstations). Capabilities are available over the network and accessed through standard mechanisms.

Access services over any networks, on any devices, using standard protocols and methods.

Resource pooling: The provider’s computing resources are pooled to serve multiple consumers using a multi-tenant model, with different physical and virtual resources dynamically assigned and reassigned according to consumer demand. There is a sense of location independence in that the customer generally has no control or knowledge over the exact location of the provided resources but may be able to specify location at a higher level of abstraction (e.g., country, state, or datacenter). Examples of resources include storage, processing, memory, and network bandwidth. There is a sense of location independence… no control or knowledge over the exact location of the resources. Resources are pooled to serve multiple consumers using a multi-tenant model.

Economies of scale, cheaper service.

Rapid elasticity: Capabilities can be elastically provisioned and released, in some cases automatically, to scale rapidly outward and inward commensurate with demand. To the consumer, the capabilities available for provisioning often appear to be unlimited and can be appropriated in any quantity at any time. Capabilities can be elastically provisioned and released to scale rapidly outward and inward with demand. To the consumers, the capabilities available for provisioning ofter appear to be unlimited.

Scale UP (OUT) and DOWN (IN) automatically in response to system load.

Measured service: Cloud systems automatically control and optimize resource use by leveraging a metering capability at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency for both the provider and consumer of the utilized service. Resource usage can be monitored, controlled, reported and BILLED.

Usage is measured. Pay for what you consume.

Public vs Private vs Hybrid vs Multi Cloud

💡 Public cloud: AWS, Azure, Google. Meet the essential characteristics of cloud computing.

Multi-cloud: Using more than one of the public cloud platforms.

Private cloud: Run on business premises. AWS Outpost, Azure Stack, Anthos.

Hybrid cloud: Using private cloud and public cloud in cooperation as a single environment.

Cloud Service Models

X as a Service

Infrastructure Stack

• Application
• Data
• Runtime
• Container
• OS
• Virtualization
• Servers
• Facilities

Parts you manage, parts managed by the vendor.

Unit of consumption is what makes each service model different - application vs OS

XaaS Services

On-Premises

• Application
• Data
• Runtime
• Container
• OS
• Virtualization
• Servers
• Infrastructure
• Facilities

DC Hosted

• Application
• Data
• Runtime
• Container
• OS
• Virtualization
• Servers
• Infrastructure
• Facilities

Data centre

IaaS

• Application
• Data
• Runtime
• Container
• OS
• Virtualization
• Servers
• Infrastructure
• Facilities

EC2 uses the IaaS service model

PaaS

• Application
• Data
• Runtime
• Container
• OS
• Virtualization
• Servers
• Infrastructure
• Facilities

Heroku is a PaaS

SaaS

• Application
• Data
• Runtime
• Container
• OS
• Virtualization
• Servers
• Infrastructure
• Facilities

Netflix, Dropbox, Office 365 etc.

++ Faas, CaaS, DBaaS

🗣YAML - YAML Ain't Markup Language

Human readable data serialization language. A YAML document is an unordered collection of key:value pairs, each key has a value. YAML support strings, integers, floats, booleans, lists, dictionary.

cats: ["ben", "bin", "ban"]
# Same list can also be represented as below. Indentation matters.
cats:
	- "ben"
	- "bin" 
	- ban # values can be enclosed in "", '' or not - all valid but enclosing can be more precise

cats:
	- name: ben
		color: [black, white]
	- name: bin
		color: "mixed"
	- name: ban
		color: "white"
		numofeyes: 1

Resources:
	s3bucket:
		Type: "AWS::S3::Bucket"
		Properties:
			BucketName: "1337"

🐣 AWS Fundamentals

Public vs Private Services

AWS Global Infrastructure

AWS Regions + AWS Edge Locations

Geographic Separation: Isolated Fault Domain

Geopolitical Separation: Different governance

Location Control: Performance

Region Code: us-east-1

Region Name: N. Virginia

Availability Zone (AZ): Level of granularity below regions. Isolated infrastructure within a region.

us-east-1a, us-east-1b, …, us-east-1f

Service Resilience:

• Globally resilient: IAM and Route 53. Can tolerate failure of multiple regions without affecting service.
• Region resilient: If an AZ in a region fails, the service can continue operating. If all AZ fails, the service fails.
• AZ resilient

Virtual Private Cloud (VPC) Basics

💡 A virtual network inside AWS

A VPC is within 1 account & 1 region ❗

Private and isolated unless you decide otherwise

Two types: Default VPC and Custom VPCs

Custom VPCs used in almost all AWS deployments. More later.

VPCs are created within a region. VPCs cannot communicate outside their network unless you specifically allow it. **By default a VPC is entirely private.**❗

VPC CIDER (Classless Inter-Domain Routing): Every VPC is allocated a range of IP addresses. If you allow anything to communicate to a VPC, it needs to communicate to that VPC CIDR. Any outgoing connection is going to originate from that VPC CIDR. Custom VPCs can have multiple CIDR ranges, but the default VPC only gets one, which is always the same. ❗Default VPC IP range: 172.31.0.0/16 ❗

****Each subnet within a VPC is located within a AZ, and can never be changed. Default VPC is configured to have a subnet in every AZ. Each use a part of the IP range and cannot overlap. This is how a VPC is resilient.

Default VPC Basic

• One per region - can be removed and recreated
• Default VPC CIDR is always 172.31.0.0/16 ❗
• /20 subnet in each AZ in the region
• The higher the /number is the smaller the range. /17 is half the size of /16.
• IGW: Internet Gateway
  • VPC
• SG: Security Group
  • (EC2) Instances
  • Stateful
    • Incoming rule change = allow outgoing response traffic
      • Open port 80 for incoming will allow port 80 for outgoing response
  • Allow rules only
  • Instances can have multiple SGs
  • Allow CIDR, IP, SG as destination
• NACL: Network Access Control List
  • Subnet
  • Stateless
    • Open rule 80 for incoming does not allow port 80 for outgoing
  • Allow and deny rules
  • Subnets can have only one NACL
  • Only allow CIDR as destination
• Subnets assign public IPv4 addresses
• Best practice not to use default VPC

Source: https://medium.com/awesome-cloud/aws-difference-between-security-groups-and-network-acls-adc632ea29ae

Elastic Compute Cloud (EC2) Basics

EC2 is AWS’s implement of IaaS - Infrastructure as a Service Default compute service within AWS. Provide access to VMs known as instances

EC2 Key Facts & Features

• IaaS - Provides Virtual Machines (Instances)
• Private service by default - uses VPC networking
• AZ resilient - Instance fails if AZ fails
• Different instance sizes and capabilities
• On-Demand Billing - Per second
• Local on-host storage or Elastic Block Store (EBS)
• Instance composition: CPU, memory, disk and networking. All four are billed when running. ❗
  • Only disk storage is billed when stopped (EBS).

Instance Lifecycle

• Running
• Stopped
• Terminated

Amazon Machine Image (AMI)

• An EC2 instance can be created from an AMI, or an EC2 can be used to create an AMI
• Contains
  • Permissions - who can and can’t use the AMI
    • Public - Everyone can launch instances from that AMI (Linux and Windows)
    • Owner - Implicit allow
    • Explicit - specific AWS accounts allowed
  • Boot Volume
    • The drive that boots the OS
  • Block Device Mapping
    • Links the volumes the AMI have
    • Mapping between volumes

Connecting to EC2

• EC2 instances can run different OS’s
• Windows: RDP - Remote Desktop Protocol
  • Protocol Port 3389
• Linux: SSH protocol
  • Port 22

Simple Storage Service (S3) Basics

• Global Storage Platform - regional based/resilient
  • Data is replicated across AZs in that region
• Public service, unlimited data & multi-user
• Movies, audio, photos, text, large data sets
• Economical & accessed via UI/CLI/API/HTTP
• Should be your default storing point
• Objects & Buckets
  • Objects is the data you store
  • Buckets are container for objects

S3 Objects

• A file made up of two parts: key and value
  • E.g koala.jpg : koala-image
  • Value is the content being stored
• 0 - 5 TB data
• Version ID
• Metadata
• Access Control
• Subresources

S3 Buckets

• Never leaves a region unless you configure it to do so
• A bucket is identified by its bucket name, which must be globally unique
  • Often AWS stuff is only unique within an account or region - bucket is exception to this
• Unlimited Objects
• Flat Structure - all objects are stored at root level in the bucket
  • Folders are prefixed names - but objects are still stored at the same level

Summary

• Bucket names are globally unique
• 3-63 characters, all lower case, no underscores
• Start with a lowercase letter or a number
• Can’t be IP formatted e.g. 1.1.1.1
• Buckets - 100 soft limit, 1000 hard per account
• Unlimited objects in bucket, 0 bytes to 5TB
• Key = Name, Value = Data
• ARN: Amazon Resource Name

S3 Patterns and Anti-Patterns

• S3 is an object store - not file or block
• S3 has no file system - it is flat
• You can’t mount an S3 bucket as (K:\ or /images)
• Great for large scale data storage, distribution or upload
• Great for “offload”
• INPUT and/or OUTPUT to MANY AWS products

CloudFormation Basics

CloudFormation is a Infrastructure as Code (IaC) product in AWS which allows automation infrastructure creation, update and deletion Templates created in YAML or JSON Templates used to create stacks, which are used to interact with resources in an AWS account

YAML

AWSTemplateFormatVersion: "version date"

Description: # Must directly follow AWSTemplateFormatVersion if defined
	String

Metadata: # Control the UI
	template metadata

Parameters: # Add fields that prompt the user for more information
	set of parameters

Mappings: # Key/Value pairs which can be used for lookups
	set of mappings

Conditions: # Allow decision making. Create Condition / Use Condition. 
	set of conditions

Transform: 
	set of transforms

Resources: 
	set of resources

Outputs: # Outputs from the template being applied
	set of outputs

Template

• All those other things

• Resources

  Resources:
  	Instance:
  		Type: 'AWS::EC2::Instance' # Logical Resource
  		Properties:
  			ImageId: !Ref LatestAmiId
  			InstanceType: !Ref InstanceType
  			KeyName: !Ref KeyName
  
  

Stack

• A living representation of a template
• Class/Instance ~ Template/Stack
• Physical Resource is the actual EC2 instance
• Create, Update or Delete Stack

CloudWatch Basics

Core supporting service within AWS which provides metric, log and event management services. Used through other AWS services for health and performance monitoring, log management and nerveless architectures

• Collects and manages operational data
• Metrics - AWS Products, Apps, on-premises
• CloudWatch Agent to monitor outside AWS
  • Also to monitor certain things within certain products requires the CW Agent
• UI, API, CLI
• CloudWatch Logs - AWS Products, Apps, on-premises
  • Same as above for CW Agent
• CloudWatch Events - AWS Services & Schedules

Namespace

• Can think of as a container - separate things into different areas
  • Reserved: AWS/service → AWS/EC2

Metric

• Collection of Time Ordered Set of Data points
• CPU Usage, Network I/O, Disk I/O

Datapoint

• CPU Utilization Metric
• Consist of two things in its simplest form:
  • Timestamp: 2019-12-03T08:45:45Z
  • Value: 98.3 (% CPU utilization)

Dimension

• Dimensions separate datapoints for different things or perspectives within the same metric
• Use dimensions to look at the metric for a specific InstanceId

Alarm

• Linked to a specific metric
• Can set criteria for an alarm to move into an alarm state and further define an SNS or action
  • Billing alarm is an example of this
• Three states: OK, ALARM, INSUFFICIENT DATA

Shared Responsibility Model

The Shared Responsibility Model - is how AWS provide clarity around which areas of systems security are theirs, and which are owned by the customer.

• AWS responsible for the security of the cloud
  • Hardware/AWS Global Infrastructure
  • Regions, AZ, Edge Locations
  • Compute, Storage, Database, Networking
  • Software
• Customer responsible for security in the cloud
  • Client-side data encryption, integrity & authentication
  • Server-side encryption (File system and/or data)
  • Networking traffic protection (encryption, integrity, identity)
  • OS, Network and Firewall configuration
  • Platform, applications, identity and access management
  • Customer Data

High-Availability (HA) vs Fault-Tolerance (FT) vs Disaster Recovery (DR)

High-Availability (HA)

Aims to ensure an agreed level of operational performance, usually uptime, for a higher than normal period. Maximizing a system’s uptime / minimize outages.

• E.g.
  • 99.9% = 8.77 hours /year downtime
  • 99.999% = 5.26 minutes /year downtime
• User disruption, such as re-login, is okay
• If a server goes down, but another is ready on standby, users may notice small disruptions, but thats okay
• Often require redundant service or architecture to achieve the agreed SL

Fault-Tolerance (FT)

Is the property that enables a system to continue operating properly in the event of the failure of some (one or more faults within) of its components. Operate through faults.

• High availability is not enough
• If a server goes down, disruption is not okay
• The system must be able to tolerate the failure
  • Levels of redundancy and system of components which can route around failures
• Implementing FT when you need HA is expensive and is harder to implement
• Implementing HA when you need FT can be a disaster

Disaster Recovery (DR)

A set of policies, tools and procedures to enable the recovery or continuation of vital technology infrastructure and system following a natural or human-induced disaster. Used when FT and HA don’t work

• Parachute

Domain Name System (DNS) Basics

DNS 101

• DNS is a discovery service
• Distributed database
• Translates machine into human and vice-versa
• amazon.com → 104.98.34.131
• It’s huge and has to be distributed
• Zone files that can be queried

❗Remember these ❗

• DNS Client: Your laptop, phone, tablet, PC, etc.
• Resolver: Software on your device, or a server which queries DNS on your behalf
• Zone: A part of the DNS database (e.g. amazon.com)
• Zonefile: Physical database for a zone
• Nameserver: Where zonefiles are hosted

DNS Root

• Starting point of DNS
• www.amazon.com
  • Read right to left
• Hosted on 13 Root servers
  • Operated by 12 different large companies and organization
  • Only operates the servers, not the database itself
  • Each root server can be a cluster of servers
• Root Hints
  • Provided by Vendor
  • List of these root servers, pointer to DNS root servers
• Root Zone is operated by IANA - Internet Assigned Numbers Authority

DNS Hierarchy

• Root zone - Database of top level domains | IANA
  • .com, .org, .uk, etc.
• .com zone | Verisign
  • amazon.com
  • NS - w.x.y.z
• amazon.com zone
  • www ⇒ 104.98.34.131

Registry

• Organization that maintains the zones for a TLD

Registrar

• Organization with relationship with .org TLD zone manager allowing domain registration

DNS Resolution

❗Remember these❗

• Root hints: Config points at the root servers IPs and addresses
• Root Server: Hosts the DNS root zone
• Root zone: Point at TLD authoritative servers
• gTLD: generic Top Level Domain (.com .org etc)
• ccTLD: country-code Top Level Domain (.uk, .eu, etc)

Route53 Fundamentals

R53 Basics

• Register domains
• Host Zones … managed nameservers
• Global servers … single database
• Globally Resilient

Register domains

• Registries
  • .com .io .net
• Create a zonefile
  • animals4life.org
• Put zonefile to four nameservers

Hosted Zones

• Zone files in AWS
• Hosted on four managed name servers
• Can be public
• Or private … linked to VPC(s)
• Stores records (recordsets)

DNS Record Types

Nameserver (NS)

• Record types that allow delegation to occur in DNS
• .com zone
  • Multiple nameserver records inside it for amazon.com
    • Point at servers managed by the amazon.com team

A and AAAA Records

• Map host names to IP
• A: www → ipv4
• AAAA: → ipv6

CNAME Records

• Host to host
• ftp, mail, www (references) → A server
• Cannot point directly at an IP address, only other names

MX Records

• Important for email
• MX records are used as part of the process of sending email
• E.g. inside google.com zone
  • MX 10 mail
    • means mail.google.com
  • MX 20 mail.other.domain.
    • Fully qualified domain name
    • means mail.other.domain
  • Lower values for the priority field means higher priority
  • MX 20 is only used if MX 10 doesn’t work

TXT Records

• Allow you to add arbitrary text to a domain
• E.g. animals4life.org zone
  • Add: TXT cats are the best
  • Important to prove that you own domain (animals4life.com)

TTL - Time To Live

• TTL 3600 (seconds)
  • Value configured by amazon.com admin
  • Results of query stored at the resolver server for 1 hour
  • Authoritative: Query results directly from amazon.com server
  • Non-authoritative: If another client queries the resolver within 3600 seconds, the resolver can immediately return the results of the query
  

🦠 IAM, Account and AWS Organizations

IAM Identity Policies

• Users, groups and roles
• Grants access or denies access

IAM Policy Document

• At high level just one or more statements that grant or deny access
• Need to identify
• Statement only applies if the interaction with AWS match the action and the resource
• Wildcards (*) match any action
• Effect defines what to do if the action and resource match
• Often statements overlap, and you may be allowed and denied at the same time.
• Explicit denies are first priority. Deny always win.
• Priority list
  1. Explicit DENY
  2. Explicit ALLOW
  3. Default DENY

{
	"Version": "2012-10-17",
	"Statement": [
		{
			"Sid": "Fullaccess", # StatementID
			"Effect": "Allow",
			"Action": ["s3:*"],
			"Resource": ["*"],
		},
		{
			"Sid": "DenyCatBucket",
			"Effect": "Deny",
			"Action": ["s3:*"],
			"Resource": ["arn:aws:s3:::catgifs", "arn:aws:s3:::catgifs/*"],	
		}
	]
}

Inline Policy

• Write a JSON for multiple users individually
  • Bad practice for many users - have to change a lot of JSONs if there are 100 users
• Only use in special or exceptional allow or deny situations

Managed Policy

• Reusable
• Low management overhead
• Should be the default

IAM Users and ARNs

IAM Users are an identity used for anything requiring long-term AWS access e.g. humans, applications or service accounts

• Principal: Something or someone wanting access resources in AWS
  • Must authenticate to gain access
    • Access Keys
    • Username/password
• When a principal is authenticated, it is known as a authenticated identity
• When the authenticated user tries to do an action, e.g. upload something to a S3 bucket, IAM checks that the authenticated user have access to perform that action (authorization)

Amazon Resource Name (ARN)

Uniquely identify resources within any AWS accounts

aws:partition:service:region:account-id:resource-id
aws:partition:service:region:account-id:resource-type
aws:partition:service:region:account-id:resource-type:resource-id

arn:aws:s3:::catgifs # Bucket
arn:aws:s3:::catgifs/* # Objects in bucket

# These two don't overlap. First is access to manage the bucket, second is to manage objects in bucket. 

• 5000 IAM Users per account
• IAM User can be a member of 10 groups
• This has systems design impacts
  • Internet-scale applications
  • Large orgs and org merges
• IAM Roles and Identity Federation fix this (more later)

IAM Groups

IAM Groups are containers for Users

• Allow for easier management
• Groups can have (identity) policies attached to them
  • Users can have individual (identity) policies too
• Trick question exam: “All users” group does not exist natively (but you can technically create it)
• ❗300 groups ❗
• ❗10 groups per user ❗
• No nesting
• Resource policies (e.g. for a bucket) can allow one or more specific user to allow access
• Resource policies cannot grant access to a group!
  • Further, cannot be referenced from a resource policy at all

IAM Roles

*An IAM role is an IAM identity that you can create in your account that has specific permissions. An IAM role is similar to an IAM user, in that it is an AWS identity with permission policies that determine what the identity can and cannot do in AWS. However, instead of being uniquely associated with one person, a role is intended to be assumable by anyone who needs it. Also, a role does not have standard long-term credentials such as a password or access keys associated with it. Instead, when you assume a role, it provides you with temporary security credentials for your role session.

-* https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html

• Role best suited for unknown number of principals or more than 5000 users
• IAM Roles are assumed. You become that role.
• ❗**Two types of policy for a role:**❗
  • Trust policy
  • Permissions policy
• If a role is assumed by something that is allowed to assume it, temporary security credentials are created.
  • ❗STS: Secure Token Service ❗
    • Generates the security tokens
    • sts:AssumeRole
  • Permissions policy define what they have access to
  • When they expire the role has to be assumed again to regain access

❓When to use IAM Roles

• Most common use case is for other AWS services
• E.g. AWS Lambda
  • No permissions by default
  • Lambda Execution Role
  • Runtime environment assumes the role.
  • Better to use a role than to hardcode access keys to the Lambda function
• Emergency or unusual situations
• E.g. team with read-only access:
  • 99% read-only access is OK
  • “Break glass for key”
  • User of team can assume an emergency role to perform a certain write action
• A corporation with > 5000 ids
  • ID federation
  • Can allow an organization to use previous existing accounts for SSO (Active Directory)
  • AD users are allowed to assume a role to gain access to e.g. a bucket
• App with millions of users
  • Web Identity Federation
  • Users might need to interact with a DynamoDB
  • Users are allowed to assume a role to interact with the db
  • No AWS credentials on the app
  • Uses existing customer logins (twitter, fb, google)
  • Scales to large number of accounts
• Cross AWS accounts

Service-linked Roles & PassRole

A service-linked role is a unique type of IAM role that is linked directly to an AWS service. Service-linked roles are predefined by the service and include all the permissions that the service requires to call other AWS services on your behalf. The linked service also defines how you create, modify, and delete a service-linked role. A service might automatically create or delete the role. It might allow you to create, modify, or delete the role as part of a wizard or process in the service. Or it might require that you use IAM to create or delete the role.

• IAM role linked to a specific AWS service
• Predefined by a service
• Providing permissions that a service needs to interact with other AWS services on your behalf
• Or allow you to during the setup or within IAM
• You can’t delete the role until it’s no longer required
• PassRole: Grant a user permission to pass a role to an AWS service
  • Bad: Bob may create and assign a role to a AWS service that has permissions that exceeds the permissions that Bob has himself
    • E.g. create resources
  • Good: Bob cannot assign roles with permissions that exceeds his own

AWS Organizations

Suitable for organization with multiple AWS accounts

• Use a standard AWS account to create a AWS organization
  • This account will be the management account or master account - can only be one
  • The organization is not within the AWS account
• Invite other standard accounts into the organization
• Organization Root is a container within AWS Organization which contains either other AWS account or other organizational units
• Consolidated billing: Member accounts pass their billing to the payment/management/master account
  • Removes financial overhead
  • Consolidation of reservation and volume discounts
• Two important concepts of AWS Organizations:
  • In a organization you can create accounts directly within the organization - one step process instead of invitation
  • Don’t need to have IAM Users inside every AWS account. IAM Roles can be used. Can role switch into different accounts.

Service Control Policies (SCP)

JSON doc with policies. Can be attached to organizations as a whole. Cascade to all orgs below that which it is attached to. Management account is special and is unaffected by SCP!

• SCPs are account permissions boundaries
• They limit what the account (including account root user) can do
• SCPs can e.g. limit the size of an EC2 instance within a specific region
• SCPs don’t grant any permissions!
• Allow list vs Deny list
  • Default is a deny list
  • FullAWSAccess Default for new account
  • DenyS3 - Deny S3 to organizations - even though they have FullAWSAccess (deny, access, deny)
• To implement allow list:
  • Remove FullAWSAccess - add a new list: AllowS3EC2
    • Explicit say which services are allowed
    • More overhead, may block access to services you don’t intend to block
• Best practice is deny list architecture

CloudWatch Logs

CloudWatch Logs is a service which can accept logging data, store it and monitor it. It is often the default place where AWS Services can output their logging too. CloudWatch Logs is a public service and can also be utilized in an on-premises environment and even from other public cloud platforms.

💡 Public Service: Usable from AWS or on-premises

• Store, Monitor and access logging data
• AWS Integrations - EC2, VPC Flow logs, Lambda, CloudTrail, R53 and more
• Metric filter: Can generate metrics based on logs
• Regional service

CloudTrail Essentials

CloudTrail Basic

• Logs API calls/activities as a CloudTrail Event
• 90 days stored by default in Event History
• Enabled by default - no cost for 90 day history. No S3.
• To customize the service, create one or more Trails
• Management events
  • Provide information about management operation that are performed on resources in your AWS account
  • AKA Control Plane Operations
  • Create EC2 instance etc
  • Enabled by default ❗
• Data events
  • Objects being uploaded to S3
  • Lambda being invoked
  • Not enabled by default. Come at an extra cost. ❗
• Trails can be set to one region or all regions
• Organizational trail - it is what it sounds like
• Trails are how you configure S3 and CWLogs.
• Management event only by default
• IAM, STS, CloudFront → Global Service Events
  • Only these logs global
• NOT REALTIME - There is a delay
  • Typical 15 minutes ❗

AWS Control Tower

*AWS Control Tower offers a straightforward way to set up and govern an AWS multi-account environment, following prescriptive best practices. AWS Control Tower orchestrates the capabilities of several other AWS services, including AWS Organizations, AWS Service Catalog, and AWS IAM Identity Center (successor to AWS Single Sign-On), to build a landing zone in less than an hour. Resources are set up and managed on your behalf.

AWS Control Tower orchestration extends the capabilities of AWS Organizations. To help keep your organizations and accounts from drift, which is divergence from best practices, AWS Control Tower applies preventive and detective controls (guardrails). For example, you can use guardrails to help ensure that security logs and necessary cross-account access permissions are created, and not altered.*

• Quick and easy setup of multi-account environment
• Orchestrates other AWS services to provide this functionality
• Organizations, IAM Identity Center, CloudFormation, Config and more
• Landing Zone - multi-account environment
• SSO/ID Federation, Centralized Logging and Auditing
• Guard Rails - Detect/Mandate rules/standard across all accounts
• Account Factory - Automates and standardizes new account creation
• Dashboard - single page oversight of the entire environment

Landing Zone

• Well Architected multi-account environment. Home region.
• Built with AWS Organizations, AWS Config, CloudFormation
• Security OU (Organizational Unit) - Log Archive and Audit Accounts (CloudTrail & Config Logs)
• Sandbox OU - Test/less rigid security
• You can create other OU’s and Accounts
• IAM Identity Center (AWS SSO) - SSO, multiple-accounts, ID Federation
• Monitoring and Notifications - CloudWatch and SNS
• End User account provisioning via Service Catalog

Guard Rails

• Guardrails are rules for multi-account governance
• Mandatory, strongly recommended or elective
• Preventive - Stop you doing things (AWS ORG SCP)
• Enforced or not enabled
  • i.e. allow or deny regions or disallow bucket policy changes
• Detective - compliance checks (AWS CONFIG Rules)
• Clear, in violation or not enabled
• Detect CloudTrail enabled or EC2 Public IPv4

Account Factory

• Automated Account Provisioning
• Cloud admins or end users (with appropriate permissions)
• Guardrails - automatically added
• Account admin given to a named user (IAM Identity Center)
• Account & network standard configuration
• Account can be closed or repurposed
• Can be fully integrated with a business SDLC (Software Development Life Cycle)

💾 Simple Storage Service S3

S3 Security

S3 is private by default

S3 Bucket Policies

• A form of resource policy ❗

• Like identity policies, but attached to a bucket

• Resource perspective permissions

• ALLOW/DENY same or different accounts

• ALLOW/DENY anonymous principals

  

Access Control Lists (ACLs)

• ACLs on objects and bucket
• A subresource
• **LEGACY!**❗
• Inflexible and simple permissions

Block Public Access

• Fail safe

Summary

• Identity: Controlling different resources
• Identity: You have a preference for IAM
• Identity: Same account
• Bucket: Just controlling S3
• Bucket: Anonymous or Cross-Account
• ACLs: Never - unless you must

S3 Static Hosting

Static Website Hosting

• Normal access is via AWS APIs
• This feature allows access via HTTP - e.g. Blogs
• Index and Error documents are set
• Website Endpoint is created
• Custom Domain via R53 - Bucket name matters!
• Offloading: Large data files such as pictures can be saved in a static S3 bucket to offload the page being accessed
• Out-of-band pages: During maintenance of a server, configure DNS to point at an error HTML page hosted at static S3.

S3 Pricing

• Per GB month charge ❗
• Every GB in is free ❗
• Every GB out of S3 is charged ❗
• GET, PUT, POST etc pricing per 1000 requests ❗

Object Versioning & MFA Delete

Once enabled, you can never disable it again! Can be suspended and reenabled. Versioning lets you store multiple versions of objects within a bucket. Operations which would modify objects generate a new version.

Almost guaranteed to feature on the exam ❗

• Without versioning each object is identified by their key
• With versioning disabled on an object, the id of the object is set to null
• If an object is requested without specifying the id, you always retrieve the latest object
• If we delete an object without specifying id, the objects is not actually deleted but it adds a delete marker.
  • Delete markers can be deleted
• To fully delete you must provide the id of the object you delete
• OBJECT VERSIONING CANNOT BE SWITCHED OFF
• Space is consumed by all versions
• You are billed for all versions

MFA Delete (Multi-Factor Authentication)

• Enabled in versioning configuration
• MFA is required to change bucket versioning state
• MFA is required to delete versions
• Serial number (MFA) + Code passed with API CALLS

S3 Performance Optimization

Single PUT Upload

• Single data stream to S3
• Stream fails - upload fails
• Requires full start
• Speed & reliability = limit of 1 stream
• Any upload to to 5 GB

Multipart Upload

• Data is broken up
• Min data size 100 MB
• 10 000 max parts, 5MB → 5GB ❗
• Parts can fail, and be restarted
• Transfer rate = speed of all parts

S3 Accelerated Transfer (Off)

• Uses the network of edge locations
• Default turned off
• Some restrictions to enable it
• Transfers data via the AWS network - more efficient than public internet
• Lower, consistent latency
• The worse the initial connection, the bigger the gain of uses accelerated transfer

Key Management Service (KMS)

Regional & Public Service Create, Store and Manage Keys Symmetric and Asymmetric Keys Cryptographic operations (encrypt, decrypt &…) Keys never leave KMS - Provides FIPS 140-2 (L2)

KMS Keys

• Consider it a container
• Logical - ID, date, policy, desc & state
• … backed by physical key material
• Generated or imported
• KMS Keys can be used for up to 4KB of data
• Everything on disk is encrypted, never in plaintext form ❗
  • May be in plaintext in memory ❗

KMS and KMS Keys

CMK - Customer Managed Keys

Data Encryption Keys (DEKs)

• GenerateDataKey - works on > 4KB

1. Plaintext Version → Lock (Encrypt data)
2. Ciphertext Version → Unlock (Decrypt data)
3. Encrypt data using plaintext key
4. Discard plaintext version
5. Store encrypted key with data

Key Concepts

• KMS Keys are isolated to a region and lever leave
• Multi-region keys exist
• AWS Owned & Customer Owned
• Customer Owned: AWS Managed og Customer Managed KEYS
• Customer Managed keys are more configurable
• KMS Keys support rotation
• Backing Key (and previous backing keys)
• Aliases

Key Policies and Security

• Key Policies (Resource)
• Every KEY has one
• Key Policies + IAM Policies
• Key Policies + Grants

{
	"Sid": "Enable IAM User Permissions",
	"Effect": "Allow",
	"Principal": {"AWS": "arn:aws:iam:1122334455:root"},
	"Action": "kms:*",
	"Recource": "*"
}

{
	"Version": "2012-10-17",
	"Statement": {
		"Effect":"Allow",
		"Action": [
			"kms:Encrypt",
			"kms:Decrypt"
		]
		"Resource": [
			"arn:aws:kms:*:1122334455:key/*"
		]
	}
}

S3 Encryption

Buckets aren’t encrypted. Objects are!

• Client-Side Encryption
  • Encrypted by client before upload
  • Keys, process, tooling
• Server-Side Encryption
  • Objects themselves aren’t encrypted. Reaches S3 in plaintext, and is then encrypted.

Server-Side Encryption (SSE-C , SSE-S3 and SSE-KMS)

• Server-Side Encryption with Customer-Provided Keys (SSE-C)
  • Customer is responsible for keys used to encrypt/decrypt
  • S3 manages the actual encryption - no CPU requirement on client, but still need to manage the actual keys ❗
  • When uploading an object, provide object and key ❗
  • Encrypted objects is stored on S3
  • To decrypt you must provide a key to decrypt and specify the object you wish to retrieve
• Server-Side Encryption with Amazon S3-Managed Keys (SSE-S3) [AES256]
  • AWS Manages encryption & keys
  • S3 creates a root key
  • Creates a key thats unique for every object
    • This key encrypts plaintext object, then root key is used to encrypt that key
    • Original unencrypted version of this key is discarded
    • ?Root key decrypts unique key, that is again used to decrypt object?
  • Cons:
    • No access to keys
    • No control over rotation of keys
    • No role separation
• Server-Side Encryption with KMS KEYS Stored in AWS KMS (SSE-KMS)

  • Root key is handled by KMS

  • The KMS key is used to generate a unique key for every object that is encrypted using SSE-KMS

  • You are not restricted to use the KMS Key provided på AWS. You can use your own customer-managed KMS key.

    • You can control permissions and rotation

  • Role separation! S3 admin with full access can’t see the unencrypted version of objects - need access to the KMS key

    

    | Method | Key Management | Encryption Processing | Extras | | | | | | | Client-Side | You | You | | | SSE-C | You | S3 | | | SSE-S3 | S3 | S3 | | | SSE-KMS | S3 & KMS | S3 | Rotation Control Role Separation |

    

    Bucket Default Encryption

    • PUT operation when uploading
    • header
      • x-amz-server-side-encryption : “AES256” eller “aws:kms”
        • How you specify to use S3 encryption
        • AES-256: SSE-S3
        • aws:kms : SSE-KMS
    • Can set a default for a bucket when you don’t specify this header
    • Can also restrict what encryption is possible on a bucket

S3 Object Storage Classes

S3 Standard

S3 Standard-IA (Infrequent Access)

Cheaper! But, retrieval fee. Overall cost increases with frequent access.

S3 One Zone-IA

S3 Glacier Instant

Like S3 Standard-IA… cheaper storage, more expensive retrieval, longer minimum

S3 Glacier Flexible

Cold objects Objects cannot be made publicly accessible. Any Access of data requires a retrieval process.

S3 Glacier Deep Archive

Cheapest alternative. LONG time to retrieve - hours to days.

S3 Intelligent-Tiering

S3 Lifecycle Configuration

Automatically transition or expire objects in a bucket. Optimize costs.

• A lifecycle configuration is a set of rules
• Rules consist of actions
  • on a bucket or groups of objects
• Transition actions
  • e.g. to S3 Glacier
• Expiration actions
  • Delete object(s) after a certain time

Transitions

Sort of waterfall between the S3 Storage Classes

S3 Replication

• CRR: Cross-Region Replication
  • Replicate buckets across regions
• Same-Region Replication:
  • Replicate buckets within the same region
• Only differ by whether they are in the same or different account
• For different accounts:
  • Role is not trusted by default since its configured by another account
  • Add bucket policy to allow role
  

S3 Replication Options

• All objects or a subset
• Storage Class - default is to maintain
• Ownership - default is the source account
  • Can override such that destination account is the owner
• RTC: Replication Time Control
  • Make sure that buckets are in sync
  • 15 minutes

S3 Replication Considerations

• Not retroactive! Versioning needs to be ON

• One-way replication: Source to destination

  • Objects added to destination wont be added to source

• Unencrypted, SSE-S3 & SSE-KMS (with extra config)

  • Not SSE-C! ❗

• Source bucket owner needs permissions to objects

• No system events, Glacier or Glacier Deep Archive

  • Lifecycle actions wont be replicated at destination
  • Can’t replicate any objects within Glacier+

• NO DELETES

  • Delete markers are not replicated
  • Not enabled by default

  Why use replication?

  SSR: Same Region Replication CRR: Cross Region

  • SSR - Log Aggregation
  • SSR - Prod and Test Sync
  • SSR - Resilience with strict sovereignty
  • CRR - Global Resilience Improvements
  • CRR - Latency Reduction

S3 Presigned URLs

Give another person or application access to a object in a bucket using your credentials in a safe way!

• Expire at a certain time
• Person using URL is acting as the person who created the presigned URL
  • PUT, GET
• Offload media to S3
• You can create a URL for an object you have no access to
  • Few use cases, but possible
• When using the URL, the permissions match the identity which generated
• Access denied could mean the generating ID never had access, or doesn’t now
• Don’t generate with a role! URL stops working when the temporary credentials expire.

S3 Select and Glacier Select

Ways to retrieve parts of objects rather than the object. SQL-Like statement

• S3 can store objects up to 5 TB
• You often want to retrieve the entire objects
• S3/Glacier select let you use SQL-Like statements
  • select part of the object, pre-filtered by S3
• CSV, JSON, Parquet, BSZIP2 compression for CSV and JSON

Architecture

S3 Events

Receive notifications when certain events happen in your bucket

S3 Notifications

• Notification generated when events occur in a bucket
  • can be delivered to SNS, SQS and Lambda functions
• Object Created (Put, Post, copy, CompleteMultiPartUpload)
• Object Delete (*, Delete, DelteMarkedCreated)
• Object Restore (Post(Initiated), Completed)
• Replication
• Use EventBridge as default!
  • Newer and adds support for more services and events

S3 Access Logs

Provides detailed records for the requests that are made to a bucket

S3 Object Lock

You can use S3 Object Lock to store objects using a write-once-read-many (WORM) model. It can help you prevent objects from being deleted or overwritten for a fixed amount of time or indefinitely. You can use S3 Object Lock to meet regulatory requirements that require WORM storage, or add an extra layer of protection against object changes and deletion.

• Object Lock enabled on “new” buckets* (Support for existing)
• Write-Once-Read-Many (WORM) - No delete, No owerwrite
• Requires versioning - individual versions are locked
• 1 - Retention Period
• 2 - Legal Hold
• Both, One or the other, or none
• A bucket can have default object lock settings

Retention

• Specify DAYS & YEARS - A Retention Period
• COMPLIANCE - Cant be adjusted, deleted, overwritten
  • even by account root user
  • until retention expires
  • Use due to compliance
• GOVERNANCE - special permissions can be granted allowing lock settings to be adjusted
• s3:ByPassGovernanceRetention
  • x-ams-bypass-governance-retention:true (console default)

Legal Hold

• Set on an object version - ON or OFF
  • No retention
• NO DELETES or changes until removed
• s3:PutObjectLegalHold is required to add or remove
• Prevent accidental deletion of object version

⛅ Virtual Private Cloud (VPC)

VPC Sizing and Structure

VPC Considerations

• VPC CIDR range
• What size should the VPC be
• Are there any networks we can’t use?
• VPC’s, Cloud, On-premises, Partners & Vendors
• Try to predict the future
• VPC Structure - Tiers & Resiliency (Availability) Zones
• Global architecture
  • E.g. ranges to avoid in a real-case scenario
• VPC minimum /28 (16 IPs), maximum /16 (65536 IPs)
• Personal preference for the 10.x.y.z range
• Avoid common ranges - avoid future issues
• Reserve 2+ networks per region being used per account

| VPC Size | Netmask | Subnet Size | Hosts/Subet* | Subnets/VPC | Total IPs* | | | | | | | | | Micro | /24 | /27 | 27 | 8 | 216 | | Small | /21 | /24 | 251 | 8 | 2008 | | Medium | /19 | /22 | 1019 | 8 | 8152 | | Large | /18 | /21 | 2043 | 8 | 16344 | | Extra Large | /16 | /20 | 4091 | 16 | 65456 |

VPC Structure

• Number of AZs for VPC
• Start with 3 as default
  • 1 as spare for future
• Four tiers default
  • Web, app, db, spare
  

Custom VPCs

VPC Design - End state

Custom VPC Fundamentals

• Regional service - All AZ’s in the region
• Isolated network
• Nothing IN or OUT without explicit configuration
• Flexible configuration - simple or multi-tier
• Hybrid networking - other cloud & on-premises
• Default or dedicated tenancy
• IPv4 Private CIDR Blocks & Public IPs
• 1 Primary Private IPv4 CIDR Block
  • min /28 max /16 (16 - 65536 IPs)
• Optional secondary IPv4 Blocks
• Optional single assigned IPv6 /56 CIDR Block

DNS in a VPC

• Provided by R53
• VPC ‘Base IP +2’ Address
• enableDnsHostnames
  • gives instances DNS Names
• enableDnsSupport
  • enables DNS resolution in VPC

VPC Subnets

• AZ resilient
• A subnetwork of a VPC - within a particular AZ
• 1 subnet → 1 AZ, 1 AZ → 0+ Subnets
• IPv4 CIDR is a subset of the VPC CIDR
• Cannot overlap with other subnets
• Optional IPv6 CIDR (/64 subset of the /56 VPC - space for 256)
• Subnets can communicate with other subnets in the VPC

Subnet IP Addressing

• Reserved IP addresses (5 in total)
• 10.16.16.0/20 (10.16.16.0 → 10.16.16.255)
• Reserved addresses
  • Network Address (10.16.16.0)
    • First in network is always reserved. Goes for all networks.
  • Network+1 (10.16.16.1)
    • VPC Router
  • Network+2 (10.16.16.2)
    • Reserved (DNS*)
  • Network+3 (10.16.16.3)
    • Reserved Future Use
  • Broadcast Address 10.16.31.255
    • Last IP in subnet
• DHCP Option Set (Dynamic Host Configuration Protocol)
  • How devices receive IP addresses automatically
• Per subnet:
  • Auto assign public IPv4
  • Auto assign public IPv6

VPC Routing and Internet Gateway

VPC Router

• Every VPC has a VPC Router - Highly available
• In every subnet ’network+1’ address
• Routes traffic between subnets
• Controlled by ‘route tables’ each subnet has one
• A VPC has a Main route table - subnet default
• Route tables are attached to 0 or more subnets
• /n higher n = more specific = higher priority
  • A subnet has to have a route table. Either main by VPC or a custom.
• Route table controls what happens to data as it leaves the subnet that route table is associate with
• A subnet can only be associated with 1 route table at the time

Internet Gateway (IGW)

• Region resilient gateway attached to a VPC
• 1 VPC = 0 or 1 IGW, 1 IGW = 0 or 1 VPC
• Runs from within the AWS Public Zone
• Gateways traffic between the VPC and the Internet or AWS Public Zone (S3, SQS, SNS, etc)
• Managed - AWS handles performance
• Self note:
  • Maps private IP to Public IP and vice versa

Using an IGW

IPv4 Addresses with a IGW

• OS on EC2 is at no point aware of its public IPv4!

Bastion Host / Jumpbox

• Bastion Host = Jumpbox
• An instance in a public subnet
• Incoming management connections arrive there
• Then access internal VPC resources
• Often the only way IN to a VPC

Stateful vs Stateless Firewalls

Transmission Control Protocol (TCP)

TCP is a connection based protocol. A connection is established between two devices using a random port on a client and a known port on the server. Once established the connection is bi-directional. The “connection” is a reliable connection, provided via the segment encapsulated in IP packets.

💡 HTTP: Port 80 HTTPS: Port 443

Stateful vs Stateless Firewalls

Stateless Firewalls

2 Rules (1 IN, 1 OUT) per connection (inbound application) 2 Rules (1 OUT, 1 IN) per connection (outbound application)

Stateful Firewalls

Intelligent enough to identify the request and response components of a connection as being related

Network Access Control Lists (NACL)

Can be considered a traditional firewall within AWS VPC Every subnet has an associated NACL

• Inbound rules and Outbound rules.
• Inbound: Traffic entering the subnet
• Outbound: Traffic leaving the subnet
• Rules match the DST IP/Range, DST Port and Protocol and Allow or Deny based on that match
• Rules are processed in order, lowest rule number first. Once a match occurs, processing STOPS.
  • • is an implicit DENY if nothing else matches

• NACLs are STATELESS. Both request and response need individual rules.
  • These rule-pairs (app port and ephemeral ports) are needed on each NACL for each communication type which occurs
    1. Within a VPC
    2. TO a VPC
    3. FROM a VPC
• A VPC is created with a default NACL
  • Inbound and outbound rules have the implicit deny (*) and an ALLOW ALL rule
  • The result - all traffic is allowed, the NACL has no effect

Custom NACL

Custom NACLs can be created for a specific VPC and are initially associated with no subnets

• They only have 1 INBOUND rule - implicit (*) DENY
  • All traffic is denied
• They only have 1 OUTBOUND rule - the implicit (*) DENY

NACL Key Points

• Stateless: Request and Response seen as different
• Only impacts data crossing subnet boundary
• NACL can explicitly ALLOW and DENY
• IPs/CIDR, Ports & Protocols - no logical resources
• NACLs cannot be assigned to AWS resources - only subnets
• Use together with Security Groups to add explicit DENY (Bad IPs/Nets)
• Each subnet can have ONE NACL (default or custom)
• A NACL can be associated with MANY Subnet

VPC Security Groups (SG)

Security Groups (SGs) are another security feature of AWS VPC ... only unlike NACLs they are attached to AWS resources, not VPC subnets.

SGs offer a few advantages vs NACLs in that they can recognize AWS resources and filter based on them, they can reference other SGs and also themselves.

But.. SGs are not capable of explicitly blocking traffic - so often require assistance from NACLs

💡 STATEFUL NO EXPLICIT DENY - Need assistance from NACL

• STATEFUL - detect response traffic automatically
• Allowed (IN or OUT) request = allowed response
• NO EXPLICIT DENY - only allow or Implicit DENY
  • can’t block specific bad actors
• Support IP/CIDR and logical resources
  • including other security groups and itself
• Attached to ENI’s (Elastic Network Interfaces) not instances (even if the UI shows it this way)

Logical References

Logical referencing scales. Any new instances which use the webSG are allowed to communicate with any instances using the APP SG. Reduce admin overhead

SG Self References

Anything with the same security group can communicate

Network Address Translation (NAT) and NAT Gateways

Giving a private resource outgoing access to the internet

What is NAT?

• A set of processes - remapping source og dest IPs
• IP masquerading: Hiding CIDR Blocks behind one IP
• Gives Private VID range outgoing internet* access

NAT Architecture

NAT Gateways

• Runs from a public subnet
• Uses ELASTIC IPs (Static IPv4 Public)
• Don’t support security groups! Only NACLs
• AZ resilient Service (HA in that AZ)
  • Need a NATGW in every AZ
• For region resilience - NATGW in each AZ
  • RT in for each AZ with that NATGW as target
• Managed, scales to 45 Gpbs
  • $ Duration & Data Volume

VPC Design - NATGW Full Resilience

Nat Instance vs NAT Gateway

What about IPv6?

• NAT isn’t required for IPv6
• All IPv6 addresses in AWS are publicly routable
• The internet gateway works with all IPv6 IPs directly
• NAT Gateways don’t work with IPv6
• ::/0 Route + IGW for bi-directional connectivity
• ::/0 Route + Egress-Only Internet Gateway - Outbound Only

🖥 Elastic Compute Cloud (EC2) Basics

AZ resilient - very reliant on the AZ it is running in

Virtualization 101

EC2 is virtualization as a Service (IaaS)

💡 Virtualization is running more than one operating system on a physical hardware or server Kernel is the only part of the operating system that is able to directly interact with the hardware (CPU & MEM, Network, Devices)

Emulated Virtualization (Software Virtualization)

• Software run i privileged mode and had access to HW
• Emulated hardware, but OS believed it was running on real hardware.
  • OS tried to control HW despite it
  • Overwrite each other, crash
• Slow!

Para-Virtualization

• Only works on a small subset of OS
  • Modified source code to call the hypervisor rather than the hardware
• OS became almost aware of virtualization

Hardware Assisted Virtualization

• Hardware itself is aware of virtualization

SR-IOV (Single Root IO Virtualization)

In EC2 - This is enhanced networking

• Network card can present themself as multiple cards rather than one
• Less CPU usage for the host CPU

EC2 Architecture and Resilience

• EC2 instances are virtual machines (OS + Resources)
• EC2 Instances run on EC2 Hosts
• Shared hosts or Dedicated hosts
  • Shared hosts default
• Hosts = 1 AZ - AZ Fails, Host Fails, Instances Fails
• EBS: Elastic Block Storage

EC2 Architecture

What’s EC2 Good for?

• Traditional OS+Application Compute
• Long-Running compute
• Server style applications
  • either burst or steady-state load
• Monolithic application stacks
• Migrated application workloads or Disaster Recovery
• Tends to be default compute service within AWS!

EC2 Instance Types

• Raw CPU, Memory, Local Storage Capacity & Type
• Resource Ratios
• Storage and Data Network Bandwidth
• System Architecture / Vendor
  • ARM vs x86
• Additional Feature and Capabilities
  • GPUs, FPGAs

EC2 Categories

Five main categories

• General Purpose. Default. Diverse workloads, equal resource ratio.
• Computed Optimized. Media Processing, HPC, Scientific Modeling, gaming, Machine Learning
• Memory Optimized. Processing large in-memory datasets, some database workloads
• Accelerated Computing. Hardware GPU, fields programmable gate arrays (FPGAs)
• Storage Optimized. Sequential and Random IO - scale-out transactional databases, data warehousing, Elasticsearch, analytics workloads

Decoding EC2 Types

R5dn.8xlarge - Instance type **R - Instance Familiy 5 - generation dn - can vary. (d NVMe storage, n network optimized) 8xlarge - Instance Size

https://aws.amazon.com/ec2/instance-types/

https://instances.vantage.sh/

Storage Refresher

Key Terms

• Direct (local) attached Storage - Storage on the EC2 Host
• Network attached Storage - Volumes delivered over the network (EBS)
• Ephemeral storage - Temporary Storage
• Persistent storage - Permanent storage - lives on past the lifetime of the instance
• Block storage - Volume presented to the OS as a collection of blocks. No Structure provided.
  • Mountable
  • Bootable
• File storage - Presented as a file share. Has structure.
  • Mountable.
  • NOT Bootable
• Object storage. Collection of objects, flat.
  • Not mountable
  • Not bootable
  • S3

Storage Performance

• IO (block size)
  • “Bigger wheels”
• IOPS (Input Output Per Second)
  • “Rev of wheels”
• Throughput (MB/s)
  • “End speed”
• Block size: 16 KB, IOPS: 100 → 1.6 MB/s
  • 1 MB block size wont necessarily lead to 1000 MB/s - throughput limits etc

Elastic Block Storage (EBS)

Amazon Elastic Block Store (Amazon EBS) provides block level storage volumes for use with EC2 instances. EBS volumes behave like raw, unformatted block devices. You can mount these volumes as devices on your instances. EBS volumes that are attached to an instance are exposed as storage volumes that persist independently from the life of the instance. You can create a file system on top of these volumes, or use them in any way you would use a block device (such as a hard drive).

• Block storage: Raw disk allocations (volume). Can be encrypted using KMS.
  • Instances see block device and create file system on this device (ext3/4, xfs)
• Storage is provisioned in ONE AZ (AZ Resilient)
• Attached to *one EC2 instance (or other service) over a storage network
  • Detached and reattached. Not lifecycle linked to one instance. Persistent.
• Snapshot (backup) into S3. Create a volume from snapshot (migrate between AZs).
• Different physical storage types, different sizes, different performance profiles.
• Billed based on GB-month (and is some cases performance)

EBS Volume Types - General Purpose SSD

GP2 and GP3

GP2

• 1GB to 16 TB
• 1 IO credit = 16 KB chunk of data
• IO Credit bucket capacity of 5.4 million IO Credits
  • Fills at rate of Baseline Performance
• Bucket fills with min 100 IO Credits per second
  • Regardless of volume size
  • Beyond this, bucket fill with 3 IO credits per second, per GB of volume size (Baseline Performance)
• Burst up to 3000 IOPS by depleting the bucket
• Bucket starts off full! 5.4 million IO credits
• If you’re depleting the bucket at a higher rate than it’s refilling you’re losing credits
• Volumes up to 1 TB use this IO credit architecture
• Above 1 TB baseline is above burst. Credit system isn’t used and you always achieve baseline
• Up to maximum for GP2 of 16000 IO credit per second (baseline performance)

GP3

Removes credit bucket architecture

• 3000 IOPS
• 125 MiB/s - Standard
• GP3 is cheaper (20%) vs GP2
• Extra cost for up to 16000 IOPS or 1000 MiB/s
• 4x Faster max throughput vs GP2
  • 1000 MiB/s vs 250 MiB/s
• Benefits of both GP2 and IO1
• Suitable for
  • Virtual desktops, medium sized single instance databases such as MSSQL Server and Oracle DB, low-latency interactive apps, dev&test, boot volumes

Provisioned IOPS SSD (io1/2)

• io1/2/BlockExpress
• IOPS can be adjusted independently of size
• Consistent Low latency and jitter
• Up to:
  • 64000 IOPS per volume (4x GP2/3)
  • 256000 IOPS per volume (Block Express)
  • 1000 MB/s throughput
  • 4000 MB/s throughput (Block Express)
• 4GB - 16TB io1/2
• 4GB-64TB BlockExpress
• Limits:
  • io1 50 IOPS/GB (max)
  • io2 500 IOPS/GB (max)
  • BlockExpress 1000 IOPS/GB (max)
• Per instance restriction:
  • io1 - 260000 IOPS & 7500 MB/s
  • io2 - 160000 IOPS & 4750 MB
  • io2 Block Express - 260000 IOPS & 7500 MB/s
  

HDD-Based

• Two types (three, but legacy)
• st1
  • Throughput optimized
  • Cheap
  • 125GB - 16 GB
  • Max 500 IOPS (1MB blocks)
  • Max 500 MB/s
  • 40MB/s TB Base
  • 250 MB/s Burst
  • Frequent Access
  • Throughput-intensive
  • Sequential
  • Big data, data warehouses, log processing
• sc1
  • Cheaper
  • Cold
  • Max 250 IOPS (1 MB blocks)
  • Max 250 MB/s
  • 12 MB/s/TB Base
  • 80 MB/s/TB Burst
  • Coder data requiring fewer scans per day
  • Lowest cost HDD volume designed for less frequently accessed workloads

Instance Store Volumes

An instance store provides temporary block-level storage for your instance. This storage is located on disks that are physically attached to the host computer. Instance store is ideal for temporary storage of information that changes frequently, such as buffers, caches, scratch data, and other temporary content, or for data that is replicated across a fleet of instances, such as a load-balanced pool of web servers.

An instance store consists of one or more instance store volumes exposed as block devices. The size of an instance store as well as the number of devices available varies by instance type.

The virtual devices for instance store volumes are ephemeral[0-23]. Instance types that support one instance store volume have ephemeral0. Instance types that support two instance store volumes have ephemeral0 and ephemeral1, and so on.

• Block Storage devices
• Physically connected to one EC2 host
• Instances on that host can access them
• Highest storage performance in AWS!
• Included in instance price
• ATTACH AT LAUNCH!
  • Can’t be added after launch

• When instances move across volumes their storage will be blank
  • Stop and start will migrate to a new host
• D3 = 4.6 GB/s throughput
• I3 = 16 GB/s of sequential throughput
• More IOPS and throughput vs EBS!

Key points

• Local on EC2 Host
• Add at launch ONLY
• Lost if instance move, resize or hardware failure
• High performance
• Tradeoff - much higher performance but higher risk
• You pay for it anyway - included in instance price
• TEMPORARY!
  • Not for persistent storage of data

Instance Store vs EBS

EBS

• Persistence
• Resilience
• Storage isolated from lifecycle
• Resilience with app in-built replication
• High performance needs

Instance Store

• Resilience with app in-built replication
• High performance needs
• Super high performance needs
• Cost (often included)

Instance Store vs EBS

• Cheap = ST1 or SC1
• Throughput, streaming… = ST1
• Boot = NOT ST1 or SC1
• GP2/3 - up to 16000 IOPS
• IO1/2 up to 64000 IOPS (*256000)
• RAID0 + EBS up to 260000 IOPS (io1/2-BE/GP2/3)
• More than 260000 IOOPS → INSTANCE STORE!

EBS Snapshots

EBS Snapshots are backups of data consumed within EBS Volumes - Stored on S3.

Snapshots are incremental, the first being a full backup - and any future snapshots being incremental.

Snapshots can be used to migrate data to different availability zones in a region, or to different regions of AWS.

• Snapshots are incremental volume copies to S3
• The first is a full copy of “data” on the volume
  • If 10GB of 40GB is used, the 10GB is copied
• Future snaps are incremental
  • They only store the difference between this and previous snapshot
• Volumes can be created (restored) from snapshots
• Snapshots can be copied to another region
• STOP and START of EC2 instances will move to another host
  • You will lose your data

EBS Snapshots/Volume Performance

• New EBS volume = full performance immediately
• Snaps restore lazily - fetched gradually
• Requested blocks are fetched immediately
• Force a real of all data immediately
• Fast Snapshot Restore (FSR) - Immediate restore
  • Up to 50 snaps per region. Set on the Snap & AZ

Snapshot Consumption and Billing

• GB per month
• Used NOT allocated data

CLI Commands to Mount Filesystem on a EBS Volume

# Commands User

## Instance 1

lsblk
sudo file -s /dev/xvdf # Output data, because EBS is only attached but has no mounted fs
sudo mkfs -t xfs /dev/xvdf # Make file system on EBS volune
sudo file -s /dev/xvdf # Will output file system
sudo mkdir /ebstest # Make directory to mount EBS on 
sudo mount /dev/xvdf /ebstest # Mounts attached EBS volume to directory
cd /ebstest
sudo nano amazingtestfile.txt
# add a message
# save and exit
ls -la

## Reboot Instance 1

sudo reboot

## Instance 1 After Reboot

df -k # Volume won't show - must configure st volume is auto mounted on reboot
sudo blkid # List unique IDs for all mounted volumes
sudo nano /etc/fstab
  ADD LINE 
  UUID=YOURUUIDHEREREPLACEME  /ebstest  xfs  defaults,nofail
sudo mount -a # Will mount all files in the /etc/fstab file
cd /ebstest
ls -la # Amazingtestfile.txt still exists - volume is persistent even after reboot

## Instance 2
# We mount the same volume we detached from instance 1, and see that content is still the same
lsblk 
sudo file -s /dev/xvdf
sudo mkdir /ebstest
sudo mount /dev/xvdf /ebstest
cd /ebstest
ls -la

## Instance 3
# Instance in another AZ - we created a snapshot and created a volume from the snapshot in another AZ
lsblk 
sudo file -s /dev/xvdf
sudo mkdir /ebstest
sudo mount /dev/xvdf /ebstest
cd /ebstest
ls -la

## InstanceStoreTest

lsblk
sudo file -s /dev/nvme1n1 
sudo mkfs -t xfs /dev/nvme1n1
sudo file -s /dev/nvme1n1
sudo mkdir /instancestore
sudo mount /dev/nvme1n1 /instancestore
cd /instancestore
sudo touch instancestore.txt

## InstancStoreTest - After Restart

df -k
its not there
but we can mount it
sudo mount /dev/nvme1n1 /instancestore
cd /instancestore
ls -la

## InstanceStoreTest - After Stop/Start

sudo file -s /dev/nvme1n1

EBS Encryption

By default no encryption is applied. This adds risk - encryption helps mitigate this risk.

💡 Data only exist in encrypted form on the volume. Plaintext data only ever exist in the memory of the EC2 host KMS Keys - aws/ebs or customer managed

Key Concepts

• Accounts can be set to encrypt by default - default KMS Key
• Otherwise choose a KMS Key to use
• Each volume uses 1 unique DEK (Data Encryption Key)
• Snapshots & future volumes use the same DEK
• Can’t change a volume to NOT be encrypted!!
• OS isn’t aware of the encryption
  • No performance loss!
• If you need the OS to encrypt things, you must configure volume encryption (software disk encryption) by yourself

(Elastic) Network Interfaces, Instance IPs and DNS

EC2 Network & DNS Architecture

ENI - Elastic Network Interface

• Every EC2 instance has at least one ENI
  • Must be in same AZ
• When you launch an instance with SGs, that SG is on the ENI, not the instance itself
• (Primary) (Elastic) Network interfaces have…
  • MAC Addresses!
  • IPv4 Private IP → 10.16.0.10 → (dns) ip-10-16-0-10.ec2.internal
    • DNS can be used for internal use
  • 0 or more secondary IPs
  • 0 or 1 Public IPv4 Address → random IP → random dns based on IP
  • 1 elastic IP per private IPv4 address
    • If you assign it
    • Removes the Public IPv4
    • Replaces with the Elastic IP
    • You can’t regain the old public IPv4 if you remove Elastic IP
  • 0 or more IPv6 addresses
  • Security Groups
  • Source/Destination Check
    • Enable/disable
    • Disable to use EC2 instance as NAT
• Secondary ENI
  • As above, but can be detached and moved to other EC2 instances

Key Concepts

• Secondary ENI + MAC = Licensing
  • Move licensing between instances by moving ENI
• Multi-homed (subnets) Management and Data
• Different Security Groups - multiple interfaces with different SG on each
• OS - DOESN’T SEE PUBLIC IPv4.
  • Stop & Start = Change
• Public DNS = private IP in VPC
  • Public IP everywhere else

DEMO: Installation of Wordpress on EC2

# DBName=database name for wordpress
# DBUser=mariadb user for wordpress
# DBPassword=password for the mariadb user for wordpress
# DBRootPassword = root password for mariadb

# STEP 1 - Configure Authentication Variables which are used below
DBName='a4lwordpress'
DBUser='a4lwordpress'
DBPassword='REPLACEME'
DBRootPassword='REPLACEME'

# STEP 2 - Install system software - including Web and DB
sudo yum install -y mariadb-server httpd wget
sudo amazon-linux-extras install -y lamp-mariadb10.2-php7.2 php7.2

# STEP 3 - Web and DB Servers Online - and set to startup

sudo systemctl enable httpd
sudo systemctl enable mariadb
sudo systemctl start httpd
sudo systemctl start mariadb

# STEP 4 - Set Mariadb Root Password
mysqladmin -u root password $DBRootPassword

# STEP 5 - Install Wordpress
sudo wget http://wordpress.org/latest.tar.gz -P /var/www/html
cd /var/www/html
sudo tar -zxvf latest.tar.gz
sudo cp -rvf wordpress/* .
sudo rm -R wordpress
sudo rm latest.tar.gz

# STEP 6 - Configure Wordpress

sudo cp ./wp-config-sample.php ./wp-config.php
sudo sed -i "s/'database_name_here'/'$DBName'/g" wp-config.php
sudo sed -i "s/'username_here'/'$DBUser'/g" wp-config.php
sudo sed -i "s/'password_here'/'$DBPassword'/g" wp-config.php   
sudo chown apache:apache * -R

# STEP 7 Create Wordpress DB

echo "CREATE DATABASE $DBName;" >> /tmp/db.setup
echo "CREATE USER '$DBUser'@'localhost' IDENTIFIED BY '$DBPassword';" >> /tmp/db.setup
echo "GRANT ALL ON $DBName.* TO '$DBUser'@'localhost';" >> /tmp/db.setup
echo "FLUSH PRIVILEGES;" >> /tmp/db.setup
mysql -u root --password=$DBRootPassword < /tmp/db.setup
sudo rm /tmp/db.setup

# STEP 8 - Browse to http://your_instance_public_ipv4_ip

Amazon Machine Images (AMI)

Amazon Machine Images (AMI) 's are the images which can create EC2 instances of a certain configuration.

In addition to using AMI's to launch instances, you can customize an EC2 instance to your bespoke business requirements and then generate a template AMI which can be used to create any number of customized EC2 instances.

• AMI’s can be used to launch EC2 instance
• AWS or Community provided
• Marketplace (can include commercial software)
• Regional. Unique ID. e.g. ami-0a893824e0928592f20
• Permissions (Public, Your Account, Specific Accounts)
• You can create an AMI from an EC2 instance you want to template
• AMI’s are containers that reference snapshots

AMI Lifecycle

Key Concepts

• AMI = One region. Only works in that one region.
• AMI Baking - Creating an AMI from a configured instance + application
• An AMI can’t be edited. Launch instance, update configuration and make a new AMI
• Can be copied between regions (includes its snapshots)
• Remember permissions. Default = your account

DEMO: A4L AMI

# DBName=database name for wordpress
# DBUser=mariadb user for wordpress
# DBPassword=password for the mariadb user for wordpress
# DBRootPassword = root password for mariadb

# STEP 1 - Configure Authentication Variables which are used below
DBName='a4lwordpress'
DBUser='a4lwordpress'
DBPassword='4n1m4l$L1f3'
DBRootPassword='4n1m4l$L1f3'

# STEP 2 - Install system software - including Web and DB
sudo yum install -y mariadb-server httpd wget
sudo amazon-linux-extras install -y lamp-mariadb10.2-php7.2 php7.2

# STEP 3 - Web and DB Servers Online - and set to startup

sudo systemctl enable httpd
sudo systemctl enable mariadb
sudo systemctl start httpd
sudo systemctl start mariadb

# STEP 4 - Set Mariadb Root Password
mysqladmin -u root password $DBRootPassword

# STEP 5 - Install Wordpress
sudo wget http://wordpress.org/latest.tar.gz -P /var/www/html
cd /var/www/html
sudo tar -zxvf latest.tar.gz
sudo cp -rvf wordpress/* .
sudo rm -R wordpress
sudo rm latest.tar.gz

# STEP 6 - Configure Wordpress

sudo cp ./wp-config-sample.php ./wp-config.php
sudo sed -i "s/'database_name_here'/'$DBName'/g" wp-config.php
sudo sed -i "s/'username_here'/'$DBUser'/g" wp-config.php
sudo sed -i "s/'password_here'/'$DBPassword'/g" wp-config.php   
sudo chown apache:apache * -R

# STEP 7 Create Wordpress DB

echo "CREATE DATABASE $DBName;" >> /tmp/db.setup
echo "CREATE USER '$DBUser'@'localhost' IDENTIFIED BY '$DBPassword';" >> /tmp/db.setup
echo "GRANT ALL ON $DBName.* TO '$DBUser'@'localhost';" >> /tmp/db.setup
echo "FLUSH PRIVILEGES;" >> /tmp/db.setup
mysql -u root --password=$DBRootPassword < /tmp/db.setup
sudo rm /tmp/db.setup

# STEP 8 - Browse to http://your_instance_public_ipv4_ip

# Step 9

sudo yum install -y cowsay

cowsay "oh hi"

Create file /etc/update-motd.d/40-cow

sudo nano /etc/update-motd.d/40-cow

#!/bin/sh
cowsay "Amazon Linux 2 AMI - Animals4Life"

sudo chmod 755 /etc/update-motd.d/40-cow
sudo rm /etc/update-motd.d/30-banner

sudo update-motd
sudo reboot

Relogin

## STEP 10 - CREATE AMI
## STEP 11 - USE AMI to launch an instance

EC2 Purchase Options (Launch Types)

On-Demand

• Default
• No specific pros or cons
• Instances of different sizes run on the same EC2 hosts - consuming a defined allocation of resources
• On-Demand instances are isolated but multiple customer instances run on shared hardware
• Per-second billing while an instance is running. Associated resources such as storage consume capacity, so bill, regardless of instance state
• Default purchase option.
• No interruption
• Predictable pricing
• No upfront cost
• No discount
• Short term workloads
• Unknown workloads
• Apps which can’t be interrupted

Spot

• SPOT pricing is AWS selling unused EC2 host capacity for up to 90% discount - the spot price is based on the spare capacity at a given time
• If spot price goes above your limit the instances are terminated
  • Makes Spot unreliable
• Never use spot for workloads which can’t tolerate interruptions
• Non time critical
• Anything which can be rerun
• Bursty capacity needs
• Cost sensitive workloads
• Anything which is stateless

Reserved

Long term consistent usage of EC2

• Matching instances - reduced or no per sec price
• Unused reservation still billed
• Partial coverage of larger instance
• You commit to AWS that you will use the instance for a longer period of time - regardless of whether you use them or not
• Reservations are for one or three years
• No-Upfront:
  • Some savings for agreeing to the term
  • Per second
• All upfront:
  • Means no per second fee
• Partial upfront:
  • Reduced per second fee

Dedicated Instance

• No other customers use the same hardware
  • You have the hardware to yourself
• You neither own or share the host
• Extra charges for instances, but dedicated hardware
• You don’t manage capacity

Dedicated Host

The host is allocated to you in its entirety

• Pay for HOST
• No instance charges
• You must managed the capacity and the resources
• Use because of licensing based on sockets/cores requirements
• Host affinity links instances to hosts

Reserved Instances

Aka Standard Reserved

Scheduled Reserved Instances

• Ideal for long term usage which doesn’t run constantly
• Options:
  • Batch processing daily for 5 hours starting at 23:00
  • Weekly data, sales analysis. Every friday for 24 hours
  • 100 hours of EC2 per month
• Doesn’t support all instance types or regions. 1200 hours per year and 1 year term minimum

Capacity Reservations

In case of disaster and lack of capacity, AWS uses a priority list of whom to give capacity to

• Regional Reservation provides a billing discount for valid instances launched in any AZ in that region
• While flexible they don’t reserve capacity within in AZ - which is risky during major faults when capacity can be limited
• Zonal reservations only apply to one AZ providing billing discounts and capacity reservation in that AZ
• On-demand capacity reservations can be booked to ensure you always have access to capacity in an AZ when you need it - but at full on-demand price. No term limits - but you pay regardless of if you consume it.

EC2 Savings Plan

• A hourly commitment for a 1-3 year term
• A reservation of general compute $ amounts($20 per hour for 3 years)
  • Or a specific EC2 Savings plan - flexibility on size & OS
• Compute products, currently EC2, Farge & Lambda
• Products have an on-demand rate and a savings plan rate
• Resource usage consumes savings plan commitment at the reduced savings plan rate
• Beyond your commitment on-demand is used

Instance Status Checks & Auto Recovery

With instance status monitoring, you can quickly determine whether Amazon EC2 has detected any problems that might prevent your instances from running applications. Amazon EC2 performs automated checks on every running EC2 instance to identify hardware and software issues. You can view the results of these status checks to identify specific and detectable problems.

You can create an Amazon CloudWatch alarm that monitors an Amazon EC2 instance and automatically recovers the instance if it becomes impaired due to an underlying hardware failure or a problem that requires AWS involvement to repair. Terminated instances cannot be recovered. A recovered instance is identical to the original instance, including the instance ID, private IP addresses, Elastic IP addresses, and all instance metadata

Instance Status Checks

• Every EC2 instance have 2 status check
• First
  • System status
    • Loss of system power
    • Loss of network connectivity
    • Host software issues
    • Host hardware issues
• Second
  • Instance status
    • Corrupted file system
    • Incorrect instance networking
    • OS Kernel issues

Termination Protection

💡 Termination Protection is a feature which adds an attribute to EC2 instances meaning they cannot be terminated while the flag is enabled.

It provides protection against unintended termination and also allows role separation, where junior admins can be allowed to terminate but ONLY for instances with no protection attribute set.

Horizontal and Vertical Scaling

*Within AWS Horizontal and Vertical scaling are two ways which systems have to deal with increasing or decreasing user-side load.

Adding or removing resources to a system*

Vertical Scaling

• Resizing EC2 instance
  • t3.large → t3.xlarge
• Each resize requires a reboot - disruption
• Larger instances often carry a $ premium
• There is an upper cap on performance - instance size
• No application modification required
• Works for ALL applications - even monoliths

Horizontal Scaling

• Adds more instances as load increases
• Load Balancer
  • Between servers and customers
  • Distribute load over all servers
• Sessions, sessions, sessions
• Requires application support OR off-host sessions (stateless sessions)
• No disruption when scaling
• Connections can be moved between servers (if stateless sessions without disruption)
• Often less expensive - no large instance premium
• More granular

Instance Metadata

Instance metadata is data about your instance that you can use to configure or manage the running instance. Instance metadata is divided into categories, for example, host name, events, and security groups.

Instance metadata is accessed from an EC2 instance using

http://169.254.169.254/latest/meta-data/

• EC2 Service provides data to instances
• Accessible inside ALL instances
• http://169.254.169.254
  • http://169.254.169.254/latest/meta-data/
  • REMEMBER THIS
• All information about environment can be queried
• Networking
• Authentication
• User-Data
• NOT AUTHENTICATED or ENCRYPTED
  • Treat metadata as something that can and will be exposed

🐳 Containers & ECS

Introduction to Containers

Virtualization Problems

Containerization

Image Anatomy

• Running copy of a docker image
• Made up of multiple layers
• Dockerfile creates docker image
  • Each step creates fs layers
• Images are created from a base image or scratch
  • Images contain readonly layers, changes are layered onto the image using a differential architecture

Container Anatomy

• Running copy of a docker image with one difference - one additional read/write layer
  • Anything happening during running is only stored in this layer

Container Registry (e.g. Docker Hub)

Container Key Concepts

• Dockerfiles are used to build images
• Portable - self-contained, always run as expected
• Lightweight - Parent OS used, fs layers are shared
• Container only runs the application & environment it needs
• Provides much of the isolations VM’s do
• Ports are exposed to the host and beyond
• Application stack can be multi-container…

Elastic Container Service (ECS) Concepts

Remove admin overhead of managing containers

ECS

• Runs in two modes
  • EC2
  • Fargate
    • 20 GB of free ephemeral storage
• Create ECS Cluster
• ECR - Elastic Container Registry (AWS alt to Docker Hub)
• Container definition - Tell container where container image is
• Task definition - One or many container inside it
  • Represents the application as a whole
  • Store the resources used by the task
    • CPU, Memory, Network mode, compatibility (ec2 vs fargate)
    • Task role
      • IAM role that the task can use
      • Best way to give tasks access to resources
• Service definition
  • How many copies of a task we want to run
  • Add Load balancer
  • Scaling
  • High availability
  • Service is what is deployed into the ECS Cluster!

ECS Concepts

• Container Definition - Image & Ports
• Task Definition - Security (Task Role), Container(s), Resources
• Task Role - IAM Role which the TASK assumes
• Service - How many copies, HA, Restarts

ECS - Cluster Mode

ECS is capable of running in EC2 mode or Fargate mode.

EC2 mode deploys EC2 instances into your AWS account which can be used to deploy tasks and services.

With EC2 mode you pay for the EC2 instances regardless of container usage

Fargate mode uses shared AWS infrastructure, and ENI's which are injected into your VPC

You pay only for container resources used while they are running

EC2 Mode

• EC2 cluster is created within a VPC - benefit from multiple AZ’s
• ASG - Auto Scaling Group
  • Horizontal scaling
• Container Registry (ECR)
• If you want to use containers, but need to manage the host the container is running on - EC2!
  • Keep overhead and flexibility

Fargate Mode

• “Serverless” - No servers to manage
• Not paying for EC2 instances regardless of you’re using them or not
• How containers are hosted are different from EC2 mode
• Fargate Shared Infrastructure
• Tasks are services actually running from a shared infrastructure platform
• Tasks injected into the VPC - given ENI
• A lot of customizability
• You only pay for the containers you are using based on the resources you consume!

EC2 vs ECS (EC2) vs Fargate

• If you use containers - ECS!
• Large workload - price conscious - EC2 Mode
  • Beware of management overhead
• Large workload - overhead conscious - Fargate
• Small/burst workloads - Fargate
• Batch/periodic workloads - Fargate

Elastic Container Registry (ECR)

• Managed container image registry service
  • like Dockerhub but for AWS
• Each AWS account has a public and private registry
• Each registry can have many repository
• Each repository can contain many images
• Images can have several tags
• Public = public R/O
  • R/W requires permissions
• Private = permissions required for any R/O or R/W
• Integrated with IAM
• Image scanning, basic and enhanced (inspector)
• nr real-time Metrics → CW(auth, push, pull)
• API actions = CloudTrail
• Events → EventBridge
• Replication
  • Cross-region AND Cross-account

Kubernetes 101

Kubernetes, also known as K8s, is an open-source system for automating deployment, scaling, and management of containerized applications.

Cluster Structure

Cluster Detail

Key Concepts

• Cluster - A deployment of Kubernetes, management, orchestration …
• Node - Resources; pods are placed on nodes to run
• Pod - 1+ containers; smallest unit in Kubernetes; often 1 container 1 pod
• Service - Abstraction, service running on 1 ore more pods
• Job - ad-hoc, creates one ore more pods until completion
• Ingress - Exposes a way into a service (Ingress → Routing → Service → 1+ Pods)
• Ingress Controller - used to provide ingress (e.g. AWS LB Controller uses ALB/NLB)
• Persistent Storage (PV) - Volume whose lifecycle lives beyond any 1 pod using it

Elastic Kubernetes Service (EKS) 101

Amazon Elastic Kubernetes Service (Amazon EKS) is a fully-managed, Kubernetes implementation that simplifies the process of building, securing, operating, and maintaining Kubernetes clusters on AWS. Kubernetes as a Service (KaaS?)

• AWS Managed Kubernetes - open source & cloud agnostic
• AWS, Outposts, EKS Anywhere, EKS Distro
• Control plane scales and runs on multiple AZs
• Integrates with AWS services - ECR, ELB, IAM, VPC
• EKS Cluster = EKS Control Plane & EKS Nodes
• etcd distributed across multiple AZs
• Nodes - Self managed, managed node groups or Fargate pods
  • Windows, GPU, Inferentia, Bottlerocket, Outposts, Local zones
    • Check node type
• Storage Providers include - EBS, EFS, FSx Lustre, FSx for NetApp ONTAP
• Two VPC!
  • AWS Managed
  • Customer VPC
  • These will communicate

🉐 Advanced EC2

Bootstrapping EC2 Using User Data

EC2 Bootstrapping is the process of configuring an EC2 instance to perform automated install & configuration steps 'post launch' before an instance is brought into service. With EC2 this is accomplished by passing a script via the User Data part of the Meta-data service - which is then executed by the EC2 Instance OS

EC2 Bootstrapping

• Bootstrapping is a process which allows a system to self-configure

• Bootstrapping allows EC2 Build Automation

• http://169.254.169.254/latest-user-data

• Anything in User Data is executed by the instance OS

• ONLY on launch

• EC2 doesn’t interpret, the OS needs to understand the User Data

  

User Data Key Points

• It’s opaque to EC2 - its just a block of data

• It’s NOT secure - don’t use it for passwords or long term credentials (ideally)

• User data is limited to 16 KB in size

• Can be modified when instance is stopped

• But only executed once at launch

  

Enhanced Bootstrapping with CFN-INIT

CFN-INIT is a powerful desired-state-like configuration engine which is part of the CFN suite of products.

It allows you to set a state for things like packages, users, groups, sources and files within resources inside a template - and it will make that change happen on the instance, performing whatever actions are required.

Creation policies create a 'WAIT STATE' on resources .. not allowing the resource to move to CREATE_COMPLETE until signalled using the cfn-signal tool.

• cfn-init helper script - installed on EC2 OS
• Simple configuration management system
• Procedural (User Data) vs Desired State (cfn-init)
• Packages, Groups, Users, Sources, Files, Commands and Services
• Provided with directives via Metadata and AWS::ClodFormation::Init on a CFN resource
• Variables passed into User Data by CloudFormation

cfn-init

CreationPolicy and Signals

• -e $? = output of previous command

EC2 Instance Roles & Profile

EC2 Instance roles and Instance Profiles are how applications running on an EC2 instance can be given permissions to access AWS resources on your behalf.

Short Term Temporary credentials are available via the EC2 Instance Metadata and are renewed automatically by the EC2 and STS Services.

Starts with an IAM role with a permissions policy. EC2 instance role allows the EC2 service to assume that role.

The instance profile is the item that allows the permissions to get inside the instance. When you create an instance role in the console, an instance profile is created with the same name.

When IAM roles are assumed, you are provided temporary roles based on the permission assigned to that role. These credentials are passed through instance meta-data.

EC2 and the secure token service ensure the credentials never expire.

EC2 Instance Roles

• Credentials are inside meta-data
• iam/security-credentials/role-name
• Automatically rotated - Always valid
• Should always be used rather than adding access keys into instance
• CLI tools will use ROLE credentials automatically

AWS Systems Manager Parameter Store

The SSM Parameter store is a service which is part of Systems Manager which allows the storage and retrieval of parameters - string, stringlist or secure string.

The service supports encryption which integrates with KMS, versioning and can be secured using IAM.

The service integrates natively with many AWS services - and can be accessed using the CLI/APIs from anywhere with access to the AWS Public Spare Endpoints.

aws ssm get-parameters --names /my-app/dbstring # return JSON object

aws ssm get-parameters-by-path --path /my-app/ # return three parameters - three JSON objects

aws ssm get-parameters-by-path --path /my-app/ --with-decryption # decrypt encrypted parameters. require permissions to both interact with SSM and KMS

SSM Parameter Store

• Storage for configuration & secrets
• String, StringList & SecureString
• License codes, Database Strings, Full Configs & Passwords
• Hierarchies & Versioning
• Plaintext and Ciphertext
• Public Parameters - Latest AMIs per region

System and Application Logging on EC2

CloudWatch and CloudWatch Logs cannot natively capture data inside an instance.

Logging on EC2

• CloudWatch is for metrics
• CloudWatch Logs is for logging
• Neither capture data inside an instance
• CloudWatch Agent is required - runs inside the instance
  • Needs configuration and permissions

EC2 Placement Groups

Allows you to influence placement, having instances physically closer to each other

Cluster Placement Groups

Pack Instances close together. PERFORMANCE!

• Absolute highest performance possible within EC2
• In a single AZ
• Same Rack
  • Sometime same host
• All members have direct connections to each other
• Up to 10Gbps per stream
• 5Gbps normally
• Lowest latency and max PPS possible
• Tradeoff: Little to no resilience
• Can’t span AZs - one AZ only - locked when launching first instance
• Can span VPC peers - but impacts performance
• Requires a supported instance type
• Use the same type of instance (not mandatory)
• Launch at the same time (not mandatory, very recommended)
• 10Gbps single stream performance
• Use cases:
  • Performance
  • Fast speeds
  • Low latency

Spread Placement Groups

Keep instances separated

• Can span multiple AZs
• Distinct racks - if a single rack fail, fault is isolated to rack
• 7 instances per AZ - HARD LIMIT - Isolated infrastructure limit
• Provides infrastructure isolation
• Each rack has its own network and power source
• Not supported for Dedicated Instances or Hosts
• Use case
  • Small number of critical instances that need to be kept separated from each other

Partition Placement Groups

Groups of instances spread apart

• Across multiple AZs
• Divided into “partitions”
  • MAX 7 per AZ
• Each partition has its own racks - no sharing between partitions
• Instances can be placed in a specific partition
  • or auto placed
• Great for topology aware applications
  • HDFS, HBase and Cassandra
• Contain the impact of failure to part of an application

EC2 Dedicated Hosts

Dedicated hosts are EC2 Hosts which support a certain type of instance which are dedicated to your account.

You can pay an on-demand or reserved price for the hosts and then you have no EC2 instance pricing to pay for instances running on these dedicated hosts.

Generally dedicated hosts are used for applications which use physical core/socket licensing

• EC2 Host dedicated to you
• Specific family, e.g. a1, c5, m5
• No instance charges - you pay for the host
• On-demand & Reserved options available
• Host hardware has physical sockets and cores

Limitations & Features

• AMI Limits - RHEL, SUSE Linux, and Windows AMIs aren’t supported
• Amazon RDS instances are not supported
• Placement groups are not supported for dedicated hosts
• Hosts can be shared with other ORG Account… RAM

Enhanced Networking & EBS Optimized

Enhanced networking is the AWS implementation of SR-IOV, a standard allowing a physical host network card to present many logical devices which can be directly utilized by instances.

This means lower host CPU usage, better throughput, lower and consistent latency

EBS optimization on instances means dedicated bandwidth for storage networking - separate from data networking.

Enhanced Networking

• Uses SR-IOV - NIC (Network Interface Card) is virtualization aware
• The host has multiple logical cards per physical card, which interacts with the instance
• Higher I/O & Lower Host CPU Usage
• More bandwidth
• Higher packets-per-second (PPS)
• Consistent lower latency
• Either enabled by default or available free of charge (for most instances)

EBS Optimized

• EBS = Block storage over the network
• Historically network was shared
  • Data and EBS
• EBS Optimized means dedicated capacity for EBS
• Most instances support and have enabled by default
  • Some support, but enabling costs extra

🛣️ Route 53 - Global DNS

R53 Public Hosted Zones

A public hosted zone is a container that holds information about how you want to route traffic on the internet for a specific domain which is accessible from the public internet

💡 Two types of zones in R53: Public and Private

R53 Hosted Zones

• A R53 Hosted Zone is a DNS DB for a domain, e.g. a4l.org
• Globally resilient (multiple DNS Servers)
• Created with domain registration via R53 - can be created separately
• Host DNS Records (A, AAAA, MX, NS, TXT,…)
• Hosted Zones are what the DNS system references - Authoritative for a domain e.g. a4l.org
• DNS Database

R53 Public Hosted Zones

• DNS Database (Zone file) hosted by R53 (Public Name Servers)
• Accessible from the public internet & VPCs
• Hosted on “4” R53 Name Servers (NS) specific for the zone
  • use “NS records” to point at these NS (connect to global DNS)
• Resource Records (RR) created within the Hosted Zone
• Externally registered domains can point at R53 Public Zone

R53 Private Hosted Zones

A private hosted zone is a container that holds information about how you want Amazon Route 53 to respond to DNS queries for a domain and its subdomains within one or more VPCs that you create with the Amazon VPC service

• A public hosted zone, which isn’t public
• Associated with VPCs
• Only accessible in those VPCs
• Using different accounts is supported via CLI/API
• Split-view (overlapping public & private) for PUBLIC and INTERNAL use with the same zone name

R53 Split View Hosted Zones

• Public zone is a subset of the private zone, limiting access to some resources

CNAME vs R53 Alias

This lesson steps through the shortcomings of the CNAME record type, the differences between CNAME and ALIAS and when to use one v's the other.

R53 CNAME vs Alias (the problem)

• “A” Maps a NAME to an IP Address
  • catagram.io → 1.3.3.7
• CNAME maps a NAME to another NAME
  • www.catagram.io → catagram.io
• CNAME is invalid for naked/apex (catagram.io)
• Many AWS services us a DNS Name (ELBs)
• With just CNAME - catagram.io → ELB would be invalid

Alias

• ALIAS records map a NAME to an AWS resource
• Can be used both for naked/apex and normal records
• For non apex/naked - functions like CNAME
• There is no charge for ALIAS requests pointing at AWS resources
• For AWS services - default to picking ALIAS
• Should be the same “type” as what the records is pointing at
• Use ALIAS when pointing at:
  • API Gateway
  • CloudFront
  • Elastic Beanstalk
  • ELB
  • Global Accelerator
  • S3

R53 Health Checks

Amazon Route 53 health checks monitor the health and performance of your web applications, web servers, and other resources. Each health check that you create can monitor one of the following:

• The health of a specified resource, such as a web server
• The status of other health checks
• The status of an Amazon CloudWatch alarm

• Health check are separate from, but are used by records

• Health checkers located globally

• Health checker check every 30s (every 10s costs extra)

• TCP, HTTP/HTTPS, HTTP/HTTPS with String Matching

• Healthy or Unhealthy

• Endpoint, CloudWatch Alarm, Check of Checks (Calculated)

  ---

❗Failover: Active/Passive ❗

Active-Active Failover

Use this failover configuration when you want all of your resources to be available the majority of the time. When a resource becomes unavailable, Route 53 can detect that it’s unhealthy and stop including it when responding to queries.

In active-active failover, all the records that have the same name, the same type (such as A or AAAA), and the same routing policy (such as weighted or latency) are active unless Route 53 considers them unhealthy. Route 53 can respond to a DNS query using any healthy record.

Active-Passive Failover

Use an active-passive failover configuration when you want a primary resource or group of resources to be available the majority of the time and you want a secondary resource or group of resources to be on standby in case all the primary resources become unavailable. When responding to queries, Route 53 includes only the healthy primary resources. If all the primary resources are unhealthy, Route 53 begins to include only the healthy secondary resources in response to DNS queries.

Configuring an Active-Passive Failover with Weighted Records and configuring an Active-Passive Failover with Multiple Primary and Secondary Resources are incorrect because an Active-Passive Failover is mainly used when you want a primary resource or group of resources to be available most of the time and you want a secondary resource or group of resources to be on standby in case all the primary resources become unavailable. In this scenario, all of your resources should be available all the time as much as possible which is why you have to use an Active-Active Failover instead.

Configuring an Active-Active Failover with One Primary and One Secondary Resource is incorrect because you cannot set up an Active-Active Failover with One Primary and One Secondary Resource. Remember that an Active-Active Failover uses all available resources all the time without a primary nor a secondary resource.

Routing Policy 1: Simple Routing

Simple routing lets you configure standard DNS records, with no special Route 53 routing such as weighted or latency. With simple routing, you typically route traffic to a single resource, for example, to a web server for your website.

Routing Policy 2: Failover Routing

Failover routing lets you route traffic to a resource when the resource is healthy or to a different resource when the first resource is unhealthy 1st of four routing policies

💡 Create two records of the same name and the same type. One is set to be the primary and the other is the secondary. This is the same as the simple policy except for the response. Route 53 knows the health of both instances. As long as the primary is healthy, it will respond with this one. If the health check with the primary fails, the backup will be returned instead. This is set to implement active - passive failover.

Routing Policy 3: Multi Value Routing

Multivalue answer routing lets you configure Amazon Route 53 to return multiple values, such as IP addresses for your web servers, in response to DNS queries. You can specify multiple values for almost any record, but multivalue answer routing also lets you check the health of each resource, so Route 53 returns only values for healthy resources

💡 Simple records use one name and multiple values in this record. These will be health checked and the unhealthy responses will automatically be removed. With multi-value, you can have multiple records with the same name and each of these records can have a health check. R53 using this method will respond to queries with any and all healthy records, but it removes any records that are marked as unhealthy from those responses. This removes the problem with simple routing where a single unhealthy record can make it through to your customers. Great alternative to simple routing when you need to improve the reliability, and it's an alternative to failover when you have more than two records to respond with, but don't want the complexity or the overhead of weighted routing.

Routing Policy 4: Weighted Routing

Weighted routing lets you associate multiple resources with a single domain name (catagram.io) and choose how much traffic is routed to each resource. This can be useful for a variety of purposes, including load balancing and testing new versions of software.

💡 Create multiple records of the same name within the hosted zone. For each of those records, you provide a weighted value. The total weight is the same as the weight of all the records of the same name. If all of the parts of the same name are healthy, it will distribute the load based on the weight. If one of them fails its health check, it will be skipped over and over again until a good one gets hit. This can be used for migration to separate servers.

Routing Policy 5: Latency Routing

If your application is hosted in multiple AWS Regions, you can improve performance for your users by serving their requests from the AWS Region that provides the lowest latency.

💡 Multiple records in a hosted zone can be created with the same name and same type. When a client request arrives, it knows which region the request comes from. It knows the lowest latency and will respond with the lowest latency.

Routing Policy 6: Geolocation Routing

Geolocation routing lets you choose the resources that serve your traffic based on the geographic location of your users, meaning the location that DNS queries originate from.

💡 Focused to delivering results matching the query of your customers. The record will first be matched based on the country if possible. If this does not happen, the record will be checked based on the continent. Finally, if nothing matches again it will respond with the default response. This can be used for licensing rights. If overlapping regions occur, the priority will always go to the most specific or smallest region. The US will be chosen over the North America record.

• Good for restricting content to a certain location

Routing Policy 7: Geoproximity Routing

Geoproximity routing lets Amazon Route 53 route traffic to your resources based on the geographic location of your users and your resources. You can also optionally choose to route more traffic or less to a given resource by specifying a value, known as a bias. A bias expands or shrinks the size of the geographic region from which traffic is routed to a resource.

• As close to customers as possible
• Calculate distance between customer and records
• Define rules and a bias
  • Bias: + or - bias can be added to rules

R53 Interoperability

This lesson details how Route53 provides Registrar and DNS Hosting features and steps through architectures where it is used for BOTH, or only one of those functions - and how it integrates with other registrars or DNS hosting.

• R53 normally has two jobs - Domain registrar and Domain Hosting
• R53 can do BOTH, or either registrar or hosting
• R53 Accepts your money (domain registration fee)
• R53 allocates 4 Names Servers (NS) (Domain hosting)
• R53 Creates a zone file (domain hosting) on the above NS
• R53 communicates with the registry of the TLD (Domain Registrar)
  • sets the NS records for the domain to point at the 4 NS above

R53: Both Roles

R53: Registrar Only

“Worst way to manage domains”

R53: Hosting Only

💽 Relational Database Service (RDS)

ACID vs BASE

💡 CAP Theorem: Consistency, Availability, Partition Tolerant - Choose two

**ACID:

• Atomic:** All or nothing - Consistent: From one valid state to another - Isolated: Transactions don’t interfere with each other - Durable: Stored on non-volatile memory. Resilient to crash.

**BASE:

• Basicly Available:** Read and write available as much as possible without consistency guarantees - Soft State: Db doesn’t enforce consistency. Offload onto app/user - Eventually: Eventually consistent (wait long enough)

• DynamoDB is BASE

Database on EC2

• Splitting DB and App into different AZs introduce dependencies between AZs

Reasons to host DB on EC2:

• Access to the DB instance OS
• Advanced DB Option tuning (DBROOT)
• Vendor demands
• DB or DB version AWS don’t provide
• Specific OS/DB Combination AWS don’t provide
• Architecture AWS don’t provide (replication/resilience)
• Decision makers who just want it

Reasons to NOT host DB on EC2:

• Admin overhead - managing EC2 and DBHost
• Backup / DR Management
• EC2 is single AZ
• Features - some of AWS DB products are amazing
• EC2 is ON or OFF - no serverless, no easy scaling
• Replication - skills, setup time, monitoring & effectiveness
• Performance - AWS invest time into optimization and features

Relational Database Service (RDS)

The Relational Database Service (RDS) is a Database(server) as a service product from AWS which allows the creation of managed databases instances.

• ❌ “Database as a Service” (DBaaS)
  • Not completely true
• ✅ DatabaseServer-as-a-Service!
• Managed Database Instance (1+ Databases)
• Multiple engines MySQL, MariaDB, PostgresSQL, Oracle, Microsoft SQL Server
• Amazon Aurora
  • Different from the other engines

RDS Architecture

RDS Database Instance

RDS Multi AZ

MultiAZ is a feature of RDS which provisions a standby replica which is kept in sync Synchronously with the primary instance.

The standby replica cannot be used for any performance scaling ... only availability.

Backups, software updates and restarts can take advantage of MultiAZ to reduce user disruption.

In case of failure of the primary DB, the CNAME points to the standby DB

• No Free-tier! Extra cost for standby replica
• Standby can’t be directly used
• 60-120 seconds failover
• Same region only (other AZs in the VPC)
• Backups taken from Standby (removes performance impact)
• AZ Outage, Primary Failure, Manual failover, Instance type change and software patching

RBD Backups and Restores

RDS is capable of performing Manual Snapshots and Automatic backups

Manual snapshots are performed manually and live past the termination of an RDS instance

Automatic backups can be taken of an RDS instance with a 0 (Disabled) to 35 Day retention.

Automatic backups also use S3 for storing transaction logs every 5 minutes - allowing for point in time recovery.

Snapshots can be restored .. but create a new RDS instance

RTO vs RPO

RTO: Recovery Time Objective

• Time between DR event and full recovery
• Influenced by process, staff, tech and documentation
• Generally lower values cost more

RPO: Recovery Point Objective

• Time between last backup and the incident
• Amount of maximum data loss
• Influences technical solution and cost
• Generally lower values cost more

RDS Backups

Automatic Backups

• Delete after 0 to 35 days
  • Restore to any point in time in this window

Manual Snapshots

• Don’t expire - manual deletion

AWS Managed S3 Bucket → Region Resilient

First snap is FULL → Next incremental (only diff is size)

RDS Backups are snapshots of the entire RDS - not only one database

Every 5 minutes Transaction Logs is written to S3

RDS Restores

• Creates a NEW RDS Instance - new address
• Snapshots = single point in time, creation time
• Automated = any 5 minute point in time
• Backup is restores and transaction logs are replayed to bring DB to desired point in time
• Restores aren’t fast - Think about RTO

RDS Read-Replicas

RDS Read Replicas can be added to an RDS Instance - 5 direct per primary instance.

They can be in the same region, or cross-region replicas.

They provide read performance scaling for the instance, but also offer low RTO recovery for any instance failure issues

N.B they don't help with data corruption as the corruption will be replicated to the RR.

Read-Replica Architecture

Writes to replica after primary write is complete. Can be accessed for read operation, unlike Standby Replica.

(read) Performance Improvements

• 5x direct read-replicas per DB instance
• Each providing an additional instance of read performance
• Read-replicas can have read-replicas - but lag starts to be a problem
• Global performance improvements

Availability Improvements

• Snapshots & Backups Improve RPO
• RTO’s are a problem
• RR’s offer nr. 0 RPO
• RR’s can be promoted quickly - low RTO
• Failure only - watch for data corruption
• Read only - until promoted
  • Not reversible - delete and create new RR
• Global availability improvements → Global resilience
• Scale READS, NOT WRITES

Amazon RDS Security

• SSL/TLS (in transit) is available for RDS, can be mandatory
• RDS supports EBS volume encryption - KMS
• Handled by HOST/EBS
• AWS or Customer Managed CMK generates data keys
• Data keys used for encryption operations
• Storage, logs, snapshots and replicas are encrypted with the same master key
  • encryption can’t be removed
• RDS MSSQL and RDS Oracle Support TDE
• TDE: Transparent Data Encryption
• Encryption handled within the DB engine
• RDS Oracle supports integration with CloudHSM
• Much stronger key controls (even from AWS)

Amazon RDS IAM Authentication

Amazon Aurora Architecture

Aurora is a AWS designed database engine officially part of RDS

Aurora implements a number of radical design changes which offer significant performance and feature improvements over other RDS database engines.

Aurora Key Differences

• Aurora architecture is VERY different from RDS
  • Uses a Cluster
• A single primary instance + 0 or more replicas
  • Replicas can read and be standby
• No local storage - uses cluster volume
• Faster provisioning and improved availability and performance

Aurora Storage Architecture

• Replication happens at storage level
• Primary is the only allowed to write to storage - other nodes can read
• In case of damage or error, data is immediately repaired
• More resilient than normal RDS
• Up to 15 different replicas to failover to
  • Quicker failover

• All SSD Based - high IOPS, low latency
• Storage is billed based on what’s used
• High water mark - billed for the most used
  • Being changed
• Storage which is freed up can be re-used
• Replicas can be added and removed without requiring storage provisioning
• Multiple endpoints
  • Cluster endpoint
  • Reader endpoint
    • Load balance across replicas
  • Custom endpoints

Cost

• No free-tier
• Aurora doesn’t support Micro Instances
• Beyond RDS singleAZ (micro) Aurora offers better value
• Compute - hourly charge, per second, 10 minute minimum
• Storage - GB-month consumed, IO cost per request
• 100% DB size in backups are included

Aurora Restore, Clone and Backtrack

• Backups in Aurora work in the same way as RDS
• Restores create a new cluster
• Backtrack can be used which allow in-place rewinds to a previous point in time
• Fast clones make a new database MUCH faster than copying all the data - copy-on-write
  • Uses a tiny amount of storage - only stores the data changed since the clone was created

Aurora Serverless

Is to Aurora what Fargate is to EC2

Aurora Serverless Concepts

• Scalable - ACU : Aurora Capacity Units
• Aurora Serverless cluster has a MIN and MAX ACU
• Cluster adjusts based on load
• Can go to 0 and be paused
• Consumption billing per-second basis
• Same resilience as Aurora (6 copies across AZs)

Aurora Serverless Architecture

Aurora Serverless: Use Cases

• Infrequently used application
• New applications
• Variable workloads
• Unpredictable workloads
• Development and test databases
• Multi-tenant applications