Database Freedom How to Migrate Your Databases to the Cloud - 20 September - 11:00

Database Freedom How to Migrate Your Databases to the Cloud - 20 September - 11:00

• 1. Slide1 © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
• 2. Slide2 © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Serdar Nevruzoglu Solutions Architect Migrate Your Databases to the Cloud
• 3. Agenda © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Agenda •Introduction to AWS Databases •AWS Database Migration Service and AWS Schema Conversion Tool •DMS/SCT Product Highlights •Migration Playbooks
• 4. Slide1531 © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Introduction to AWS Databases
• 5. Our portfolioBroad and deep portfolio, purpose-built for builders © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Our portfolio Broad and deep portfolio, purpose-built for builders Redshift Data warehousing EMR Hadoop + Spark Athena Interactive analytics Kinesis Data Analytics Real time Elasticsearch Service Operational Analytics RDS MySQL, PostgreSQL, MariaDB, Oracle, SQL Server Aurora MySQL, PostgreSQL QuickSight SageMaker DynamoDB Key value ElastiCache Redis, Memcached Neptune Graph Timestream Time Series QLDB Ledger Database S3/Glacier Glue ETL & Data Catalog Lake Formation Data Lakes Database Migration Service | Snowball | Snowmobile | Kinesis Data Firehose | Kinesis Data Streams Data Movement Analytics Databases Business Intelligence & Machine Learning Data Lake Managed Blockchain Blockchain Templates Blockchain RDS on VMware NEW NEW NEW NEW NEW DocumentDB Document NEW
• 6. Three type of projects © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Migrate existing apps to the cloud Quickly build new apps in the cloud Gain new insights Three type of projects
• 7. Migrate existing apps to the cloud © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Migrate existing apps to the cloud Quickly build new apps in the cloud Gain new insights Migrate existing apps to the cloud
• 8. Challenges with on-premises deployments © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Challenges with on-premises deployments Large CapEx investments needed in data centers and servers IT resources needed to rack, stack, power servers Overbuying and guessing on future capacity needs Manual maintenance (software upgrades, etc.)
• 9. Old world commercial relational databases © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Old world commercial relational databases Lock-in Proprietary Punitive licensing Very expensive You’ve got mail #DBfreedom
• 10. Customers want to migrate to the cloud © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Customers want to migrate to the cloud Relational databases Non-relational databases Data warehouses Hadoop and Spark Redshift EMR Operational analytics Elasticsearch Service Aurora DynamoDB Business Intelligence QuickSight RDS DocumentDB
• 11. AWS purpose-built databases © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS purpose-built databases Relational Key-value Document In-memory Graph Time-series Ledger DynamoDB Neptune Amazon RDS Timestream QLDB ElastiCache Redis Memcached DocumentDB Aurora Commercial Community
• 12. Amazon Aurora © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Aurora MySQL and PostgreSQL-compatible relational database built for the cloud Performance and availability of commercial-grade databases at 1/10th the cost Performance and scalability Availability and durability Highly secure Fully managed 5x throughput of standard MySQL and 3x of standard PostgreSQL; scale-out up to 15 read replicas Fault-tolerant, self-healing storage; six copies of data across three Availability Zones; continuous backup to Amazon S3 Network isolation, encryption at rest/transit Managed by RDS: No hardware provisioning, software patching, setup, configuration, or backups q
• 13. Amazon Relational Database Service (RDS) © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Relational Database Service (RDS) Managed relational database service with a choice of six popular database engines Easy to administer Available and durable Highly scalable Fast and secure No need for infrastructure provisioning, installing, and maintaining DB software Automatic Multi-AZ data replication; automated backup, snapshots, failover Scale database compute and storage with a few clicks with no app downtime SSD storage and guaranteed provisioned I/O; data encryption at rest and in transit
• 14. Amazon DynamoDBFast and flexible key value database service for any scale © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon DynamoDB Fast and flexible key value database service for any scale Performance at scale Consistent, single-digit millisecond response times at any scale; build applications with virtually unlimited throughput Comprehensive security Encrypts all data by default and fully integrates with AWS Identity and Access Management for robust security Global database for global users and apps Build global applications with fast access to local data by easily replicating tables across multiple AWS Regions Serverless No server provisioning, software patching, or upgrades; scales up or down automatically; continuously backs up your data
• 15. Amazon DocumentDBFast, scalable, highly available MongoDB-compatible database © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon DocumentDB Fast, scalable, highly available MongoDB-compatible database Performance at scale 2X the throughput of currently available MongoDB managed services; scale read capacity to millions of requests per second by adding up to 15 low latency read replicas across three AZs in minutes, regardless of data size Highly Available Automatic Multi-AZ data replication; automated backup, snapshots, failover; MongoDB-compatible Implements the Apache 2.0 open source MongoDB 3.6 API by emulating the responses that a MongoDB client expects from a MongoDB server Durable Fault-tolerant, self-healing storage; six copies of data across three Availability Zones; continuous backup to Amazon S3
• 16. Amazon RedshiftHighly scalable cloud data warehouse at 10x the performance and 1/10th the cost of traditional data warehouses © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Redshift Highly scalable cloud data warehouse at 10x the performance and 1/10th the cost of traditional data warehouses Virtually unlimited concurrency Extends your data lake Dynamically scales to support virtually unlimited number of concurrent users and growing data volumes Analyze exabytes of data in the Amazon S3 data lake together with petabytes of data loaded into Amazon Redshift’s high performance SSDs 10x performance 1/10th the cost Get faster time-to-insight for all types of analytics workloads; powered by machine learning, columnar storage and MPP Start at $0.25 per hour, scale out as low as $1,000 per terabyte per year
• 17. Customers are migrating their workloads to AWS © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Customers are migrating their workloads to AWS Verizon is migrating over 1,000 business-critical applications and database backend systems to AWS, several of which also include the migration of production databases to Amazon Aurora. Wappa migrated from their Oracle database to Amazon Aurora and improved their reporting time per user by 75 percent. Trimble migrated their Oracle databases to Amazon RDS and project they will pay about 1/4th of what they paid when managing their private infrastructure. Intuit migrated from Microsoft SQL Server to Amazon Redshift to reduce data-processing timelines and get insights to decision makers faster and more frequently. Equinox Fitness migrated its Teradata on-premises data warehouse to Amazon Redshift. They went from static reports to a modern data lake that delivers dynamic reports. Eventbrite moved from Cloudera to Amazon EMR and were able to cut costs dramatically, spinning clusters up/down on-demand and using Spot (saving > 80%) and Reserved Instances. By December 2018, Amazon.com have migrated 88% of their Oracle DBs (and 97% of critical system DBs) moved to Amazon Aurora and Amazon DynamoDB. They also migrated their 50 PB Oracle Data Warehouse to AWS (Amazon S3, Amazon Redshift, and Amazon EMR). Samsung Electronics migrated their Cassandra clusters to Amazon DynamoDB for their Samsung Cloud workload with 70% cost savings.
• 18. Slide1517 © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. CHALLENGE Needed to analyze data to find insights, identify opportunities, and evaluate business performance. The Oracle DW did not scale, was difficult to maintain, and costly. SOLUTION Deployed a data lake with Amazon S3, and run analytics with Amazon Redshift, Amazon Redshift Spectrum, and Amazon EMR. Result: They doubled the data stored (100PB), lowered costs, and was able to gain insights faster. To add an image, select Click to insert image, and find the image you want to use. 50 PB of data 600,000 analytics jobs/day
• 19. Equinox Fitness migrated from Teradata to Redshift © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Equinox Fitness migrated from Teradata to Redshift Migrated from Teradata data warehouse Built a DW with Redshift and data lake with S3 Analytics on data lake with Amazon Athena, Amazon Redshift Spectrum, and Amazon EMR Increased user productivity to move faster Amazon Redshift costs ~20% of its original Teradata maintenance & support Maximilian (ELT scripts) Spark on EMR Redshift S3 Clickstream Cycling logs Club management software Applications Social Redshift Spectrum EMR Athena Equinox apps 3rd party apps
• 20. How can I get to the cloud? © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. How can I get to the cloud? •How will my on-premises data migrate to the cloud? •How can I make it transparent to my users? •Afterwards, how will on-premises and cloud data interact? •How can I integrate my data assets within AWS? •Can I get help moving off of commercial databases?
• 21. Migration used to be cost + complexity + time © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Migration used to be cost + complexity + time •Commercial data migration and replication software •Complex to set up and manage •Application downtime •Database-engine-specific application code
• 22. Migration options © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Migration options If you’re not switching engines and can take downtime: -SQL Server: bak file import -MySQL: read replicas -Oracle SQL Developer, Data Pump, Export/Import -PostgreSQL: pg_dump -SAP ASE: bcp
• 23. Slide1532 © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Database Migration Service AWS Schema Conversion Tool
• 24. What are DMS and SCT? © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. What are DMS and SCT? Our goal: Allow customers the freedom to choose the best data platform for their needs #DBFreedom AWS Schema Conversion Tool converts your commercial database and data warehouse schemas to open-source engines or AWS-native services, such as Amazon Aurora and Amazon Redshift AWS Database Migration Service easily and securely migrates and/or replicate your databases and data warehouses to AWS
• 25. AWS Database Migration Service © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Database Migration Service MIGRATING DATABASES TO AWS Migrate between on-premises and AWS Migrate between databases Automated schema conversion Data replication for zero downtime migration https://youtu.be/ouia1Sc5QGo
• 26. 160,000+ databases migrated with DMS © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. 160,000+ databases migrated with DMS
• 27. Slide73 © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Schema Conversion Tool (AWS SCT)
• 28. When to use DMS and SCT? © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. When to use DMS and SCT? Modernize Migrate Replicate
• 29. AWS Schema Conversion Tool © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Schema Conversion Tool •Modernize your data warehouse Convert your Oracle, SQL Server, Netezza, Greenplum, Vertica or Teradata to Amazon Redshift Modernize •Modernize your database Convert your Oracle, SQL Server, or Db2 LUW to PostgreSQL, MySQL, or Amazon Aurora Amazon Aurora Amazon Redshift MySQL PostgreSQL PostgreSQL
• 30. AWS Database Migration Service © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Database Migration Service Migrate •Migrate business-critical applications •Migrate data warehouse to Amazon Redshift •Upgrade to a minor version •Consolidate shards into Aurora •Archive old data •Migrate from NoSQL to SQL, SQL to NoSQL or NoSQL to NoSQL Amazon RDS Amazon Redshift Amazon Aurora Amazon DynamoDB Amazon S3
• 31. Slide1309 © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Customer premises AWS VPN ®Start a replication instance ®Connect to source and target databases ®Select tables, schemas, or databases ®Let AWS DMS create tables, load data, and keep them in sync ®Switch applications over to the target at your convenience AWS DMS Keep your apps running during the migration Application users Internet
• 32. AWS Database Migration Service © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Database Migration Service Replicate •Create cross region Read Replicas •Run your analytics in the cloud •Populate your data lake
• 33. Slide1533 © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS DMS/SCT Highlights
• 34. AWS DMS/SCT Highlights © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS DMS/SCT Highlights Secure Verify Validate Homogeneous & Heterogeneous Assess Convert Schema & Code
• 35. AWS DMS Highlights © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS DMS Highlights •Checks migration task settings prior to launch •Alerts for potential issues, such as unsupported data types •Prevent unnecessary, time consuming runtime migration failures Pre-migration Assessment •Validates that all data selected for migration migrated properly •Includes both stages of full load and CDC •Applies for both homogeneous and heterogeneous migrations Data Validation Use AWS Snowball & AWS DMS to migrate data to AWS •Migrate large databases (over 5TB) •Migrate many databases at once •Migrate over slow network •Push vs. Pull Snowball Integration
• 36. AWS DMS – Endpoint Support Expansion © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS DMS – Endpoint Support Expansion MariaDB Sybase ASE Db2 LUW Azure SQL MariaDB Sybase ASE MongoDB Cassandra Sources Targets Relational NoSQL Data Warehouse Data Lake Amazon Redshift Amazon DynamoDB Amazon S3 Amazon S3 Amazon Aurora Amazon Aurora Oracle SQL Server Netezza Greenplum Vertica Teradata AWS Snowball Edge Amazon ES Amazon Kinesis
• 37. AWS SCT Highlights © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS SCT Highlights •Assessment of migration compatibility of source databases with open-source database engines – Amazon RDS for MySQL, Amazon RDS for PostgreSQL, and Amazon Aurora •Recommends best target engine •Provides details level of efforts to complete migration •Attempts to convert all schema and code objects to the target engine, including stored procedures and functions •Scans and converts embedded SQL statements in app code •Generates a report with recommendations Converts Schema and Code •Extracts data through local migration agents •Files are loaded to an Amazon Simple Storage Service (Amazon S3) bucket and to Amazon Redshift •Netezza •Vertica •Greenplum •Teradata •Oracle •SQL Server Extracts and Migrates DW to Amazon Redshift Assessment Report
• 38. Slide1534 © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Migration Playbooks
• 39. Slide208 © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. •Topic-by-topic overview of how to migrate databases and data warehouses to AWS services •Covers all proprietary features and the different database objects •Migration best practices •Oracle to Aurora PostgreSQL •SQL Server to Aurora MySQL •SQL Server to Aurora PostgreSQL Old World to Aurora Migration Playbooks To download the playbooks, please visit: https://aws.amazon.com/dms/resources/
• 40. Need help to migrate your databases to AWS?  © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Need help to migrate your databases to AWS? [Tools] [Programs] database freedom [Partners]
• 41. Slide1536 © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. serdarn@amazon.com.tr @sernevr Thank You!