Cookies on this site

We would like to use cookies to understand how the site is used and to measure our campaigns. Nothing is set until you choose, and you can change your mind at any time from “Cookie settings” at the foot of every page. Privacy policy

Contact UsContact Us

AWS Database Migration Service, set up so replication keeps up and the data matches

AWS DMS copies a database to AWS and then keeps applying changes until you are ready to switch, which is what makes a short cut-over possible. It is also easy to start and hard to finish: tasks fall behind, large objects stall, and a table nobody mapped arrives empty. We design the replication, tune it on your data and prove the target matches before anyone repoints an application.

How DMS works

A full copy, then every change after it

A DMS migration runs on a replication instance in your AWS account. Its tasks read from a source endpoint and write to a target endpoint in two phases: a full load of the existing tables, then change data capture, which reads the source's transaction log and applies each insert, update and delete to the target. While change data capture runs, the old system stays live and the new one stays seconds or minutes behind it.

Source and target do not have to be the same engine. DMS moves Oracle, SQL Server, PostgreSQL, MySQL and other databases to Amazon RDS, Aurora, Redshift or S3. When the engine changes, the schema is converted first, tables, indexes, views and stored code, with AWS Schema Conversion Tool or DMS Schema Conversion, and code that does not convert automatically is rewritten by hand.

What we do

  • A replication instance sized for your change volume
  • Source and target endpoints, with the logging that change capture needs
  • Schema conversion for a change of engine, and hand-written fixes for what does not convert
  • Full load and ongoing replication in one task, or in stages

Where DMS projects stall

Lag, large objects and tables nobody mapped

Most DMS trouble comes from a few places. Replication lag grows when the instance is too small, when one task carries too many busy tables, or when indexes and triggers slow the target during the full load. Large objects, such as documents stored in the database, need their own settings or they hold up every table around them. Mapping rules with wildcards include or exclude more than intended. And tables without a primary key cannot be kept in step reliably by change capture.

We find these on a rehearsal rather than on the night. We split busy tables across tasks, size the instance from measured throughput, load before building secondary indexes, choose the large-object mode table by table, and check every mapping rule against the list of tables that must arrive. Task logs and Amazon CloudWatch metrics show the lag, and alarms say when it grows.

What we do

  • Throughput and lag measured on a rehearsal run
  • Busy tables split across parallel tasks
  • Large-object settings chosen table by table
  • Mapping rules checked against the tables that must arrive

Proving the data

Validation before the cut-over, not after it

DMS can compare source and target rows as it migrates and report every mismatch. We turn validation on, add our own checks on row counts, totals and the records the business cares most about, and fix the causes of any difference before the cut-over date is set.

On the night, the applications stop writing to the source, replication drains until there is no lag, the last validation runs and the applications are pointed at the new database. The source stays as it was until the business signs off, so the way back is still there if it is needed. Some clients keep a task running afterwards, to feed a reporting copy in Amazon Redshift or S3 from the live system.

What we do

  • DMS data validation on every migrated table
  • Business checks on totals and key records
  • Cut-over with no lag, and the source kept for fallback
  • Ongoing replication to Redshift or S3 for reporting, where it is needed

When to use something else

DMS, native tools or GoldenGate

DMS is not always the right tool. For a one-time move to the same engine with an outage window that fits, native tools such as Oracle Data Pump or a SQL Server backup are simpler; our AWS RDS migration page describes that route. For large Oracle estates that need very low lag or replication in both directions, Oracle GoldenGate is often the better fit. It is how we moved a 30 TB Oracle database for a global retail bank with zero downtime, and continuous replication is how an e-discovery company moved from Sybase to Oracle without interrupting its business.

We recommend the tool after looking at your data, your outage limit and your budget, and we say so when the simpler route is enough.

Case studies

AWS Database Migration Service in practice

Client names are withheld. We are happy to share details on a call.

All case studies
  • Delivered

    Zero downtime

    Legal technology

    Cloud Consulting

    Sybase to Oracle with zero downtime for an e-discovery company

    Sybase was holding back a platform that processes large volumes of legal data. We converted the schema, kept both databases in sync with continuous replication, validated the data and switched over without interrupting the business.

    • Zero downtime at cut-over
    • No disruption to client workflows
  • Delivered

    30 TB, zero downtime

    Financial services

    Cloud Consulting

    Zero-downtime move of a 30 TB Oracle database for a global retail bank

    The bank's transaction database served thousands of users around the clock. We built the new Oracle 12c RAC platform on IBM AIX alongside the old Solaris system, kept the two in sync with Oracle GoldenGate, tested everything and switched over with no downtime and no data loss.

    • 30 TB moved with zero downtime
    • Zero data loss at cut-over
  • Delivered

    On-prem → Amazon RDS

    Nonprofit

    Cloud Consulting

    Oracle from on-premises to Amazon RDS for a national nonprofit

    The nonprofit's on-premises Oracle database had become hard to maintain and scale. We planned the move with automated workflows and validation tests, migrated to Amazon RDS for Oracle with minimal disruption and left a repeatable process for future changes.

    • Automated, tested migration
    • Less day-to-day database upkeep

Questions

Common questions

Does AWS DMS mean zero downtime?

It means very little downtime. The source stays live while DMS copies it and applies changes, so the outage is the time it takes to stop writes, let replication catch up and repoint the applications, often minutes rather than hours. That figure is measured in rehearsal, on your data, before a date is set.

Can DMS change the database engine, for example Oracle to PostgreSQL?

It moves the data between engines, but it does not convert stored code. The schema is converted first with AWS Schema Conversion Tool or DMS Schema Conversion, and procedures, functions and triggers that do not convert are rewritten and tested. That work, not the data, usually decides how long the project takes.

What does DMS cost to run?

You pay AWS for the replication instance while it runs, for its storage and for data transfer where it applies, and there is a serverless option that scales capacity with the load. We size the instance from a rehearsal so you are not paying for idle capacity, and we shut it down when the migration is finished unless you want ongoing replication.

Can DMS keep replicating after the migration?

Yes. A task can run indefinitely, which some clients use to keep a reporting copy in Amazon Redshift or S3 up to date from the live system. We set up monitoring and lag alarms so a stalled task does not go unnoticed.

Our database is on premises. Does it have to be opened to the internet?

No. The replication instance reaches your database over a VPN or AWS Direct Connect, using a database account that has only the rights DMS needs, and traffic can be encrypted in transit.

From the blog

Writing on AWS Database Migration Service

All Cloud & apps articles

Related services

On-site IT support across New Jersey, New York City and Philadelphia. Remote services wherever you are. Areas we serve →

Planning a move with a short outage window?

Tell us the source, the target and how long you can be offline. We will tell you whether DMS fits and what a rehearsal would show.

Tell us what you are working on

A few lines is enough. The engineers who do this work will reply within one business day.

Or email it.services@princetonits.com or call +1 (212) 507-9350.