æ¬æçš¿ã¯ãVanshika Nigam ã«ããèšäº ã Improve AWS DMS continuous replication performance by using column filters to parallelize high-volume tables ãã翻蚳ãããã®ã§ãã ããŒã¿ããŒã¹ã®ç§»è¡ã¯é£ãããç¹ã«ãæ°ååä»¶ã®ãšã³ããªãŒãæã€å·šå€§ãªããŒãã«ã§ãé »ç¹ã«æŽæ°ãè¡ãããå Žåã¯å°æŽã§ãã AWS Database Migration Service (AWS DMS) ã¯ã 䞊åããŒã ã ãã£ã«ã¿ãªã³ã°æ©èœ ãªã©ãåæã®äžæ¬ããŒã¿è»¢éããã»ã¹ã倧å¹
ã«å éã§ãã䟿å©ãªæ©èœãæäŸããŠããŸãã ãããããœãŒã¹ããŒã¿ããŒã¹ã§æ¥éãã€ç¶ç¶çãªå€æŽãè¡ãããé«é »åºŠã®å€æŽããŒã¿ãã£ãã㣠(CDC) ã·ããªãªã§ã¯ãç¹å®ã®èª²é¡ãæ®ã£ãŠããŸãã ãã®ãããªã·ããªãªã§ã¯ãã¿ãŒã²ããããŒã¿ããŒã¹ã¯æ¬¡ã®ãããªèª²é¡ã«é »ç¹ã«çŽé¢ããŸãã é«ã¬ã€ãã³ã·ãŒ â CDC ããã»ã¹ã倿Žã®ããŒã¹ã«è¿œãã€ããªãå¯èœæ§ããããŸãã倧éã®ç¶ç¶çãªå€æŽã«ããããœãŒã¹ãšã¿ãŒã²ããã®ããŒã¿ããŒã¹éã§å€§å¹
ãªé
å»¶ãçããCDC ã®ããã©ãŒãã³ã¹ã«åœ±é¿ãäžããå¯èœæ§ããããŸãã ããŒã¿æå€±ã®ãªã¹ã¯ â Oracle ãªã©ã®ããŒã¿ããŒã¹ã§ã¯ãAWS DMS ãåŠçããåã«ãredo ãã°ãã¢ãŒã«ã€ããã°ãæ¶å»ãããå¯èœæ§ããããããŒã¿æå€±ã«ã€ãªããæãããããŸãã ä¿ææéã®å¶é â ã¹ãã¬ãŒãžã®å¶çŽãçµç¹ã®ããªã·ãŒã«ãããä¿ææéãå»¶é·ããããšãåžžã«å¯èœãšããããã§ã¯ãããŸããã ãªãœãŒã¹ã®è² è· â é«é »åºŠã® CDC ã«ãããã¿ãŒã²ããããŒã¿ããŒã¹ãš AWS DMS ã¬ããªã±ãŒã·ã§ã³ã€ã³ã¹ã¿ã³ã¹ã®äž¡æ¹ã«è² è·ãããããã¬ã€ãã³ã·ãŒã®å¢å ãã¹ã«ãŒãããã®äœäžããªãœãŒã¹æ¶è²»ã®å¢å ã«ã€ãªããå¯èœæ§ããããŸãã ãããã®èª²é¡ã«å¯ŸåŠããã«ã¯ãéåžžã é©åãªãµã€ãºã®ã¬ããªã±ãŒã·ã§ã³ã€ã³ã¹ã¿ã³ã¹ãéžæãã ã ã¿ã¹ã¯ã®äžŠåå ã AWS DMS ã®ãããé©çšæ©èœã䜿çšãã ãAWS DMS ã¿ã¹ã¯èšå®ãæé©åãããå¹ççãªãã£ã«ã¿ãªã³ã°ããããŠæã«ã¯ç¬èªã®ãœãªã¥ãŒã·ã§ã³ãªã©ãå€é¢çãªã¢ãããŒããå¿
èŠã«ãªããŸãã ãªã¬ãŒã·ã§ãã«ããŒã¿ããŒã¹ã®ã¿ãŒã²ããã§ã¯ããã«ããŒã䞊åèšå®ãšã¯ç°ãªããããã©ãŒãã³ã¹åäžã®ããã®äžŠå CDC ã¹ã¬ããã䜿çšãããªãã·ã§ã³ã¯ãããŸãããåã¿ã¹ã¯ã¯åäžã®ã¹ã¬ããã䜿çšããŸããAWS DMS ã® CDC ãå«ãã¿ã¹ã¯ãæ°çŸäžã®å€æŽã€ãã³ããé©çšããå¿
èŠãããå Žåã¯ãè«ççã«ç¬ç«ããããŒãã«ã°ã«ãŒãã«å¯ŸããŠè€æ°ã® CDC ã¿ã¹ã¯ã䜿çšããããšããå§ãããŸããããã«ãããè€æ°ã® CDC ã¹ã¬ããã䜿çšã§ããŸããããã«ãé©åãªã«ã©ã ãã£ã«ã¿ãŒã䜿çšããŠãå€§èŠæš¡ãªããŒãã«ãè€æ°ã® CDC ã¿ã¹ã¯ã«åå²ããããšãã§ããŸãã ãã®æçš¿ã§ã¯ãCDC ãã§ãŒãºã§ã¢ã¯ãã£ããã£ã®å€ãããŒãã«ãè€æ°ã®ã¿ã¹ã¯ã«åå²ããããã« åãã£ã«ã¿ãŒ ã䜿çšããæ¹æ³ã«ã€ããŠèª¬æããŸãã ãã®ææ³ã«ãããç§»è¡ããã»ã¹ãå éããã¿ãŒã²ããã®ã¬ã€ãã³ã·ãŒãåæžã§ããŸãã ãœãªã¥ãŒã·ã§ã³ã®æŠèŠ AWS DMS ã® ã¿ã¹ã¯ ã¯ç§»è¡ããã»ã¹ã®äžå¿ãšãªããã®ã§ãã ããŒãã«ãããã³ã° ãéããŠãç§»è¡å¯Ÿè±¡ã®ç¹å®ã®ããŒãã«ããã¥ãŒãã¹ããŒããå®çŸ©ããŸãã ãã®ãããã³ã°ã«ã¯ããã£ã«ã¿ãªã³ã°æ©èœãããã WHERE å¥ãé©çšããããšã§ãè¡æ°ãå¶éãããã倧ããªããŒãã«ã管çå¯èœãªå°ããªã»ã°ã¡ã³ãã«åå²ããããšãã§ããŸãã ãã®æ©èœã¯åŸæ¥ããã«ããŒãæäœã«é©çšãããŠããŸãããããã®ãœãªã¥ãŒã·ã§ã³ã§ã¯ãCDC ã¿ã¹ã¯ã®æé©åã«ãããå¯èœæ§ãæ¢ããŸãã ãã®æçš¿ã§ã¯ãOracle ãã PostgreSQL ãžã®ç§»è¡ã«ãã®ãœãªã¥ãŒã·ã§ã³ãå®è£
ããŸãã åãã¢ãããŒããä»ã®ãªã¬ãŒã·ã§ãã«ããŒã¿ããŒã¹éã®ç§»è¡ã«ãé©çšã§ããŸãã ãã®ãœãªã¥ãŒã·ã§ã³ãå®è£
ããæé ã¯æ¬¡ã®ãšããã§ãã Oracle ããŒã¿ããŒã¹ã«ãµã³ãã«ããŒãã«ãäœæã (ãã®æçš¿ã§ã¯ã Amazon RDS for Oracle ã䜿çš)ã代衚çãªããŒã¿ãå
¥åããŸãã ããŒã¿ããŒã¹ãšããŒãã«ã¬ãã«ã§ãµããªã¡ã³ã¿ã«ãã®ã³ã°ã確èªããŠæå¹åããŸãã è¡ãã¿ã¹ã¯éã§åçã«åæ£ããã®ã«é©ããäžå€ã«ã©ã ãç¹å®ããŸããäžå€ã«ã©ã ã®è©³çްã«ã€ããŠã¯ã次ã®ã»ã¯ã·ã§ã³ãåç
§ããŠãã ããããã®ãããªåãååšããªãå Žåã¯ãæ¬¡ã®æé ãå®è¡ããŸããããã§ãªãå Žåã¯ãã¹ããã 4 ã«é²ãã§ãã ããã äžæçãªäžå€ã«ã©ã ã NUMBER(1) ããŒã¿åã§è¿œå ããŸãã æ¢åã®ããŒã¿ã«ã€ããŠã¯ãäž»ããŒã«å¯Ÿããå°äœæŒç®ã䜿çšããŠãæ°ããäœæããäžå€ã«ã©ã ã«å€ãå
¥åããŸãããã®æçš¿ã§ã¯ã 3 ã§å²ã£ãå°äœã䜿çšããäŸã瀺ããŸãããã®æäœã«ãããããŒã¿ã 3 ã€ã®ç°ãªãã°ã«ãŒãã«å¹æçã«åå²ãããåã°ã«ãŒãã¯äž»ããŒã 3 ã§å²ã£ãå°äœã§èå¥ãããŸãã æ°ããäžå€ã«ã©ã ã«å¯ŸããŠãµããªã¡ã³ã¿ã«ãã®ã³ã°ãæå¹åããŸãã æ°èŠæ¿å
¥æã«èªåçã«å°äœåãå
¥åããããªã¬ãŒãäœæããŸãã å°äœæŒç®ã®çµæã«åºã¥ããŠãåãã£ã«ã¿ãŒã䜿çšããŠè€æ°ã® AWS DMS ã¿ã¹ã¯ãäœæããŸãããããã¯ãèŠä»¶ã«å¿ããŠããã«ããŒããš CDC ã¿ã¹ã¯ãŸã㯠CDC ã®ã¿ã®ã¿ã¹ã¯ã®ããããã«ãªããŸãããã®æçš¿ã§ã¯ã3 ã§å²ã£ãå°äœã«åãã㊠CDC ã®ã¿ã® 3 ã€ã®ã¿ã¹ã¯ãäœæããŸãã ç§»è¡äžã«ã¿ãŒã²ããã§äžæçãªäžå€ã«ã©ã ãé€å€ããããã«ãAWS DMS ã¿ã¹ã¯ã«é€å€åãã£ã«ã¿ãŒã远å ããããšãå¿ããªãã§ãã ããã éžæããã¿ãŒã²ããããŒã¿ããŒã¹ (ãã®æçš¿ã§ã¯ã Amazon Aurora PostgreSQL äºæãšãã£ã·ã§ã³ ãããŒã¿ããŒã¹ãšããŠäœ¿çš) ã«ç§»è¡ããããŒã¿ãæ€èšŒããŸãã Amazon CloudWatch ã䜿çšã㊠AWS DMS ã¿ã¹ã¯ã®ããã©ãŒãã³ã¹ãç£èŠããŸãã æ¬¡ã®å³ã¯ããã®ãœãªã¥ãŒã·ã§ã³ã®ã¢ãŒããã¯ãã£ã瀺ããŠããŸãã ããŒã¿ããŒã¹ã®ã€ãã¥ãŒã¿ãã«ã«ã©ã ã®ç¹å®æ¹æ³ ã€ãã¥ãŒã¿ãã«ã«ã©ã ãšã¯ãäžåºŠèšå®ãããå€ã倿Žã§ããªãã«ã©ã ã®ããšã§ãããã¥ãŒã¿ãã«ã«ã©ã ãšã¯ãåææ¿å
¥åŸã«å€ã倿Žã§ããã«ã©ã ã®ããšã§ãã äž»ããŒã¯äžå€ã§ãããåè¡ã«åºæã®ãã®ã§ããããããç§ãã¡ã®çŠç¹ã¯ãè€æ°ã®è¡ã§åãå€ãæã€å¯èœæ§ã®ããäžå€ã®ã«ã©ã ãç¹å®ããããšã§ããããã«ãããããŒã¿ã®åå²ãåçã«ãªããAWS DMS ã¿ã¹ã¯éã§è² è·ãå¯èœãªéãåçã«åæ£ãããŸããäŸãšããŠã¯ãäœæã¿ã€ã ã¹ã¿ã³ããã«ããŽãªã³ãŒãããšãªã¢ã³ãŒããéšéèå¥åãªã©ãæããããŸãããã®ãããªã«ã©ã ã䜿çšããããšã§ãããŒã¿ãããåçã«åå²ã§ããå¹ççãªã¬ããªã±ãŒã·ã§ã³ãšãããŒã¿æŽåæ§ã®ç¶æãå¯èœã«ãªããŸãã éèŠãªã®ã¯ã倿°ã®ã¬ã³ãŒãã«é¢é£ããªãããäžå®ã®å€ãä¿ã€ã«ã©ã ãç¹å®ãããããããŒãã£ã·ã§ã³åå²ã®åºæºãšããŠäœ¿ãããšã§ãããŒã¿ã®åçãªååžãå®çŸããããšã§ãã åææ¡ä»¶ å§ããã«ã¯ã次ã®åææ¡ä»¶ãæºããå¿
èŠããããŸãã ã¢ã¯ãã£ã㪠AWS ã¢ã«ãŠã³ã ã Oracle ã€ã³ã¹ã¿ã³ã¹ (ãã®æçš¿ã§ã¯ Amazon RDS for Oracle ã䜿çš) ãŸãã¯ããªã³ãã¬ãã¹ã® Oracle Databaseã Aurora PostgreSQL ãŸã㯠RDS for PostgreSQL ããŒã¿ããŒã¹ (ãã®æçš¿ã§ã¯ Aurora PostgreSQL äºæã䜿çš)ããŸã Aurora PostgreSQL ã¯ã©ã¹ã¿ãŒãæã£ãŠããªãå ŽåãäœæããŸããæé ã«ã€ããŠã¯ã Amazon Aurora DB ã¯ã©ã¹ã¿ãŒã®äœæ ãåç
§ããŠãã ããã ãœãŒã¹ãšã¿ãŒã²ããã®äž¡æ¹ã®ããŒã¿ããŒã¹ã«ã¢ã¯ã»ã¹ã§ãã ã¬ããªã±ãŒã·ã§ã³ã€ã³ã¹ã¿ã³ã¹ ã詳现ã«ã€ããŠã¯ã AWS Database Migration Service ã®æŠèŠ ãåç
§ããŠãã ããã ãœãŒã¹ã§ãµã³ãã«ããŒãã«ã®æ§ç¯ ãã®èšäºã§ã¯ãRDS for Oracle ã®ããŒã¿ããŒã¹ã䜿çšããŠã次㮠DDL ã§ TRAVEL_INFO ãšããååã®ãµã³ãã«ããŒãã«ãäœæããŸãã ããŒãã« TRAVEL_INFO ãäœæããŸã: CREATE TABLE travel_info ( travel_id NUMBER PRIMARY KEY, traveler_name VARCHAR2(100), destination VARCHAR2(100), travel_mode VARCHAR2(10), travel_date DATE ); TRAVEL_INFO ããŒãã«ã«ä»£è¡šçãªããŒã¿ãæ¿å
¥ããŸã: INSERT INTO travel_info (travel_id, traveler_name, destination, travel_mode, travel_date) VALUES (1, 'John Doe', 'Paris', 'air', TO_DATE('2025-02-15', 'YYYY-MM-DD')); INSERT INTO travel_info (travel_id, traveler_name, destination, travel_mode, travel_date) VALUES (2, 'Jane Smith', 'Venice', 'water', TO_DATE('2025-03-20', 'YYYY-MM-DD')); INSERT INTO travel_info (travel_id, traveler_name, destination, travel_mode, travel_date) VALUES (3, 'Mike Johnson', 'Inca Trail', 'land', TO_DATE('2025-04-10', 'YYYY-MM-DD')); INSERT INTO travel_info (travel_id, traveler_name, destination, travel_mode, travel_date) VALUES (4, 'Emily Chen', 'Tokyo', 'land', TO_DATE('2025-05-22', 'YYYY-MM-DD')); INSERT INTO travel_info (travel_id, traveler_name, destination, travel_mode, travel_date) VALUES (5, 'Carlos Rodriguez', 'Amazon River', 'water', TO_DATE('2025-06-15', 'YYYY-MM-DD')); INSERT INTO travel_info (travel_id, traveler_name, destination, travel_mode, travel_date) VALUES (6, 'Sarah Thompson', 'Camino de Santiago', 'air', TO_DATE('2025-07-01', 'YYYY-MM-DD')); INSERT INTO travel_info (travel_id, traveler_name, destination, travel_mode, travel_date) VALUES (7, 'Alex Morgan', 'Great Barrier Reef', 'water', TO_DATE('2025-08-12', 'YYYY-MM-DD')); ãœãŒã¹ Oracle ããŒã¿ããŒã¹ã®ã»ããã¢ãã æ¬¡ã®æé ã«åŸã£ãŠããœãŒã¹ã® Oracle ããŒã¿ããŒã¹ãã»ããã¢ããããŠãã ããã AWS DMS ã§ã¯ãããŒã¿ããŒã¹ã¬ãã«ã®ãµããªã¡ã³ã¿ã«ãã®ã³ã°ãæå¹ã«ããå¿
èŠããããŸããæ¬¡ã®ã¯ãšãªãå®è¡ããŠãããŒã¿ããŒã¹ã¬ãã«ã®ãµããªã¡ã³ã¿ã«ãã®ã³ã°ã確èªããŠãã ããã SELECT supplemental_log_data_min ,supplemental_log_data_pk ,supplemental_log_data_fk ,supplemental_log_data_ui ,supplemental_log_data_all FROM v$database ; æå°éã®ãµããªã¡ã³ã¿ã«ãã®ã³ã° (supplemental_log_data_min) ã NO ã®å Žåã¯ã次ã®ã¯ãšãªãå®è¡ããŠæå¹ã«ããŠãã ããã EXEC rdsadmin.rdsadmin_util.alter_supplemental_logging('ADD'); CDC ã§ã¬ããªã±ãŒã·ã§ã³ãããã¹ããŒãã®ããŒãã«ã«ã€ããŠãããŒãã«ã¬ãã«ã®ãµããªã¡ã³ã¿ã«ãã®ã³ã°ãæå¹ã«ãªã£ãŠãããã確èªããŠãã ããã SELECT owner ,log_group_name ,table_name ,DECODE(ALWAYS,'ALWAYS', 'Unconditional','CONDITIONAL', 'Conditional') always ,log_group_type FROM dba_log_groups WHERE owner IN UPPER(''); ãã©ã€ããªããŒãæã€ããŒãã«ã«ã€ããŠã¯ã次ã®ã¯ãšãªãå®è¡ããããåŸè¿°ã® AWS DMS ãšã³ããã€ã³ãèšå®ã§è¿œå ã®æ¥ç¶å±æ§ã远å ããŠããµããªã¡ã³ã¿ã«ãã®ã³ã°ãæå¹ã«ããŠãã ããã ALTER TABLE "VANSHIKA"."TRAVEL_INFO" ADD SUPPLEMENTAL LOG DATA (PRIMARY KEY) COLUMNS ; ãã®äŸã§äœ¿çšããããŒãã«ã«ã¯äžå€ã«ã©ã ããããŸããããã®ãããAWS DMS ã¿ã¹ã¯ã§ WHERE å¥ã䜿çšããããã®äžæçãªäžå€ã«ã©ã ã远å ããŸãã ALTER TABLE travel_info ADD mod_3 NUMBER(1); ãã®æ°ããåã«ãã©ã€ããªããŒã®å°äœæŒç®ã®çµæãèšå®ãããããããŒãã«ãæŽæ°ããŸãã UPDATE travel_info SET mod_3 = MOD(travel_id, 3); æ°ããåã«å€ãæ£ããèšå®ãããããšã確èªãããããããŒãã«ãåãåãããŸãã SELECT * from travel_info ; ãã®æ°ããåã«ãµããªã¡ã³ã¿ã«ãã®ã³ã°ã远å ããŸãããµããªã¡ã³ã¿ã«ãã®ã³ã°ãæå¹ã«ãªã£ãŠããªãå ŽåãAWS DMS ã¿ã¹ã¯ã¯å€±æããŸãã ALTER TABLE "VANSHIKA"."TRAVEL_INFO" add SUPPLEMENTAL LOG GROUP LogGroupTest (MOD_3) ALWAYS ; æ°ããè¡ãæ¿å
¥ããããã³ã«å°äœæŒç®ã®åãèšå®ããããªã¬ãŒãäœæããŸãã CREATE OR REPLACE TRIGGER trg_travel_info_mod3 BEFORE INSERT OR UPDATE OF travel_id ON travel_info FOR EACH ROW BEGIN :NEW.mod_3 := MOD(:NEW.travel_id, 3); END ; / ãµã³ãã«ããŒãã«ã®ã¿ãŒã²ãããžã®æ§ç¯ ãã®æçš¿ã§ã¯ã travel_id ã«ã©ã ã® Oracle ããŒã¿å NUMBER ã PostgreSQL ã®ããŒã¿å INTEGER ã«ãããã³ã°ããŸãããããã¯ãã¿ãŒã²ããã§æ£ãã粟床ãç¶æããããã§ããOracle ã® NUMBER ã«ã©ã ãã PostgreSQL ã®çæ³çãªããŒã¿åã«ã©ã ãžã®æ£ç¢ºãªããŒã¿åãããã³ã°ãè¡ãã«ã¯ã Oracle ã® NUMBER ããŒã¿åã PostgreSQL ã«å€æãã ãåç
§ããŠãã ããã ã¿ãŒã²ããã§ããŒãã«æ§é ãäœæããã«ã¯ã次ã®ã¯ãšãªãå®è¡ããŸãããœãŒã¹ã§äœæãã mod_3 ã«ã©ã ã¯å«ãŸããŠããªãããšã«æ³šæããŠãã ããã create table if not exists vanshika.travel_info ( travel_id integer not null, traveler_name character varying(100), destination character varying(100), travel_mode character varying(10), travel_date timestamp without time zone, constraint travel_info_pkey primary key (travel_id) ); ã¿ãŒã²ããã«ããŒãã«ãååšããªãå ŽåãAWS DMS 㯠AWS DMS ã¿ã¹ã¯ã§éžæãããããŒãã«æºåã¢ãŒã ã«é¢ä¿ãªããã¿ãŒã²ããã«ããŒãã«ãäœæããŸãããã®å ŽåãDMS ãäœåãªäžèŠã«ã©ã ( mod_3 ã«ã©ã ãªã©) ãå«ãããŒãã«ãäœæããŠããŸãããšã«æ³šæãå¿
èŠã§ãã AWS DMS ã䜿çšããããŒã¿ç§»è¡ ãã®ã»ã¯ã·ã§ã³ã§ã¯ãããŒã¿ãç§»è¡ããæé ã説æããŸãã AWS DMS ãšã³ããã€ã³ãã®äœæ ãœãŒã¹ãšã¿ãŒã²ããã®ããŒã¿ããŒã¹ã«å¯Ÿã㊠AWS DMS ãšã³ããã€ã³ããäœæ ããŸããAWS DMS ãšã³ããã€ã³ãã¯ãããŒã¿ã¹ãã¢ãžã®æ¥ç¶æ
å ±ãããŒã¿ã¹ãã¢ã®çš®é¡ãå ŽæãæäŸããŸãã Oracle ãœãŒã¹ãšã³ããã€ã³ããäœæããæé ã«ã€ããŠã¯ã AWS DMS ã§ã® Oracle ããŒã¿ããŒã¹ã®ãœãŒã¹å©çš ãåç
§ããŠãã ããã ããã©ã«ãã®æ¥ç¶èšå®ã«å ããŠããªãã·ã§ã³ã§ AddSupplementalLogging=true ãšã³ããã€ã³ãèšå®ã远å ãããšããã®æçš¿ã®ååã§èª¬æããã¯ãšãªãå®è¡ããŠãµããªã¡ã³ã¿ã«ãã®ã³ã°ãæç€ºçã«è¿œå ããŠããªãå Žåã«ãAWS DMS ã Oracle ããŒã¿ããŒã¹ã«ããŒãã«ã¬ãã«ã® ãµããªã¡ã³ã¿ã«ãã®ã³ã° ãèšå®ããŸãã PostgreSQL ã¿ãŒã²ãããšã³ããã€ã³ããäœæããæé ã«ã€ããŠã¯ã AWS Database Migration Service ã®ã¿ãŒã²ãããšã㊠PostgreSQL ããŒã¿ããŒã¹ã䜿çšãã ãåç
§ããŠãã ããã AWS DMS ã¿ã¹ã¯ã®äœæ æåã« RDS for Oracle ãã Aurora PostgreSQL äºæããŒã¿ããŒã¹ãžã®å¥åã®ãã«ããŒãã¿ã¹ã¯ã䜿çšããŠãåæããŒã¿ç§»è¡ãå®è¡ããããšã«æ³šæããŠãã ããããã«ããŒããå®äºããåŸããã®ããã°ã® ãœãŒã¹ Oracle ããŒã¿ããŒã¹ã®ã»ããã¢ãã ã»ã¯ã·ã§ã³ã§èª¬æãããŠããããã«ããã£ã«ã¿ãªã³ã°ã«äœ¿çšããæ°ããåããœãŒã¹ã«è¿œå ããŸããã æ¬¡ã®ã¹ããããšããŠããã£ã«ã¿ãŒãšã㊠mod_3 ã䜿çšããŠããŒãã«ã 3 ã€ã®ã¿ã¹ã¯ã«åå²ããŸãã以äžã® JSON ã¯ãAWS DMS ã¿ã¹ã¯ã®ãããã³ã°ã«ãŒã«ã®äŸã瀺ããŠããŸããå°äœåãã¿ãŒã²ããããé€å€ããããã«ã remove-column ãã£ã«ã¿ãŒã远å ããããšãå¿ããã«ã3 ã€ã®å¥ã
ã® CDC ã¿ã¹ã¯ãäœæããŠãã ããã ã¬ããªã±ãŒã·ã§ã³ã¿ã¹ã¯ã«ã¯ã次ã®èšå®ã䜿çšããŠãã ãã: ã¿ã¹ã¯èå¥å ã«ã¯ãèå¥å¯èœãªååãå
¥åããŠãã ããã ãœãŒã¹ããŒã¿ããŒã¹ãšã³ããã€ã³ã ã§ã¯ãäœæãã Oracle ãšã³ããã€ã³ããéžæããŠãã ããã ã¿ãŒã²ããããŒã¿ããŒã¹ãšã³ããã€ã³ã ã§ã¯ãäœæãã Aurora PostgreSQL ãšã³ããã€ã³ããéžæããŠãã ããã ã¿ã¹ã¯ã¢ãŒã ã§ã¯ã[ããããžã§ã³ã] ãéžæããŠãã ããã ã¬ããªã±ãŒã·ã§ã³ã€ã³ã¹ã¿ã³ã¹ ã§ã¯ãäœæããããããžã§ã³ãã€ã³ã¹ã¿ã³ã¹ãéžæããŠãã ããã ã¬ããªã±ãŒã·ã§ã³ã¿ã€ãã [ è€è£œã®ã¿ ] ã«èšå®ããŠãã ããã ã¿ãŒã²ããããŒãã«æºåã¢ãŒãã [ äœãããªã ] ã«èšå®ããŠãã ããã ã¿ã¹ã¯èšå®ã®äžã§[ CloudWatch ãã° ããªã³ã«ãã] ãæå¹ã«ãããšãåé¡ããããã°ããããšãã§ããŸãã ããŒãã«ãããã³ã° ã§ã¯ã ç·šéã¢ãŒã ã§ JSON ãšãã£ã¿ ãéžæããŠãã ããã DMS ã¿ã¹ã¯ã®ãããã³ã°ã«ãŒã«ã«ã¯ã次㮠JSON ã䜿çšããŠãã ãããã¹ããŒãåã¯ãããŒãã«ãäœæãããã¹ããŒãåã«çœ®ãæããŠãã ããã { "rules": [ { "rule-type": "transformation", "rule-id": "259824354", "rule-name": "259824354", "rule-target": "column", "object-locator": { "schema-name": "VANSHIKA", "table-name": "TRAVEL_INFO", "column-name": "MOD_3" }, "rule-action": "remove-column", "value": null, "old-value": null }, { "rule-type": "transformation", "rule-id": "259793105", "rule-name": "259793105", "rule-target": "table", "object-locator": { "schema-name": "VANSHIKA", "table-name": "TRAVEL_INFO" }, "rule-action": "convert-lowercase", "value": null, "old-value": null }, { "rule-type": "transformation", "rule-id": "259761426", "rule-name": "259761426", "rule-target": "schema", "object-locator": { "schema-name": "VANSHIKA" }, "rule-action": "convert-lowercase", "value": null, "old-value": null }, { "rule-type": "selection", "rule-id": "259627857", "rule-name": "259627857", "object-locator": { "schema-name": "VANSHIKA", "table-name": "TRAVEL_INFO" }, "rule-action": "include", "filters": [ { "filter-type": "source", "column-name": "MOD_3", "filter-conditions": [ { "filter-operator": "eq", "value": "0" } ] } ] } ] ç§»è¡åè©äŸ¡ãã§ãã¯ããã¯ã¹ããªãã«ããŠãã ããã ç§»è¡ã¿ã¹ã¯ã®ã¹ã¿ãŒãã¢ããèšå® ã§ã¯ã[åŸã§æåã§è¡ã] ãéžæããŠãã ããã ä»ã®èšå®ã¯ããã©ã«ãã®ãŸãŸã«ããŠã[ ã¿ã¹ã¯ãäœæ ] ãéžæããŠãã ããã åãèšå®ã§ä»ã® 2 ã€ã®ã¿ã¹ã¯ ãäœæããŠãã ãããåã¿ã¹ã¯ã§ã mod_3 ã«ã©ã ã® filter-conditions å€ã 0ã1ã2 ã«çœ®ãæããããšãå¿ããã«è¡ã£ãŠãã ããã 3 ã€ã®ã¿ã¹ã¯ãåæã«å®è¡ããŠãã ããã Aurora PostgreSQL ã«ç§»è¡ãããããŒã¿ã select ã¯ãšãªãå®è¡ããŠç¢ºèªããŠãã ããã ãœãŒã¹ããŒã¿ããŒã¹ã§ DML æäœãå®è¡ãããããã®å€æŽãã¿ãŒã²ããããŒãã«ã«ã©ã®ããã«åæ ããããããã㊠DMS ã¿ã¹ã¯ãå°äœãã£ã«ã¿ãŒã«å¯Ÿå¿ããè¡ãã©ã®ããã«ããã¯ã¢ããããããç£èŠããŠãã ããã åã
ã®ã¿ã¹ã¯ã¯èªèº«ã®ãã©ã³ã¶ã¯ã·ã§ã³ã®é åºæ§ãç¶æããŠããŸããã3 ã€ã®ã¿ã¹ã¯å
šäœã§ã®ãã©ã³ã¶ã¯ã·ã§ã³ã®é åºãå¿
ãããä¿ãããããã§ã¯ãªãããšã«æ³šæãå¿
èŠã§ãããã®é åºã®äžæŽåã¯ãã¿ã¹ã¯ã®éå§ãšå®è¡ãç¬ç«ããŠããããšã«èµ·å ããŸãããããã£ãŠãã¿ãŒã²ããããŒãã«ã«çŸãããã©ã³ã¶ã¯ã·ã§ã³ã®æçµçãªé åºã¯ãåããœãŒã¹ããŒãã«ã«é¢ä¿ãããã®ã§ãã£ãŠããå CDC ã®ã¿ã¿ã¹ã¯ã®éå§ã¿ã€ãã³ã°ã®åœ±é¿ãåããå¯èœæ§ããããŸãã Oracle ããç§»è¡ããéã¯ã Oracle ãããªã¢ã©ã€ãºããã¥ãŒ ãæŽ»çšããŠãéžæããã«ã©ã ã«åºã¥ããŠãã£ã«ã¿ãªã³ã°ããããããªã¢ã©ã€ãºããã¥ãŒãäœæããå¿
èŠãªããŒã¿ãµãã»ããã®ã¿ã PostgreSQL ã«ç§»è¡ããããšãã§ããŸããããŒã¿å€æã«ã¯äŸ¿å©ã§ããããããªã¢ã©ã€ãºããã¥ãŒã§ã¯ãœãŒã¹åŽã§æŽæ°ã®ãªãŒããŒããããçºçããç§»è¡ã®ããã©ãŒãã³ã¹ã«åœ±é¿ãäžããå¯èœæ§ããããŸãã å¯å€ã«ã©ã ã䜿çšããŠããŒãã«ãåå²ããå Žåã®èŠ³å¯ æ¢åã® travel_mode ã«ã©ã ã AWS DMS ã¿ã¹ã¯ã®ãã£ã«ã¿ãŒãšããŠäœ¿çšãã代ããã«ãããŒãã«ã«äžå€ã«ã©ã ãå°å
¥ãããšããŸãããã®å Žåã§ããéžè·¯ãç©ºè·¯ãæ°Žè·¯ã®ç§»åææ®µã«å¯Ÿå¿ãã 3 ã€ã® CDC ã¿ã¹ã¯ãäœæããããšã«ãªããŸãã ãã®ã·ããªãªããã¹ãããã«ã¯ã travel_mode åã®ãµããªã¡ã³ã¿ã«ãã®ã³ã°ãæå¹ã«ããå¿
èŠããããŸããããã¯ã次㮠SQL ã³ãã³ãã䜿çšããŠè¡ããŸãã ALTER TABLE "VANSHIKA"."TRAVEL_INFO" add SUPPLEMENTAL LOG GROUP LogGroupTest (TRAVEL_MODE) ALWAYS ; ã¿ã¹ã¯ãã»ããã¢ããããåŸãã¿ã¹ã¯ãéå§ãããœãŒã¹ã§ãã£ã«ã¿ãŒã«äœ¿çšãããŠããã«ã©ã ãæŽæ°ããŠã¿ãŠãã ããã UPDATE travel_info SET travel_mode = 'land' WHERE travel_id = 1 ; AWS DMS ããã®å€æŽãæ€åºã§ããªãããšãããããŸãããã®çµæãæŽæ°ãã¿ãŒã²ããããŒã¿ããŒã¹ (ãã®å Žå㯠Aurora PostgreSQL äºæ) ã«äŒæãããŸããããã®ã«ã©ã ã®æŽæ°ã¯é »ç¹ã§ã¯ãªããããããŸããããçºçããæŽæ°ã¯ãã¹ãŠèŠèœãšãããŸãããã®åäœã¯äºæããããã®ã§ãããããŒã¿ã®äžæŽåã«ã€ãªããå¯èœæ§ããããŸãã ãããã£ãŠã广çãªããŒã¿ã¬ããªã±ãŒã·ã§ã³ãè¡ãã«ã¯ãäœæåŸã«å€æŽãããªãäžå€ãªåã«ãã£ã«ã¿ãŒãé©çšããããšãéèŠã§ãã ããã©ãŒãã³ã¹æ¯èŒ ãã®æçš¿ã§ã¯ãããžãŒãªããŒãã«ã«å¯Ÿãã倧éã® CDC ã·ããªãªã«ããã AWS DMS ã®ããã©ãŒãã³ã¹ãè©äŸ¡ããŸããAWS DMS ã¯ãOracle ããœãŒã¹ãšãã CDC æäœäžã®Redo ãã°ã®èªã¿åãã«ãOracle LogMiner ãš AWS DMS Binary Reader ã® 2 ã€ã®ã¢ãããŒããæäŸããŠããŸãããã®å®éšã§ã¯ãLogMiner (ãªã³ã©ã€ã³ããã³ã¢ãŒã«ã€ãããã Redo ãã°ãã¡ã€ã«ãèªã¿åãããã«èšèšããã Oracle API) ã䜿çšããããšã«ããŸããããã®ãœãªã¥ãŒã·ã§ã³ã¯ Binary Reader ãšãäºææ§ããããŸãããã¹ãããã»ã¹ã¯ 2 ã€ã®ç°ãªããã§ãŒãºã«åãããŠããŸãããæåã«åäžã® CDC ã¿ã¹ã¯ãè©äŸ¡ããæ¬¡ã«ã«ã©ã ãã£ã«ã¿ãŒã䜿çšããŠè€æ°ã® CDC ã¿ã¹ã¯ã«åå²ããŸããã æ¬¡ã®ã€ã³ãã©ã¹ãã©ã¯ãã£ã䜿çšããŸããã ãœãŒã¹: RDS for Oracle ã€ã³ã¹ã¿ã³ã¹ ã€ã³ã¹ã¿ã³ã¹ã¯ã©ã¹: db.r6i.xlarge ã¹ãã¬ãŒãž: 50 GB (3000 IOPS ã® gp3) Oracle ããŒãžã§ã³: 19 (19.0.0.0.ru-2024-10.rur-2024-10.r1) ã¿ãŒã²ãã: Aurora PostgreSQL äºæãšãã£ã·ã§ã³ ã€ã³ã¹ã¿ã³ã¹ã¯ã©ã¹: db.r6i.xlarge Aurora PostgreSQL äºæãšãã£ã·ã§ã³ã®ããŒãžã§ã³: 16.1 AWS DMS ã¬ããªã±ãŒã·ã§ã³ã€ã³ã¹ã¿ã³ã¹ ã€ã³ã¹ã¿ã³ã¹ã¯ã©ã¹: dms.c5.xlarge å²ãåœãŠãããã¹ãã¬ãŒãž: 50 GB ãšã³ãžã³ããŒãžã§ã³: 3.5.4 50 ã«ã©ã ãš 20 ã€ã³ããã¯ã¹ãæã€ãµã³ãã«ããŒãã«ãäœæããŸããããã®ããŒãã«ã«ã¯ããœãŒã¹ãšã¿ãŒã²ããã®äž¡åŽã§æåã« 50,000 è¡ã®ããŒã¿ãæ ŒçŽãããŸãããæ¬¡ã«ãé²è¡äžã®å€æŽãè€è£œããããã®åäžã® CDC ã®ã¿ã®ã¿ã¹ã¯ãèšå®ããŸãããè² è·ã®é«ãæ¬çªç°å¢ãã·ãã¥ã¬ãŒãããããã«ãäžé£ã® DML æäœãå®è¡ããŸããã100 äžä»¶ã®æ°èŠã¬ã³ãŒããæ¿å
¥ããåæã« 10 äžä»¶ã®æ¢åã¬ã³ãŒããæŽæ°ãããã®åŸ 10 äžä»¶ã®æŽæ°ã¹ã¯ãªããã 2 åå®è¡ããæåŸã«ã©ã³ãã ã«éžæããã 10,000 ä»¶ã®ã¬ã³ãŒãã«å¯Ÿã㊠CDC ãã©ãã£ãã¯ãçæããŸããããã®ãã¹ãã«ãããããŒãã«ã«æ¥æ¿ã§å€§èŠæš¡ãªå€æŽãå ããããã·ããªãªãæš¡å£ããããšãã§ããŸããããã®äžé£ã®ããã»ã¹ã«ããã1 æéã®æéã§çŽ 5GB ã®ã¢ãŒã«ã€ããã°ãçæãããŸããããœãŒã¹ããŒã¿ããŒã¹ã§ã®å€æŽçãé«ãããããã®ããã»ã¹ã«ã¯æéããããå¯èœæ§ããããŸããAWS DMS ããœãŒã¹ããŒã¿ããŒã¹ãã倧éã®ã¯ãŒã¯ããŒããåä¿¡ãããšãCDC ã¿ãŒã²ããã¬ã€ãã³ã·ãŒã¡ããªã¯ã¹ãæ¥äžæããããšããããŸããç§ãã¡ã®ãã¹ãã·ããªãªã§ã¯ãæ¿ããæŽ»åæéäžã« CDC ã®åŸ
ã¡æéãããŒã¯æã§çŽ 600 ç§ã«éããŠããããšããæ¬¡ã®ã¹ã¯ãªãŒã³ã·ã§ããã«ç€ºãããŠããŸãã ãã®æåã®ãã¹ãã§ã¯ãåäžã® CDC ã¿ã¹ã¯ã䜿çšããå Žåã®ãé«ããªã¥ãŒã ãé«é »åºŠå€æŽã·ããªãªã«ãããããã©ãŒãã³ã¹ãžã®åœ±é¿ã®å¯èœæ§ãæµ®ã圫ãã«ãªããŸãããããã«ãããã«ã©ã ãã£ã«ã¿ãŒã䜿ã£ãŠã¿ã¹ã¯ãåå²ããã¯ãŒã¯ããŒãã忣ããŠã¬ã€ãã³ã·ãŒãäœæžããããšãç®çãšããåŸç¶ã®å®éšã®èå°ãæŽããŸãããæ¬¡ã®ã°ã©ãã¯ãåäžã® CDC ã¿ã¹ã¯ãšåãã¯ãŒã¯ããŒãã 3 ã€ã® CDC ã®ã¿ã¿ã¹ã¯ã«åå²ããå Žåã®ã¿ãŒã²ããã¬ã€ãã³ã·ãŒã瀺ããŠããŸãã çµæã¯ãã¿ãŒã²ããã§ã®ã¬ã€ãã³ã·ãäœæžãããããšã瀺ããŠããŸãããã®æ¹åã¯ãç§»è¡ããã»ã¹ãé«éåããããšã瀺ããŠããŸããåŸæ¥ã® CDC ã¯åäžã¹ã¬ããæäœã§ããããä»å㯠CDC ãã§ãŒãºã§è€æ°ã®ã¹ã¬ãããäœæãããœãŒã¹ããã¿ãŒã²ãããžã®ããŒã¿ç§»è¡ãè¡ããããã«ãªã£ãããã§ãã å®éšã§äœ¿çšããããŒã¿éã¯å®éã®ã·ããªãªã代衚ãããã®ã§ã¯ãªããããããŸããããCDC æäœã§ã«ã©ã ãã£ã«ã¿ãŒã䜿çšããããšã®ã¡ãªããã广çã«ç€ºããŠããŸãã å
·äœçã«ã¯ãç§»è¡ããã»ã¹ãå éã§ããããšã瀺ããŠãããããã¯å€§éã®ããŒã¿ç§»è¡ã«ã¯äžå¯æ¬ ã§ãã èæ
®äºé
ãã®ã¢ãããŒããå®è£
ããéã«ã¯ã次ã®éèŠãªç¹ã«çæããå¿
èŠããããŸãã æ°ããã«ã©ã ã远å ãããªãŒããŒããã â æ¢åã®ããŒãã«æ§é ã«é©åãªäžå€ã«ã©ã ãèŠã€ãããªãå Žåããã®ç®çã®ããã«æ°ããã«ã©ã ã远å ãããšãããããã®ãªãŒããŒããããçºçããå¯èœæ§ããããŸãããã®å€æŽã¯ããŒã¿ããŒã¹ã¹ããŒãã«åœ±é¿ãäžããç¹ã«ã³ãŒããå
šããŒãã«ããã§ããããå Žåã¯ãã¢ããªã±ãŒã·ã§ã³ããžãã¯ã®å€æŽãå¿
èŠã«ãªãå¯èœæ§ããããŸããã¯ãšãªã§ãã®æ°ããã«ã©ã ã®ãã£ã«ã¿ãªã³ã°ãå®è£
ããå¿
èŠããããã¯ã©ã€ã¢ã³ãã¢ããªã±ãŒã·ã§ã³ã«åœ±é¿ãäžããå¯èœæ§ããããŸããé²ããå Žåã¯ãéçšãžã®åœ±é¿ãæå°éã«æãããããæå®ã®ã¡ã³ããã³ã¹ãŠã£ã³ããŠäžã«ãããã®å€æŽãã¹ã±ãžã¥ãŒã«ããŠãã ããã ãœãŒã¹ã¬ã€ãã³ã·ãŒã®æœåšçãªå¢å â åæã¹ã¬ããæ°ãå¢ãããšããœãŒã¹ã¬ã€ãã³ã·ãŒã®å¢å ã芳枬ãããå¯èœæ§ããããŸããããã¯ãè€æ°ã®äžŠåèªã¿åãæäœã«ãã£ãŠãœãŒã¹ã·ã¹ãã ã«è¿œå ã®è² è·ããããããã§ãã ã¬ããªã±ãŒã·ã§ã³ã€ã³ã¹ã¿ã³ã¹ã®é©åãªãµã€ãžã³ã° â AWS DMS ã¬ããªã±ãŒã·ã§ã³ã€ã³ã¹ã¿ã³ã¹ããã€ã³ã¹ã¿ã³ã¹ã¯ã©ã¹ãšã¹ãã¬ãŒãžå®¹éã®ç¹ã§é©åã«ãµã€ãºèšå®ãããŠããããšã確èªããŠãã ãããåã¿ã¹ã¯ã¯ããœãŒã¹ããŒã¿ããŒã¹ãããã¹ãŠã® REDO ãã°ãèªã¿åããé©çšã§ãããŸã§ããããæ ŒçŽããå¿
èŠããããããååãªãªãœãŒã¹ãäžå¯æ¬ ã§ãã ãã©ã³ã¹ã®åããè² è·åæ£ â ã¿ã¹ã¯åå²ã®ããã®ã«ã©ã ãéžæããéã¯ãäœæããã¿ã¹ã¯éã§äœæ¥è² è·ãã§ããã ãåçã«åæ£ãããã®ãéžãã§ãã ããããã®åçãªåæ£ã¯ãCDC ãã§ãŒãºäžã®äžŠååŠçã«ããå¹çåã®å©ç¹ãæå€§éã«æŽ»ããããã®éµãšãªããŸãã ãããã®ç¹ãæ
éã«æ€èšããããšã§ãå€§èŠæš¡ãªç§»è¡ã·ããªãªã«å¯ŸããŠããæœåšçãªåé¡ãäºæããªãå¯äœçšãæå°éã«æãããããªæé©ãª AWS DMS ãæ§ç¯ããããšãã§ããŸãã ã¯ãªãŒã³ã¢ãã ãã®èšèšã§äœæãããªãœãŒã¹ã§äžèŠã«ãªã£ããã®ã¯ãæéãçºçããªãããã«åé€ããŠãã ãã: Aurora PostgreSQL ã¯ã©ã¹ã¿ãŒãåé€ããŸã ã RDS for Oracle ããŒã¿ããŒã¹ãåé€ããŸã ã AWS DMS ã¬ããªã±ãŒã·ã§ã³ã€ã³ã¹ã¿ã³ã¹ããœãŒã¹ãšã¿ãŒã²ããã®ãšã³ããã€ã³ããã¿ã¹ã¯ãåé€ããŸã ã ãã®å®éšãæ¢åã®ããŒã¿ããŒã¹ã§å®æœããéçšãç¶ããäºå®ã®å Žåã¯ãå®éšãå®äºãããããœãŒã¹ãšã¿ãŒã²ããã®äž¡æ¹ã®ããŒã¿ããŒã¹ãããã¹ãããŒãã«ãæç€ºçã«åé€ããŠãã ããã çµè« ãã®æçš¿ã§ã¯ãäžå€ã®ã«ã©ã ã«åºã¥ããŠè€æ°ã® CDC ã¿ã¹ã¯ã«ã¯ãŒã¯ããŒãã忣ããDMS ã«ã©ã ãã£ã«ã¿ãŒã䜿çšããŠå€§èŠæš¡ã§é »ç¹ã«äœ¿çšãããã¢ã¯ãã£ãããŒãã«ãç§»è¡ããæ¹æ³ã瀺ããŸããã ãã®ææ³ã掻çšããããšã§ãããŒã¿ã®æŽåæ§ãç¶æããªããç§»è¡æéãççž®ã§ããããšã瀺ããŸããã ãã®æŠç¥ã¯ããã«ããŒããš CDC ããã»ã¹ã®äž¡æ¹ãæé©åãããããåæ§ã®å¶çŽã«çŽé¢ããæ§ã
ãªå€§èŠæš¡ç§»è¡ã«é©çšã§ããŸãã AWS DMS ãšãã®æ©èœã®è©³çްã«ã€ããŠã¯ã AWS DMS ããã¥ã¡ã³ã ãåç
§ããŠãã ãããAWS DMS ãã«ããŒãç§»è¡ã®ããã©ãŒãã³ã¹ãæ¹åããããã®ãã¹ããã©ã¯ãã£ã¹ã«ã€ããŠã¯ã 䞊åèªã¿èŸŒã¿ãšãã£ã«ã¿ãŒãªãã·ã§ã³ã䜿çšã㊠AWS DMS ã«ããããŒã¿ããŒã¹ç§»è¡ãé«éåãã ãåç
§ããŠãã ããã èè
ã«ã€ã㊠Vanshika Nigam Vanshika ã¯ãAWS Database Migration Accelerator ïŒDMAïŒããŒã ã®ãœãªã¥ãŒã·ã§ã³ã¢ãŒããã¯ãã§ããAmazon DMA Advisor ãšããŠãã客æ§ããªã³ãã¬ãã¹ããŒã¿ããŒã¹ãåçšããŒã¿ããŒã¹ããAWSã¯ã©ãŠãããŒã¿ããŒã¹ãœãªã¥ãŒã·ã§ã³ãžã®ç§»è¡ãå éã§ããããæ¯æŽããŠããŸããAmazon RDS ãš AWS DMS ã§ 7 幎以äžã®çµéšãæã€åœŒå¥³ã¯ãå¹ççãªããŒã¿ããŒã¹ç§»è¡æŠç¥ã®èšèšãšå®è£
ãå°éãšããŠããŸãã