ïŒæ¬çš¿ã¯ã2025幎12æ8æ¥ã«å
¬éããã â SAP data ingestion and replication with AWS Glue zero-ETL â ã翻蚳ãããã®ã§ãïŒ çµç¹ã§ã¯ãè€éãªããŒã¿ãã€ãã©ã€ã³ãç¶æããããšãªããSAPã·ã¹ãã ããããŒã¿ãåã蟌ã¿ãã€ã³ãµã€ããžããè¿
éã«ã¢ã¯ã»ã¹ããããŒãºãé«ãŸã£ãŠããŸãã AWS Glue zero-ETL with SAP ã¯ãOperational Data ProvisioningïŒ ODP ïŒç®¡çã®SAP Business WarehouseïŒBWïŒãšã¯ã¹ãã©ã¯ã¿ãŒãAdvanced Business Application ProgrammingïŒABAPïŒãCore Data ServicesïŒCDSïŒãã¥ãŒããã®ä»ã®éODPããŒã¿ãœãŒã¹ãªã©ã®SAPããŒã¿ãœãŒã¹ããã®ããŒã¿åã蟌ã¿ãšã¬ããªã±ãŒã·ã§ã³ããµããŒãããŠããŸããZero-ETLããŒã¿ã¬ããªã±ãŒã·ã§ã³ãšã¹ããŒãåæã«ãããæœåºãããããŒã¿ã Amazon Redshift ã Amazon SageMaker lakehouse ã¢ãŒããã¯ã㣠ã Amazon S3 Tables ãªã©ã®AWSç°å¢ã«ä¿åããããšã容æã«ããæåã§ã®ãã€ãã©ã€ã³éçºãæžãããŸããããã«ããã Amazon Q ã Kiro ã Amazon Quick Suite ãªã©ã®ãµãŒãã¹ãšçµã¿åãããŠäœ¿çšããããšã§ãèªç¶èšèªã¯ãšãªã䜿çšããŠSAPããŒã¿ãåæããèªååã®ããã®AIãšãŒãžã§ã³ããäœæãããšã³ã¿ãŒãã©ã€ãºããŒã¿ã©ã³ãã¹ã±ãŒãå
šäœã§ã³ã³ããã¹ãã«å¿ããã€ã³ãµã€ããçæã§ãããAIé§ååã€ã³ãµã€ãã®åºç€ãæ§ç¯ãããŸãã ãã®èšäºã§ã¯ãããŸããŸãªODPããã³éODP SAPãœãŒã¹ãšã®zero-ETLçµ±åãäœæããã³ç£èŠããæ¹æ³ã玹ä»ããŸãã ãœãªã¥ãŒã·ã§ã³æŠèŠ SAPãšããŒã¿é£æºããéã®äž»èŠã³ã³ããŒãã³ãã¯ãSAPããŒã¿æ§é ãšãããã³ã«ã§åäœããããã«èšèšããã AWS Glue SAP ODataã³ãã¯ã¿ ã§ãããã®ã³ãã¯ã¿ã¯ãABAPããŒã¹ã®SAPã·ã¹ãã ãžã®æ¥ç¶ãæäŸããSAPã®ã»ãã¥ãªãã£ãšã¬ããã³ã¹ãã¬ãŒã ã¯ãŒã¯ã«æºæ ããŠããŸããAWS SAPã³ãã¯ã¿ã®äž»ãªæ©èœã¯ä»¥äžã®éãã§ã: ODataãããã³ã«ã䜿çšããŠãããŸããŸãªSAP NetWeaverã·ã¹ãã ããããŒã¿æœåº BWãšã¯ã¹ãã©ã¯ã¿ãŒïŒ2LIS_02_ITMãªã©ïŒãCDSãã¥ãŒïŒC_PURCHASEORDERITEMDEXãªã©ïŒãªã©ã®è€éãªSAPããŒã¿ã¢ãã«ãã管çããã圢ã§ã¬ããªã±ãŒã·ã§ã³ SAP倿ŽããŒã¿ãã£ããã£ïŒCDCïŒãã¯ãããžãŒã䜿çšããODPããã³éODPãšã³ãã£ãã£ã®äž¡æ¹ãžã®å¯Ÿå¿ SAPã³ãã¯ã¿ã¯ãAWS Glue StudioãŸãã¯AWS管çã®zero-ETLã¬ããªã±ãŒã·ã§ã³ã®äž¡æ¹ã§åäœããŸãã AWS Glue Studio ã§ã®ã»ã«ããããŒãžãã¬ããªã±ãŒã·ã§ã³ã¯ãããŒã¿åŠçãŠããããã¬ããªã±ãŒã·ã§ã³é »åºŠãäŸ¡æ Œããã©ãŒãã³ã¹ã®èª¿æŽãããŒãžãµã€ãºãããŒã¿ãã£ã«ã¿ãŒãå®å
ããã¡ã€ã«åœ¢åŒãããŒã¿å€æãããã³éžæããã©ã³ã¿ã€ã ã§ã®ç¬èªã³ãŒãã®èšè¿°ãã³ã³ãããŒã«ã§ããç°å¢ã§ããzero-ETLã§ã®AWS管çããŒã¿ã¬ããªã±ãŒã·ã§ã³ã¯ãã«ã¹ã¿ã èšå®ã®è² æ
ãåãé€ããAWS管çã®ä»£æ¿ææ®µãæäŸãã15åãã6æ¥éã®ã¬ããªã±ãŒã·ã§ã³é »åºŠãå¯èœã«ããŸãã以äžã®ãœãªã¥ãŒã·ã§ã³ã¢ãŒããã¯ãã£ã¯ãããŸããŸãªSAPãœãŒã¹ããzero-ETLã䜿çšããŠODPããã³éODP SAPããŒã¿ãåã蟌ã¿ãAmazon RedshiftãSageMaker lakehouseãS3 Tablesã«æžã蟌ãã¢ãããŒãã瀺ããŠããŸãã ODPãœãŒã¹ã®å€æŽããŒã¿ãã£ãã㣠SAP ODPã¯ãSAPãœãŒã¹ã·ã¹ãã ããã¿ãŒã²ããã·ã¹ãã ãžã®å¢åããŒã¿ã¬ããªã±ãŒã·ã§ã³ãå¯èœã«ããããŒã¿æœåºãã¬ãŒã ã¯ãŒã¯ã§ããODPãã¬ãŒã ã¯ãŒã¯ã¯ãã¢ããªã±ãŒã·ã§ã³ïŒãµãã¹ã¯ã©ã€ããŒïŒãBWãšã¯ã¹ãã©ã¯ã¿ãŒãCDSãã¥ãŒãBWãªããžã§ã¯ããªã©ã®ãµããŒããããŠãããªããžã§ã¯ãããå¢åçã«ããŒã¿ãèŠæ±ã§ããããã«ããŸãã AWS Glue zero-ETLããŒã¿åã蟌ã¿ã¯ãã¿ãŒã²ããã·ã¹ãã ã§ããŒã¹ã©ã€ã³ããŒã¿ã»ããã確ç«ããããã«ããšã³ãã£ãã£ããŒã¿ã®åæãã«ããŒããå®è¡ããããšããå§ãŸããŸããåæãã«ããŒããå®äºãããšãSAPã¯åé€ãå«ãããŒã¿å€æŽããã£ããã£ããOperational Delta QueueïŒODQïŒãšããŠç¥ããããã«ã¿ïŒå¢åïŒãã¥ãŒãããããžã§ãã³ã°ããŸãããã«ã¿ããŒã¯ã³ã¯åæããŒãäžã«ãµãã¹ã¯ã©ã€ããŒã«éä¿¡ãããzero-ETLå
éšã®ç¶æ
管çã·ã¹ãã å
ã«æ°žç¶åãããŸãã å¢ååŠçã¯ãç¶æ
ã¹ãã¢ããæåŸã«ä¿åããããã«ã¿ããŒã¯ã³ãååŸããODataãããã³ã«ã䜿çšããŠãã®ããŒã¯ã³ã䜿çšããŠSAPã«ãã«ã¿å€æŽãªã¯ãšã¹ããéä¿¡ããŸããã·ã¹ãã ã¯ãSAP ODQã¡ã«ããºã ãéããŠè¿ãããINSERT/UPDATE/DELETEæäœãåŠçããã¬ã³ãŒãã倿ŽãããŠããªãã·ããªãªã§ãSAPããæ°ãããã«ã¿ããŒã¯ã³ãåä¿¡ããŸãããã®æ°ããããŒã¯ã³ã¯ãåã蟌ã¿ãæåããåŸãç¶æ
管çã·ã¹ãã ã«æ°žç¶åãããŸãããšã©ãŒã·ããªãªã§ã¯ãã·ã¹ãã ã¯æ¢åã®ãã«ã¿ããŒã¯ã³ç¶æ
ãä¿æããããŒã¿æå€±ãªãã§å詊è¡ã¡ã«ããºã ãå¯èœã«ããŸãã 以äžã®ã¹ã¯ãªãŒã³ã·ã§ããã¯ãSAPã·ã¹ãã ã§ã®æåããåæããŒããšããã«ç¶ã4åã®å¢åããŒã¿åã蟌ã¿ã瀺ããŠããŸãã éODPãœãŒã¹ã®å€æŽããŒã¿ãã£ãã㣠éODPæ§é ã¯ãODP察å¿ã§ãªãODataãµãŒãã¹ã§ãããããã¯ãODPãã¬ãŒã ã¯ãŒã¯ãªãã§çŽæ¥å
¬éãããAPIã颿°ããã¥ãŒããŸãã¯CDSãã¥ãŒã§ãããã®ã¡ã«ããºã ã䜿çšããŠããŒã¿ãæœåºãããŸãããå¢åããŒã¿æœåºã¯ãªããžã§ã¯ãã®æ§è³ªã«äŸåããŸããããšãã°ããªããžã§ã¯ãã«ãæçµæŽæ°æ¥ããã£ãŒã«ããå«ãŸããŠããå Žåãããã䜿çšããŠå€æŽã远跡ããå¢åããŒã¿æœåºãæäŸããŸãã AWS Glue zero-ETLã¯ããšã³ãã£ãã£ã«å€æŽã远跡ãããã£ãŒã«ãïŒæçµæŽæ°æ¥ãŸãã¯æå»ïŒãå«ãŸããŠããå ŽåãéODP ODataãµãŒãã¹ã«å¯ŸããŠããã«äœ¿ããå¢åããŒã¿æœåºãæäŸããŸãããããã®SAPãµãŒãã¹ã«ãzero-ETLã¯ããŒã¿åã蟌ã¿ã«2ã€ã®ã¢ãããŒããæäŸããŸã:ã¿ã€ã ã¹ã¿ã³ãããŒã¹ã®å¢ååŠçãšå®å
šããŒãã§ãã ã¿ã€ã ã¹ã¿ã³ãããŒã¹ã®å¢ååŠç ã¿ã€ã ã¹ã¿ã³ãããŒã¹ã®å¢ååŠçã¯ãzero-ETLã§ãŠãŒã¶ãŒãèšå®ããã¿ã€ã ã¹ã¿ã³ããã£ãŒã«ãã䜿çšããŠãããŒã¿æœåºããã»ã¹ãæé©åããŸããzero-ETLã·ã¹ãã ã¯ãåŸç¶ã®å¢ååŠçæäœã®åºç€ãšãªãéå§ã¿ã€ã ã¹ã¿ã³ãã確ç«ããŸãããŠã©ãŒã¿ãŒããŒã¯ãšããŠç¥ããããã®ã¿ã€ã ã¹ã¿ã³ãã¯ãããŒã¿ã®äžè²«æ§ãä¿é²ããããã«éèŠã§ããã¯ãšãªæ§ç¯ã¡ã«ããºã ã¯ãã¿ã€ã ã¹ã¿ã³ãæ¯èŒã«åºã¥ããŠODataãã£ã«ã¿ãŒãæ§ç¯ããŸãããããã®ã¯ãšãªã¯ãæåŸã«æåããåŠçå®è¡ä»¥éã«äœæãŸãã¯å€æŽãããã¬ã³ãŒããæœåºããŸããã·ã¹ãã ã®ãŠã©ãŒã¿ãŒããŒã¯ç®¡çæ©èœã¯ãååŠçãµã€ã¯ã«ããæé«ã®ã¿ã€ã ã¹ã¿ã³ãå€ã®è¿œè·¡ãç¶æãããã®æ
å ±ãåŸç¶ã®å®è¡ã®éå§ç¹ãšããŠäœ¿çšããŸããzero-ETLã·ã¹ãã ã¯ãèšå®ããããã©ã€ããªããŒã䜿çšããŠã¿ãŒã²ããã§ã¢ãããµãŒããå®è¡ããŸãããã®ã¢ãããŒãã¯ãããŒã¿æŽåæ§ãç¶æããªããæŽæ°ãé©åã«åŠçããããšãä¿é²ããŸããåã¿ãŒã²ããã·ã¹ãã æŽæ°ãæåããåŸããŠã©ãŒã¿ãŒããŒã¯ã¿ã€ã ã¹ã¿ã³ããé²ããããå°æ¥ã®åŠçãµã€ã¯ã«ã®ããã®ä¿¡é Œã§ãããã§ãã¯ãã€ã³ããäœæãããŸãã ãã ããã¿ã€ã ã¹ã¿ã³ãããŒã¹ã®ã¢ãããŒãã«ã¯å¶éããããŸã â SAPã·ã¹ãã ã¯åé€ã¿ã€ã ã¹ã¿ã³ããç¶æããªããããç©ççãªåé€ã远跡ã§ããŸãããã¿ã€ã ã¹ã¿ã³ããã£ãŒã«ããå©çšã§ããªããèšå®ãããŠããªãã·ããªãªã§ã¯ãã·ã¹ãã ã¯ã¢ãããµãŒãåŠçã䌎ããã«ããŒãã«ç§»è¡ããŸãã ãã«ããŒãïŒå®å
šããŒãïŒ ãã«ããŒãã¢ãããŒãã¯ãç¬ç«ããïŒååŸé¢ä¿ã®ãªãïŒåŠçãšããŠããŸãã¯ã¿ã€ã ã¹ã¿ã³ãããŒã¹ã®åŠçãå®è¡å¯èœã§ãªãå Žåã®ãã©ãŒã«ããã¯ã¡ã«ããºã ãšããŠæ©èœããŸãããã®æ¹æ³ã¯ãååŠçãµã€ã¯ã«äžã«å®å
šãªãšã³ãã£ãã£ããŒã¿ã»ãããæœåºããããšãå«ã¿ã倿Žè¿œè·¡ãå©çšã§ããªããå¿
èŠã§ãªãã·ããªãªã«é©ããŠããŸããæœåºãããããŒã¿ã»ããã¯ãã¿ãŒã²ããã·ã¹ãã ã§ã¢ãããµãŒããããŸããã¢ãããµãŒãåŠçããžãã¯ã¯ãæ°ããã¬ã³ãŒãã®æ¿å
¥ãšæ¢åã¬ã³ãŒãã®æŽæ°ã®äž¡æ¹ãåŠçããŸãã å¢åããŒããŸãã¯ãã«ããŒããéžæããã¿ã€ãã³ã° ã¿ã€ã ã¹ã¿ã³ãããŒã¹ã®å¢ååŠçã¢ãããŒãã¯ãé »ç¹ã«æŽæ°ãããå€§èŠæš¡ãªããŒã¿ã»ããã«å¯ŸããŠæé©ãªããã©ãŒãã³ã¹ãšãªãœãŒã¹äœ¿çšçãæäŸããŸãã倿Žãããã¬ã³ãŒãã®ã¿ã®éžæç転éã«ãããããŒã¿è»¢ééãåæžããããããã¯ãŒã¯ãã©ãã£ãã¯ãåæžãããŸãããã®æé©åã¯ãéçšã³ã¹ãã®åæžã«çŽæ¥ã€ãªãããŸããã¢ãããµãŒãã䌎ããã«ããŒãã¯ãå¢ååŠçãå®è¡å¯èœã§ãªãã·ããªãªã§ã®ããŒã¿åæãä¿é²ããŸãã ãããã®ã¢ãããŒãã¯ãéODP SAPæ§é ãšã®zero-ETLçµ±åã®ããã®å®å
šãªãœãªã¥ãŒã·ã§ã³ã圢æãããšã³ã¿ãŒãã©ã€ãºããŒã¿çµ±åã·ããªãªã®å€æ§ãªèŠä»¶ã«å¯Ÿå¿ããŸãããããã®ã¢ãããŒãã䜿çšããçµç¹ã¯ã2ã€ã®ã¢ãããŒãã®ãããããéžæããéã«ãç¹å®ã®ãŠãŒã¹ã±ãŒã¹ãããŒã¿éãããã³ããã©ãŒãã³ã¹èŠä»¶ãè©äŸ¡ããå¿
èŠããããŸãã SAP zero-ETLçµ±åã®ç£èŠ AWS Glueã¯ã Amazon CloudWatch ãã°ã䜿çšããŠç¶æ
管çããã°ãããã³ã¡ããªã¯ã¹ãç¶æããŸããå¯èŠ³æž¬æ§ãèšå®ããæé ã«ã€ããŠã¯ãã çµ±åã®ç£èŠ ããåç
§ããŠãã ããããã°é
ä¿¡çšã« AWS Identity and Access Management ïŒIAMïŒããŒã«ãèšå®ãããŠããããšã確èªããŠãã ãããçµ±åã¯ããœãŒã¹åã蟌ã¿ãšéžæããã¿ãŒã²ãããžã®æžã蟌ã¿ã®äž¡æ¹ããç£èŠãããŸãã ãœãŒã¹åã蟌ã¿ã®ç£èŠ AWS Glue zero-ETLãšCloudWatchã®çµ±åã«ãããããŒã¿çµ±åããã»ã¹ã远跡ããã³ãã©ãã«ã·ã¥ãŒãã£ã³ã°ããããã®ç£èŠæ©èœãæäŸãããŸããCloudWatchãéããŠãåé¡ãç¹å®ããããã©ãŒãã³ã¹ãç£èŠããSAPããŒã¿çµ±åã®éçšç¶æ
ãç¶æããã®ã«åœ¹ç«ã€è©³çްãªãã°ãã¡ããªã¯ã¹ãããã³ã€ãã³ãã«ã¢ã¯ã»ã¹ã§ããŸããæåãšãšã©ãŒã®ã·ããªãªã®ããã€ãã®äŸãèŠãŠã¿ãŸãããã ã·ããªãª1: ããŒã«ã«æš©éããªã ãã®ãšã©ãŒã¯ãSAPããŒã¿ãžã®ã¢ã¯ã»ã¹ã詊ã¿ãéã«ãAWS Glueã§ã®ããŒã¿çµ±åããã»ã¹äžã«çºçããŸãããæ¥ç¶ã¯ã400 Bad Requestã¹ããŒã¿ã¹ã³ãŒããæã€CLIENT_ERRORã«ééããããŒã«ã«æš©éããªãããšã瀺ããŠããŸãã { "eventTimestamp": 1755031897157, "integrationArn": "arn:aws:glue:us-east-2:012345678901:integration:1da4dccd-96ce-4661-8ef1-bf216623d65f", "sourceArn": "arn:aws:glue:us-east-2:012345678901:connection/SAPOData-sap-glue-dev", "level": "ERROR", "messageType": "IngestionFailed", "details": { "loadType": "", "errorMessage": "You do not have the necessary permissions to access the glue connection. make sure that you have the correct IAM permissions to access AWS Glue resources.", "errorCode": "CLIENT_ERROR" } } ã·ããªãª2: ãã«ã¿ãªã³ã¯ã®ç Žæ CloudWatchãã°ã¯ãSAPããAWS Glueãžã®ããŒã¿åæäžã«ãã«ã¿ããŒã¯ã³ãæ¬ èœããŠããåé¡ã瀺ããŠããŸãããšã©ãŒã¯ãODataãµãŒãã¹ãéããŠSAP販売ããã¥ã¡ã³ãé
ç®ããŒãã«FactsOfCSDSLSDOCITMDXã«ã¢ã¯ã»ã¹ããããšãããšãã«çºçããŸããå¢åããŒã¿ããŒããšå€æŽã®è¿œè·¡ã«å¿
èŠãªãã«ã¿ããŒã¯ã³ããªããããã·ã¹ãã ãããŒã¿æœåºAPI RODPS_REPL_ODP_OPENãéãããšãããšãã«CLIENT_ERRORïŒ400 Bad RequestïŒãçºçããŸããã { "eventTimestamp": 1760700305466, "integrationArn": "arn:aws:glue:us-east-1:012345678901:integration:f62e1971-092c-46a3-ba88-d32f4c6cd649", "sourceArn": "arn:aws:glue:us-east-1:012345678901:connection/SAPOData-sap-glue-dev", "level": "ERROR", "messageType": "IngestionFailed", "details": { "tableName": "/sap/opu/odata/sap/Z_C_SALESDOCUMENTITEMDEX_SRV/FactsOfCSDSLSDOCITMDX", "loadType": "", "errorMessage": "Received an error from SAPOData: Could not open data access via extraction API RODPS_REPL_ODP_OPEN. Status code 400 (Bad Request).", "errorCode": "CLIENT_ERROR" } ã·ããªãª3: SAPããŒã¿åã蟌ã¿ã®ã¯ã©ã€ã¢ã³ããšã©ãŒ ãã®CloudWatchãã°ã¯ãSAPãšã³ãã£ãã£EntityOf0VENDOR_ATTRãODataãµãŒãã¹ãéããŠèŠã€ãããªãããã¢ã¯ã»ã¹ã§ããªãã¯ã©ã€ã¢ã³ãäŸå€ã·ããªãªãæããã«ããŠããŸãããã®CLIENT_ERRORã¯ãAWS Glueã³ãã¯ã¿ãSAPã·ã¹ãã ããã®å¿çãè§£æããããšãããããšã³ãã£ãã£ããœãŒã¹SAPã·ã¹ãã ã«ååšããªãããSAPã€ã³ã¹ã¿ã³ã¹ãäžæçã«å©çšã§ããªãããã«å€±æãããšãã«çºçããŸãã { "eventTimestamp": 1752676327649, "integrationArn": "arn:aws:glue:us-east-1:012345678901:integration:9f1acbc0-599f-47d2-8e84-e9779976af59", "sourceArn": "arn:aws:glue:us-east-1:012345678901:connection/SAPOData-sap-glue-dev", "level": "ERROR", "messageType": "IngestionFailed", "details": { "tableName": "/sap/opu/odata/sap/ZVENDOR_ATTR_SRV/EntityOf0VENDOR_ATTR", "loadType": "", "errorMessage": "Data read from source failed for entity /sap/opu/odata/sap/ZVENDOR_ATTR_SRV/EntityOf0VENDOR_ATTR using connector SAPOData; ErrorMessage: Glue connector returned client exception. The response from the connector application couldn't be parsed.", "errorCode": "CLIENT_ERROR" } } ã¿ãŒã²ããæžã蟌ã¿ã®ç£èŠ Zero-ETLã¯ãã¿ãŒã²ããã·ã¹ãã ã«å¿ããç£èŠã¡ã«ããºã ãæ¡çšããŠããŸããAmazon Redshiftã¿ãŒã²ããã®å Žåãçµ±åã¹ããŒã¿ã¹ããžã§ãå®è¡ãããã³ããŒã¿ç§»åçµ±èšã«é¢ãã詳现æ
å ±ãæäŸããsvv_integrationã·ã¹ãã ãã¥ãŒã䜿çšããŸããSageMaker lakehouseã¿ãŒã²ããã䜿çšããå Žåãzero-ETLã¯zetl_integration_table_stateããŒãã«ãéããŠçµ±åç¶æ
ã远跡ããåæã¹ããŒã¿ã¹ãã¿ã€ã ã¹ã¿ã³ããããã³å®è¡ã®è©³çްã«é¢ããã¡ã¿ããŒã¿ãç¶æããŸããããã«ãCloudWatchãã°ã䜿çšããŠçµ±åã®é²è¡ç¶æ³ãç£èŠããæåããã³ããããã¡ã¿ããŒã¿ã®æŽæ°ãããã³ããŒã¿æžã蟌ã¿ããã»ã¹äžã®æœåšçãªåé¡ã«é¢ããæ
å ±ããã£ããã£ã§ããŸãã ã·ããªãª1: SageMaker lakehouseã¿ãŒã²ããã§ã®åŠçæå CloudWatchãã°ã¯ãCDCã¢ãŒãã䜿çšãããã©ã³ãããŒãã«ã®æåããããŒã¿åæã¢ã¯ãã£ããã£ã瀺ããŠããŸããæåã®ãã°ãšã³ããªïŒIngestionCompletedïŒã¯ãã¿ã€ã ã¹ã¿ã³ã1757221555568ã§ã®åã蟌ã¿ããã»ã¹ã®æåããå®äºã確èªããæåŸã®åæã¿ã€ã ã¹ã¿ã³ãã¯1757220991999ã§ãã2çªç®ã®ãã°ïŒIngestionTableStatisticsïŒã¯ãããŒã¿å€æŽã®è©³çްãªçµ±èšãæäŸãããã®CDCåæäžã«300ã®æ°ããã¬ã³ãŒããæ¿å
¥ããã8ã€ã®ã¬ã³ãŒããæŽæ°ããã2ã€ã®ã¬ã³ãŒããã¿ãŒã²ããããŒã¿ããŒã¹glueetlããåé€ãããããšã瀺ããŠããŸãããã®ã¬ãã«ã®è©³çްã¯ãã¿ãŒã²ããã·ã¹ãã ã«äŒæããã倿Žã®éãšã¿ã€ããç£èŠããã®ã«åœ¹ç«ã¡ãŸãã { "eventTimestamp": 1757221555568, "integrationArn": "arn:aws:glue:us-east-1:012345678901:integration:b7a1c69a-e180-4d27-b71d-5fcf196d9d2d", "sourceArn": "arn:aws:glue:us-east-1:012345678901:connection/mam301", "targetArn": "arn:aws:glue:us-east-1:012345678901:database/gluezetl", "level": "VERBOSE", "messageType": "IngestionCompleted", "details": { "tableName": "plant", "loadType": "CDC", "message": "Successfully completed ingestion", "lastSyncedTimestamp": 1757220991999, "consumedResourceUnits": "10" } } { "eventTimestamp": 1757222506936, "integrationArn": "arn:aws:glue:us-east-1:012345678901:integration:b7a1c69a-e180-4d27-b71d-5fcf196d9d2d", "sourceArn": "arn:aws:glue:us-east-1:012345678901:connection/mam301", "targetArn": "arn:aws:glue:us-east-1:012345678901:database/gluezetl", "level": "INFO", "messageType": "IngestionTableStatistics", "details": { "tableName": "plant", "loadType": "CDC", "insertCount": 300, "updateCount": 8, "deleteCount": 2 } } ã·ããªãª2: Amazon SageMaker lakehouse ã¿ãŒã²ããã®ã¡ããªã¯ã¹ SageMaker lakehouseã®zetl_integration_table_stateããŒãã«ã¯ãçµ±åã¹ããŒã¿ã¹ãšããŒã¿å€æŽã¡ããªã¯ã¹ã®ãã¥ãŒãæäŸããŸãããã®äŸã§ã¯ãããŒãã«ã¯ãtestdbããŒã¿ããŒã¹ã®çµ±åID 62b1164f-5b85-45e4-b8db-9aa7ab841e98ãæã€SAP CDSãã¥ãŒããŒãã«ã®æåããçµ±åã瀺ããŠããŸããã¬ã³ãŒãã¯ãã¿ã€ã ã¹ã¿ã³ã1733000485999ã§ã10ã®æ¿å
¥ã¬ã³ãŒããåŠçãããããšã瀺ããŠããŸãïŒrecent_insert_record_count: 10ïŒãæŽæ°ãŸãã¯åé€ã¯ãããŸããïŒäž¡æ¹ã®ã«ãŠã³ãã¯0ïŒããã®ããŒãã«ã¯ç£èŠããŒã«ãšããŠæ©èœããçµ±åç¶æ
ãšããŒã¿å€æŽã«é¢ãã詳现ãªçµ±èšã®éäžãã¥ãŒãæäŸããlakehouseã§ã®ããŒã¿åæã¢ã¯ãã£ããã£ã远跡ããã³æ€èšŒããããšãç°¡åã«ããŸãã +---+--------------------------------------+---------------+----------------------------------------------------------+-----------+--------+-----------------+-------------------------------+------------------------------+------------------------------+------------------------------+ | # | integration_id | target_database | table_name | table_state | reason | last_updated_timestamp | recent_ingestion_record_count | recent_insert_record_count | recent_update_record_count | recent_delete_record_count | +---+--------------------------------------+---------------+----------------------------------------------------------+-----------+--------+-----------------+-------------------------------+------------------------------+------------------------------+------------------------------+ | 2 | 62b1164f-5b85-45e4-b8db-9aa7ab841e98 | testdb | _sap_opu_odata_sap_zcds_po_scl_new_srv_factsofzmmpurordsldex | SUCCEEDED | | 1733000485999 | 10 | 0 | 0 | 0 | +---+--------------------------------------+---------------+----------------------------------------------------------+-----------+--------+-----------------+-------------------------------+------------------------------+------------------------------+------------------------------+ ã·ããªãª3: Redshiftçšç£èŠã·ã¹ãã ã¯ãzero-ETLçµ±åã¹ããŒã¿ã¹ã远跡ããããã«2ã€ã®ãã¥ãŒãäœ¿çš svv_integrationã¯ãçµ±åã¹ããŒã¿ã¹ã®é«ã¬ãã«ã®æŠèŠãæäŸããçµ±åID 03218b8a-9c95-4ec2-81ad-dd4d5398e42aã倱æãªãã§18ã®ããŒãã«ãæ£åžžã«ã¬ããªã±ãŒãããæåŸã®ãã§ãã¯ãã€ã³ãããã©ã³ã¶ã¯ã·ã§ã³ã·ãŒã±ã³ã¹1761289852999ã§ãã£ãããšã瀺ããŠããŸãã +--------------------------------------+---------------+-----------+-----------------+-------------+----------------------------------------------+-------------------------+-----------------------+---------------+------------------+-----------------+-----------------+------------------+-----------------+-----------------+ | integration_id | target_database | source | state | current_lag | last_replicated_checkpoint | total_tables_replicated | total_tables_failed | creation_time | refresh_interval | source_database | is_history_mode | query_all_states | truncatecolumns | accept_invchars | +--------------------------------------+---------------+-----------+-----------------+-------------+----------------------------------------------+-------------------------+-----------------------+---------------+------------------+-----------------+-----------------+------------------+-----------------+-----------------+ | 03218b8a-9c95-4ec2-81ad-dd4d5398e42a | test_case | GlueSaaS | CdcRefreshState | 771754 | {"txn_seq":"1761289852999","txn_id":"0"} | 18 | 0 | 22:54.7 | 0 | | FALSE | FALSE | FALSE | FALSE | +--------------------------------------+---------------+-----------+-----------------+-------------+----------------------------------------------+-------------------------+-----------------------+---------------+------------------+-----------------+-----------------+------------------+-----------------+-----------------+ svv_integration_table_stateã¯ãçµ±åå
ã®åã
ã®ããŒãã«ã®ã¹ããŒã¿ã¹ã瀺ãããŒãã«ã¬ãã«ã®ç£èŠè©³çްãæäŸããŸãããã®å ŽåãSAPææã°ã«ãŒãããã¹ããšã³ãã£ãã£ããŒãã«ã¯åææžã¿ç¶æ
ã«ãããæåŸã®ã¬ããªã±ãŒã·ã§ã³ãã§ãã¯ãã€ã³ãã¯çµ±åãã§ãã¯ãã€ã³ãïŒ1761289852999ïŒãšäžèŽããŠããŸããããŒãã«ã¯çŸåš0è¡ãš0ãµã€ãºã瀺ããŠãããæ°ããäœæãããããšã瀺åããŠããŸãã +--------------------------------------+---------------+-------------+--------------------------------------------------------------+-------------+----------------------------------------------+--------+-----------------------+------------+------------+-----------------+ | integration_id | target_database | schema_name | table_name | table_state | table_last_replicated_checkpoint | reason | last_updated_timestamp | table_rows | table_size | is_history_mode | +--------------------------------------+---------------+-------------+--------------------------------------------------------------+-------------+----------------------------------------------+--------+-----------------------+------------+------------+-----------------+ | 03218b8a-9c95-4ec2-81ad-dd4d5398e42a | test_case | public | /sap/opu/odata/sap/ZMATL_GRP_1_SRV/EntityOf0MATL_GRP_1_TEXT | Synced | {"txn_seq":"1761289852999","txn_id":"0"} | | 23:03.8 | 0 | 0 | FALSE | +--------------------------------------+---------------+-------------+--------------------------------------------------------------+-------------+----------------------------------------------+--------+-----------------------+------------+------------+-----------------+ ãããã®ãã¥ãŒã¯ãAmazon Redshiftã§ã®å
šäœçãªçµ±åã®å¥å
šæ§ãšåã
ã®ããŒãã«åæã¹ããŒã¿ã¹ã®äž¡æ¹ã远跡ããããã®å
æ¬çãªç£èŠãœãªã¥ãŒã·ã§ã³ãæäŸããŸãã åææ¡ä»¶ 以äžã®ã»ã¯ã·ã§ã³ã§ã¯ãSAPæ¥ç¶ãèšå®ãããã®æ¥ç¶ã䜿çšããŠzero-ETLçµ±åãäœæããããã«å¿
èŠãªæé ã説æããŸãããã®ãœãªã¥ãŒã·ã§ã³ãå®è£
ããåã«ã以äžãæºåããå¿
èŠããããŸã: SAPã¢ã«ãŠã³ã 管çè
ã¢ã¯ã»ã¹æš©ãæã€AWSã¢ã«ãŠã³ã S3 Tablesã¿ãŒã²ãã ãäœæããS3ãã±ããsap_demo_table_bucketãããŒã¿ããŒã¹ã®ãã±ãŒã·ã§ã³ãšããŠé¢é£ä»ãã zero-ETLã®Data Catalogã®ãã现ããã¢ã¯ã»ã¹å¶åŸ¡ã®ããã«ã以äžã® IAMããªã·ãŒ ã䜿çšã㊠AWS Glue Data Catalogèšå® ãæŽæ°ãã zero-ETLãSAPã¢ã«ãŠã³ãããããŒã¿ã«ã¢ã¯ã»ã¹ããããã«äœ¿çšããIAMããŒã« zero_etl_bulk_demo_role ãäœæãã SAPèªèšŒæ
å ±ãä¿åããããã«AWS Secrets Managerã§ã·ãŒã¯ã¬ãã zero_etl_bulk_demo_secret ãäœæãã SAPã€ã³ã¹ã¿ã³ã¹ãžã®æ¥ç¶ãäœæ SAPã€ã³ã¹ã¿ã³ã¹ãžã®æ¥ç¶ãèšå®ããã¢ã¯ã»ã¹ããããŒã¿ãæäŸããã«ã¯ã以äžã®æé ãå®è¡ããŸã: AWS Glueã³ã³ãœãŒã«ã§ãããã²ãŒã·ã§ã³ãã€ã³ã®Data catalogã®äžã§ãConnectionsãéžæããCreate ConnectionãéžæããŸãã Data sourcesã§ãSAP ODataãéžæãã Next ãéžæããŸãã SAPã€ã³ã¹ã¿ã³ã¹ã®URLãå
¥åããŸãã IAM service roleã§ãããŒã«zero_etl_bulk_demo_roleïŒåææ¡ä»¶ãšããŠäœæïŒãéžæããŸãã Authentication Typeã§ãSAPã«äœ¿çšããŠããèªèšŒã¿ã€ããéžæããŸãã AWS Secretã§ãã·ãŒã¯ã¬ããzero_etl_bulk_demo_secretïŒåææ¡ä»¶ãšããŠäœæïŒãéžæããŸãã NextãéžæããŸãã Nameã«ãsap_demo_connãªã©ã®ååãå
¥åããŸãã NextãéžæããŸãã zero-ETLçµ±åãäœæ zero-ETLçµ±åãäœæããã«ã¯ã以äžã®æé ãå®è¡ããŸã: AWS Glueã³ã³ãœãŒã«ã§ãããã²ãŒã·ã§ã³ãã€ã³ã®Data catalogã®äžã§ãZero-ETL integrationsãéžæããCreate zero-ETL integrationãéžæããŸãã Data sourceã§ãSAP ODataãéžæããNextãéžæããŸãã åã®æé ã§äœæããæ¥ç¶åãšIAMããŒã«ãéžæããŸãã çµ±åã«å«ããSAPãªããžã§ã¯ããéžæããŸããéODPãªããžã§ã¯ãã¯å®å
šããŒããŸãã¯å¢åããŒãçšã«èšå®ãããODPãªããžã§ã¯ãã¯å¢ååã蟌ã¿çšã«èªåçã«èšå®ãããŸãã å®å
šããŒãã®å ŽåãIncremental update fieldãNo timestamp field selectedã®ãŸãŸã«ããŸãã å¢åããŒãã®å ŽåãIncremental update fieldã®ç·šéã¢ã€ã³ã³ãéžæããã¿ã€ã ã¹ã¿ã³ããã£ãŒã«ããéžæããŸãã ãã«ã¿ããŒã¯ã³ãæäŸããODPãšã³ãã£ãã£ã®å Žåãå¢åæŽæ°ãã£ãŒã«ãã¯äºåã«éžæãããŠããããŠãŒã¶ãŒã®ã¢ã¯ã·ã§ã³ã¯å¿
èŠãããŸããã åãSAPæ¥ç¶ãšãšã³ãã£ãã£ãããŒã¿ãã£ã«ã¿ãŒã§äœ¿çšããŠæ°ããçµ±åãäœæããå Žåãæåã®çµ±åãšã¯ç°ãªãå¢åæŽæ°ãã£ãŒã«ããéžæããããšã¯ã§ããŸããã Target detailsã§ãsap_demo_table_bucketïŒåææ¡ä»¶ãšããŠäœæïŒãéžæããŸãã Target IAM roleã§ãsap_demo_roleïŒåææ¡ä»¶ãšããŠäœæïŒãéžæããŸãã NextãéžæããŸãã Integration detailsã»ã¯ã·ã§ã³ã§ãNameã«sap-demo-integrationãšå
¥åããŸãã NextãéžæããŸãã 詳现ã確èªããCreate and launch integrationãéžæããŸãã æ°ããäœæãããçµ±åã¯ãçŽ1åã§ActiveãšããŠè¡šç€ºãããŸãã ã¯ãªãŒã³ã¢ãã ãªãœãŒã¹ãã¯ãªãŒã³ã¢ããããã«ã¯ã以äžã®æé ãå®è¡ããŸãããã®ããã»ã¹ã¯ããã®èšäºã§äœæããããªãœãŒã¹ãå®å
šã«åé€ããŸããç¶è¡ããåã«éèŠãªããŒã¿ãããã¯ã¢ããããŠãã ããã zero-ETLçµ±å sap-demo-integration ãåé€ããŸãã S3 Tablesã¿ãŒã²ãããã±ãã sap_demo_table_bucket ãåé€ããŸãã Data Catalogæ¥ç¶ sap_demo_conn ãåé€ããŸãã Secrets Managerã·ãŒã¯ã¬ãã zero_etl_bulk_demo_secret ãåé€ããŸãã ãŸãšã ãããŸã§ã®æé ã§ãåŸæ¥ã®ãããªETLæ§ç¯ã®è€éããªãã«ãSAPããŒã¿åæã倧ããæ¹åã§ããããã«ãªããŸãããAWS Glue zero-ETLã䜿çšãããšãS3 TablesãSageMaker lakehouseã¢ãŒããã¯ãã£ãAmazon Redshiftã«ãå
ã®æ§é ãç¶æããªããããŒã¿ã飿ºã§ããSAPããŒã¿ã«å³åº§ã«ã¢ã¯ã»ã¹å¯èœãªç°å¢ãæ§ç¯ã§ããŸããããŒã ã¯ãã³ã¹ãå¹çã®é«ãã¯ã©ãŠãã¹ãã¬ãŒãžã«ããŒã¿ãä¿æããªãããApache Iceberg ã®ã¿ã€ã ãã©ãã«æ©èœãã¹ããŒãé²åãããã³å€§èŠæš¡ãªåæèªã¿åã/æžã蟌ã¿ãåããACIDæºæ ã®ã¹ãã¬ãŒãžã䜿çšã§ããŸããAmazon QãšSageMakerçã®AIæ©èœã¯ãããžãã¹ããªã³ããã³ãããŒã¿è£œåãäœæããããã¹ãããSQLãžã®ã¯ãšãªãå®è¡ããAmazon BedrockãšQuick Suiteã䜿çšããŠAIãšãŒãžã§ã³ãããããã€ããã®ã«åœ¹ç«ã¡ãŸãã 詳现ã«ã€ããŠã¯ã以äžã®ãªãœãŒã¹ãåç
§ããŠãã ãã: AWS Glue Documentation Zero-ETL integrations Connecting to SAP OData Amazon SageMaker lakehouse ã¢ãŒããã¯ã㣠Amazon S3 tables ãã¿ãŒã²ãããšããŠèšå®ãã Amazon Q Documentation Amazon Quick Suite Documentation SAPããŒã¿æŠç¥ãè¿ä»£åããæºåã¯ã§ããŠããŸããïŒAWS Glue zero-ETLãæ¢çŽ¢ããçµç¹ã®ããŒã¿åææ©èœã匷åããŸãããã èè
ã«ã€ã㊠Shashank Sharma Shashank ã¯ããšã³ã¿ãŒãã©ã€ãºé¡§å®¢åãã®ãã¡ãŒã¹ãããŒãã£ããã³ãµãŒãããŒãã£ã®ããŒã¿ããŒã¹ãšSaaSã®ããŒã¿çµ±åããã³ã¬ããªã±ãŒã·ã§ã³ãœãªã¥ãŒã·ã§ã³ã®æäŸã«ãããŠ15幎以äžã®çµéšãæã€ãšã³ãžãã¢ãªã³ã°ãªãŒããŒã§ãã圌ã¯AWS Glue Zero-ETLãšAmazon AppFlowã®ãšã³ãžãã¢ãªã³ã°ããªãŒãããŠããŸãã Parth Panchal Parth ã¯ã10幎以äžã®éçºçµéšãæã€çµéšè±å¯ãªãœãããŠã§ã¢ãšã³ãžãã¢ã§ãAWS Glue zero-ETLãšSAPããŒã¿çµ±åãœãªã¥ãŒã·ã§ã³ãå°éãšããŠããŸãã圌ã¯ãè€éãªããŒã¿ã¬ããªã±ãŒã·ã§ã³ã®èª²é¡ã«æ·±ãåãçµã¿ãããã©ãŒãã³ã¹ãšä¿¡é Œæ§ã®é«ãåºæºãç¶æããªããã¹ã±ãŒã©ãã«ãªãœãªã¥ãŒã·ã§ã³ãæäŸããããšã«åªããŠããŸãã Diego Lombardini Diego ã¯ãSAPãã¯ãããžãŒå
šäœã§20幎以äžã®çµéšãæã€çµéšè±å¯ãªãšã³ã¿ãŒãã©ã€ãºã¢ãŒããã¯ãã§ãSAPã€ãããŒã·ã§ã³ãšããŒã¿ããã³åæãå°éãšããŠããŸãã圌ã¯ããŒãããŒãšãŠãŒã¶ãŒã®äž¡æ¹ãšããŠåããŠãããããã·ã¹ãã ãšçµç¹ã販売ãå®è£
ãéçšããããã«å¿
èŠãªããšã«ã€ããŠå®å
šãªèŠç¹ãæã£ãŠããŸãã圌ã¯ãã¯ãããžãŒãšã€ãããŒã·ã§ã³ã«æ
ç±ãæã¡ãé¡§å®¢ã®ææãšããžãã¹äŸ¡å€ã®æäŸã«çŠç¹ãåœãŠãŠããŸãã Abhijeet Jangam Abhijeet ã¯ãè€æ°ã®æ¥çã«ãããæŠç¥ãšæäŸããªãŒããã20幎ã®SAPæè¡æ©èœçµéšãæã€ããŒã¿ããã³AIãªãŒããŒã§ããæ°åã®SAPå®è£
çµéšã«ããã圌ã¯ã¢ããªã±ãŒã·ã§ã³éçºãããŒã¿ãšã³ãžãã¢ãªã³ã°ãçµ±åã«ãããæ·±ãæè¡çå°éç¥èãšãšãã«ãå¹
åºãæ©èœããã»ã¹ç¥èããããããŸãã ç¿»èš³ïŒ äžäœç² æ (AWS Japan ãœãªã¥ãŒã·ã§ã³ã¢ãŒããã¯ã)