å€ãã®ã客æ§ãSAP ã®ãœãŒã¹ããŒã¿ãš SAP 以å€ã®ãœãŒã¹ããŒã¿ãçµã¿åãããŠæŽ»çšããããšèããŠããŸãããã®ãããªããŒã¿åæã®ãŠãŒã¹ã±ãŒã¹ã¯ãããŒã¿ãŠã§ã¢ããŠã¹ãããŒã¿ã¬ã€ã¯ãæ§ç¯ããããšã§å®çŸã§ããŸããã客æ§ã¯ AWS Glue ã® SAP OData ã³ãã¯ã¿ã䜿çšããŠãSAP ããããŒã¿ãæœåºã§ããŸããSAP OData ã³ãã¯ã¿ã¯ããªã³ãã¬ãã¹åã¯ã¯ã©ãŠã (ãã€ãã£ããš SAP RISE) ã§çšŒåããŠããã·ã¹ãã ã®äž¡æ¹ããµããŒãããŠããŸãã AWS Glue ã® SAP OData ã³ãã¯ã¿ã䜿çšãããšãAWS Glue ãš Apache Spark ã§å¹ççãªåŠçã®ããã«ããŒã¿ãã·ãŒã ã¬ã¹ã«åæ£åŠçã§ããŸããAWS Glue ã¯ããµãŒããŒã¬ã¹ã®ããŒã¿çµ±åãµãŒãã¹ã§ãåæãæ©æ¢°åŠç¿ (ML)ãã¢ããªã±ãŒã·ã§ã³éçºãªã©ã«å¯Ÿããè€æ°ã®ãœãŒã¹ããããŒã¿ãæ€çŽ¢ãæºåãç§»åãçµåããããšãç°¡åã«ãªããŸãã AWS Glue ã® SAP OData ã³ãã¯ã¿ã¯ãããŒã¿æœåºã®ããã« SAP ODP ãã¬ãŒã ã¯ãŒã¯ãš OData ãããã³ã«ã䜿çšããŸãããã®ãã¬ãŒã ã¯ãŒã¯ã¯ããããã€ãã»ãµãã¹ã¯ã©ã€ãã¢ãã«ã§åäœããSAP ã·ã¹ãã ãš SAP 以å€ã®ã¿ãŒã²ããéã®ããŒã¿è»¢éãå®çŸããŠããŸããODP ãã¬ãŒã ã¯ãŒã¯ã¯ãOperational Delta Queues (ODQ) ã¡ã«ããºã ãéããŠããã«ããŒã¿æœåºãšå·®åããŒã¿ãã£ããã£ããµããŒãããŠããŸããSAP ã®ããŒã¿æœåºã®ãœãŒã¹ãšããŠãSAP ããŒã¿ãšã¯ã¹ãã©ã¯ã¿ãABAP CDS ãã¥ãŒãSAP BWã»BW/4 HANA ãœãŒã¹ãSAP ABAP ãœãŒã¹ã® HANA Information ãã¥ãŒããŸã㯠ODP 察å¿ã®ä»ã®ããŒã¿ãœãŒã¹ã䜿çšã§ããŸãã SAP ã®ãœãŒã¹ã·ã¹ãã ã«ã¯å±¥æŽããŒã¿ãä¿æãããŠãããåžžã«æŽæ°ããããŸãããã®ããããœãŒã¹ã®å€æŽãå¢ååŠçã§ããããã«ããããšãå¿
èŠã§ãããã®ããã°ã§ã¯ãSAP ããããŒã¿ãæœåºããSAP ODP ãã¬ãŒã ã¯ãŒã¯ãšãã«ã¿ããŒã¯ã³ã䜿çšã㊠SAP ãœãŒã¹ããã®å¢åããŒã¿æœåºãå®çŸããæ¹æ³ã説æããŸãã ãœãªã¥ãŒã·ã§ã³ã®æŠèŠ SAP ãœãŒã¹ ã·ã¹ãã ã«ä¿åãããŠãã補åããŒã¿ãåæããããšèããŠããäŸã«ãªããŸããã客æ§ã¯ãçŸåšæäŸããŠãã補åã©ã€ã³ããããç¹ã«ååç®ã°ã«ãŒãã«å«ãŸãã補åã®æ°ãææ¡ããããšèããŠããŸããããã«ã¯ãSAP åç®ãã¹ã¿ãŒãš SAP ã·ã¹ãã ã®åç®ã°ã«ãŒã ããŒã¿ ãœãŒã¹ããã®ããŒã¿ãçµåããå¿
èŠããããŸããSAP åç®ãã¹ã¿ãŒ ããŒã¿ã¯å¢åæœåºã§äœ¿çšã§ããŸãããSAP åç®ã°ã«ãŒãã¯ãã« ããŒãã§ã®ã¿äœ¿çšã§ããŸãããããã®ããŒã¿ ãœãŒã¹ãçµåããåæã®ããã«ã¯ãšãªã§ããããã«ããå¿
èŠããããŸãã åææ¡ä»¶ ãã®ããã°ã§ç޹ä»ãããœãªã¥ãŒã·ã§ã³ãå®è£
ããã«ã¯ããŸã次ã®åææ¡ä»¶ã®ã¹ãããã宿œããŠãã ããã SAP ã·ã¹ãã ã® SAP Gateway ã§ãODP ããŒã¿ãœãŒã¹ã䜿ã£ãŠ SAP OData ãµãŒãã¹ãèšå®ããŸãã SAP ããŒã¿ãä¿åããããã® Amazon Simple Storage Service (Amazon S3) ãã±ãããäœæããŸãã AWS Glue Data Catalog ã§ã sapgluedatabase ãšããååã®ããŒã¿ããŒã¹ãäœæããŸãã AWS Glue ã®æœåºã倿ãããŒã (ETL) ãžã§ãã§äœ¿çšãã AWS Identity and Access Management (IAM) ããŒã«ãäœæããŸãããã®ããŒã«ã«ã¯ãAmazon S3 ãš AWS Secrets Manager ãå«ããã¹ãŠã®å¿
èŠãªãªãœãŒã¹ãžã®ã¢ã¯ã»ã¹æš©éãä»äžããå¿
èŠããããŸãããã®ããã°ã®ãœãªã¥ãŒã·ã§ã³ã§ã¯ããã®ããŒã«ã GlueServiceRoleforSAP ãšåä»ããŸãã以äžã®ããªã·ãŒã䜿çšããŸãã AWS 管çããªã·ãŒ: AWSGlueServiceRole SecretsManagerReadWrite ã€ã³ã©ã€ã³ããªã·ãŒ: { "Version": "2012-10-17", "Statement": [ { "Sid": "VisualEditor0", "Effect": "Allow", "Action": [ "s3:PutObject", "s3:GetObjectAcl", "s3:GetObject", "s3:GetObjectAttributes", "s3:ListBucket", "s3:DeleteObject", "s3:PutObjectAcl"], "Resource": [ "arn:aws:s3:::<S3-BUCKET-NAME>", "arn:aws:s3:::<S3-BUCKET-NAME>/*" ] } ] } SAP çšã® AWS Glue æ¥ç¶å
ã®äœæ SAP ã³ãã¯ã¿ã¯ãCUSTOM ( SAP BASIC èªèšŒ) ãš OAUTH ã®äž¡æ¹ã®èªèšŒæ¹åŒããµããŒãããŠããŸãããã®äŸã§ã¯ãBASIC èªèšŒã§æ¥ç¶ããŸãã AWS ãããžã¡ã³ãã³ã³ãœãŒã« ã§ AWS Secrets Manager ãµãŒãã¹ãéããŠã ODataGlueSecret ãšããååã®ã·ãŒã¯ã¬ãããäœæããŸããAWS Secrets Manager ã®è©³çްã«ã¯ã次ã®ã³ãŒããå«ããå¿
èŠããããŸãã <your SAP username> ã®éšåã«ããªãã® SAP ã·ã¹ãã ã®ãŠãŒã¶ãŒåãã <your SAP username password> ã®éšåã«ãã®ãã¹ã¯ãŒããå
¥åããŸãã { "basicAuthUsername": " <your SAP username> ", "basicAuthPassword": " <your SAP username password> ", "basicAuthDisableSSO": "True", "customAuthenticationType": "CustomBasicAuth" } SAP OData ããŒã¿ãœãŒã¹ãéžæããŠãAWS Glue ã§ SAP ãžã®æ¥ç¶å
GlueSAPOdata ãäœæããŸãã SAP ãœãŒã¹ã®é©åãªå€ã䜿çšããŠæ¥ç¶ãæ§æããŸãã ã¢ããªã±ãŒã·ã§ã³ãã¹ã URL : ãã¹ãã«ã¯ãSAP ãã¹ãåãèªèšŒããããã® SSL èšŒææžãå¿
èŠ ã¢ããªã±ãŒã·ã§ã³ãµãŒãã¹ãã¹: /sap/opu/odata/iwfnd/catalogservice;v=2; ããŒãçªå· : SAP ãœãŒã¹ã·ã¹ãã ã®ããŒãçªå· ã¯ã©ã€ã¢ã³ãçªå· : SAP ãœãŒã¹ã·ã¹ãã ã®ã¯ã©ã€ã¢ã³ãçªå· ãã°ãªã³èšèª : SAP ãœãŒã¹ã·ã¹ãã ã®ãã°ãªã³èšèª èªèšŒ ã»ã¯ã·ã§ã³ã§ã èªèšŒã¿ã€ã ã« CUSTOM ãéžæããŸãã åã®æé ã§äœæãã ã·ãŒã¯ã¬ãã SAPODataSecret ãéžæããŸãã ãããã¯ãŒã¯ãªãã·ã§ã³ ã»ã¯ã·ã§ã³ã§ãSAP ã·ã¹ãã ãžæ¥ç¶ããããã® VPC ã ãµãããã ã ã»ãã¥ãªãã£ã°ã«ãŒã ãå
¥åããŸããSAP ã·ã¹ãã ãžã®æ¥ç¶ã®è©³çްã«ã€ããŠã¯ã ETL ãžã§ãçšã® VPC ãæ§æãã ãåç
§ããŠãã ããã SAP ããããŒã¿ãåã蟌ãããã® ETL ãžã§ãã®äœæ AWS Glue ã³ã³ãœãŒã«ã§ãæ°ããããžã¥ã¢ã«ãšãã£ã¿ãŒã§ãžã§ããäœæããŸãã AWS Glue ã³ã³ãœãŒã«ã«ã¢ã¯ã»ã¹ããŸãã ããã²ãŒã·ã§ã³ãã€ã³ã® ETL ãžã§ã ã®äžã«ãã Visual ETL ãéžæããŸãã Visual ETL ãéžæããŠãããžã¥ã¢ã«ãšãã£ã¿ãŒã§ãžã§ããäœæããŸãã ãžã§ãåãããã©ã«ãã®åç§°ãã Material Master Job ã«ç·šéãã ä¿å ãéžæããŸãã ããžã¥ã¢ã«ãšãã£ã¿ãŒãã£ã³ãã¹äžã§ãSAP ãœãŒã¹ãéžæããŸãã Visual ã¿ããéžæãããã©ã¹èšå·ãã¯ãªãã¯ã㊠Add nodes ã¡ãã¥ãŒãéããŸããSAP ãæ€çŽ¢ããSAP OData Source ã远å ããŸãã 远å ããããŒãã®ååã Material Master Attributes ã«èšå®ããŸãã SAP OData connection ã§ã¯ã GlueSAPOData æ¥ç¶å
ãéžæããŸãã SAP ãœãŒã¹ãã補åãã¹ã¿ããµãŒãã¹ããšã³ãã£ãã£ã»ãããéžæããŸãã Entity Name ãš Sub Entity Name ã§ã¯ãSAP ãœãŒã¹ã® SAP OData ãšã³ãã£ãã£ãéžæããŸãã Fields ããã MaterialãCreated onãMaterial GroupãMaterial TypeãOld Matl numberãGLUE_FETCH_SQãDELTA_TOKENãDML_STATUS ãéžæããŸãã ãã£ã«ã¿ãŒã« limit 100 ãšå
¥åãããã¬ãã¥ãŒã®è¡šç€ºã«ãããæéãå¶éããŸãã ãã® OData ãµãŒãã¹ã¯å·®åæœåºããµããŒãããŠããããã å¢å転é ãªãã·ã§ã³ãããã©ã«ãã§éžæãããŠããŸãã AWS Glue ãµãŒãã¹ããŒã«ã®è©³çްãéžæããåŸãããŒã¿ãã¬ãã¥ãŒã䜿çšå¯èœã«ãªããŸãããã¬ãã¥ãŒã§è¡šç€ºãããããã«ã次㮠3 ã€ã®æ°ãããã£ãŒã«ããå«ããããšãã§ããŸããåãã£ãŒã«ãã®æå³ã¯äžèšã®éãã§ãã glue_fetch_sq : ããã¯ã·ãŒã±ã³ã¹ãã£ãŒã«ãã§ãã¬ã³ãŒããåä¿¡ãããé åºã® EPOC ã¿ã€ã ã¹ã¿ã³ãããçæãããåã¬ã³ãŒãã§æã€ãŠããŒã¯ã®å€ã§ãããœãŒã¹ã·ã¹ãã ã®å€æŽé åºã確èªããå¿
èŠãããå Žåã«äœ¿çšã§ããŸãã delta_token : æåŸã«æœåºãããã¬ã³ãŒãã®ã¿ã«å€ãå«ãŸãããã以å€ã®ãã¹ãŠã®ã¬ã³ãŒãã«ã¯ç©ºçœã«ãªããŸããããã¯å€æŽãããã¬ã³ãŒã (CDC) ããã£ããã£ããããã® ODQ ããŒã¯ã³å€ã§ãããã®ã¬ã³ãŒãã¯ãœãŒã¹ããã®ãã©ã³ã¶ã¯ã·ã§ã³ããŒã¿ã¬ã³ãŒãã§ã¯ãªãããã«ã¿ããŒã¯ã³å€ãæž¡ãç®çã ãã§ååšããŸãã dml_status : ããã¯ããœãŒã¹ããæ°èŠæ¿å
¥ããã³æŽæ°ãããã¬ã³ãŒãã«å¯Ÿã㊠UPDATEDããœãŒã¹ããåé€ãããã¬ã³ãŒãã«å¯Ÿã㊠DELETED ãšè¡šç€ºãããŸãã ãã«ã¿å¯Ÿå¿ã®æœåºã§ã¯ãæåŸã«æœåºãããã¬ã³ãŒãã« DELTA_TOKEN ã®å€ãå«ãŸããäžèšã®ããã« delta_token ãã£ãŒã«ããåããããŸãã ããžã¥ã¢ã«ãã£ã³ãã¹ã«å¥ã® SAP ODATA ãœãŒã¹æ¥ç¶ã远å ãããã®ããŒãã Material Group Text ãšåä»ããŠãã ããã SAP ãœãŒã¹ãã補åã°ã«ãŒãã® OData ãµãŒãã¹ãšãšã³ãã£ãã£ã»ãããéžæããŸã Entity Name ãš Sub Entity Name ã«ã€ããŠã¯ãSAP ãœãŒã¹ã® SAP OData ãšã³ãã£ãã£ãéžæããŸã ãã® ãµãŒãã¹ã¯ãã«æœåºã®ã¿ããµããŒãããŠããããã å®å
šè»¢éãªãã·ã§ã³ ãããã©ã«ãã§éžæãããŠããŸãããŸãããã®ããŒã¿ã»ããããã¬ãã¥ãŒè¡šç€ºããããšãã§ããŸãã ããŒã¿ããã¬ãã¥ãŒã§è¡šç€ºãããšãã¯ã language key ã«æ³šç®ããŠãã ããããã£ã«ã¿ããªããã° SAP ã¯ãã¹ãŠã®èšèªãæž¡ãããã«ãªãããã SPRAS = 'E' ãšãããã£ã«ã¿ãŒã远å ããŠãè±èªã®ã¿ãæœåºããŸããããã®ãã£ã«ã¿âã®å€ã¯ãã£ãŒã«ãã® SAP å
éšæžåŒã®å€ã䜿ããŸãã Material Group Text ã®åŸã«ããã£ã³ãã¹ã« Change Schema 倿ããŒãã远å ããŸãã target key ã§ã補åã°ã«ãŒããã£ãŒã«ãã®ååã matkl2 ã«å€æŽããŸããããã«ãããæåã®ãœãŒã¹ãšã¯ç°ãªãååã«ãªããŸãã Drop ãã§ãã¯ããã¯ã¹ã§ã spras ã odq_changemode ã odq_entitycntr ã dml_statusãdelta_token ã glue_fetch_sq ãéžæããŸãã ãã£ã³ãã¹ã« Join 倿ããŒãã远å ããäž¡æ¹ã®ãœãŒã¹ããŒã¿ã»ãããçµåããŸãã Node parents ã§ Material Master Attributes ãš Change Schema äž¡æ¹ãéžæããŸãã Join type ãšã㊠Left join ãéžæããŸãã åãœãŒã¹ã®ããŒãã£ãŒã«ãã Join conditions ãšããŠèšå®ããŸãã Material Master Attributes ã§ã¯ matkl ãéžæããŸã Change Schema ã§ã¯ matkl2 ãéžæããŸã åºåããã¬ãã¥ãŒããŠãæ£ããããŒã¿ãèªã¿èŸŒãŸããŠããããšã確èªã§ããŸããããã§çµæãä¿åããæºåãã§ããŸããã ãã£ã³ãã¹ã«ã¿ãŒã²ããã® S3 ãã±ããã远å ããŸãã ããŒãã®èŠªã Join ã«æå®ããŸãã ãã©ãŒããã ã§ã¯ã Parquet ãéžæããŸãã S3 ã¿ãŒã²ãããã±ãŒã·ã§ã³ ã§ã¯ãåææ¡ä»¶ã®ç« ã§äœæãã S3 ãã±ãããåç
§ãã materialmaster/ ã S3 ã¿ãŒã²ãããã±ãŒã·ã§ã³ ã«è¿œå ããŸãã ããŒã¿ã«ã¿ãã°æŽæ°ãªãã·ã§ã³ ã§ã¯ã Create a table in the Data Catalog and on subsequent runs, update the schema and add new partitions ãéžæããŸãã ããŒã¿ããŒã¹ ã§ã¯ãåã«äœæãã AWS Glue ããŒã¿ããŒã¹ sapgluedatabase ã®ååãéžæããŸãã ããŒãã«å ã«ã¯ materialmaster ãšå
¥åããŸãã ä¿åãã¿ã³ ãã¯ãªãã¯ããŠãžã§ããä¿åããŸãããžã§ãã¯æ¬¡ã®å³ã®ããã«ãªããŸãã ETL ãžã§ãã®ã¯ããŒã³ããŠå·®å察å¿å®æœ ETL ãžã§ããäœæãããããã«ã¿ããŒã¯ã³ã䜿çšããŠã€ã³ã¯ãªã¡ã³ã¿ã«ããŒã¿åŠçãå«ããããã®ã¯ããŒã³ã®æºåãæŽããŸãã ãããè¡ãã«ã¯ããžã§ãã¹ã¯ãªãããçŽæ¥ä¿®æ£ããå¿
èŠããããŸããã¹ã¯ãªãããä¿®æ£ããŠãæåŸã®ãã«ã¿ããŒã¯ã³ (ãžã§ãã¿ã°ã«æ ŒçŽããã) ãååŸããã¹ããŒãã¡ã³ãã远å ãããã«ã¿ããŒã¯ã³å€ããªã¯ãšã¹ã (ãŸãã¯ãžã§ãã®å®è¡) ã«è¿œå ããŸããããã«ãããæ¬¡åã®ãžã§ãå®è¡æã«ããŒã¿ãååŸããéã«ããã«ã¿å¯Ÿå¿ã® SAP OData ãµãŒãã¹ãæå¹ã«ãªããŸãã æåã®ãžã§ãã®å®è¡ã§ã¯ãã¿ã°ã«ãã«ã¿ããŒã¯ã³å€ããªããããåŒã³åºãã¯ååå®è¡ãšãªãããã®åŸãã«ã¿ããŒã¯ã³ãã¿ã°ã«æ ŒçŽãããŠãå°æ¥ã®å®è¡ã«äœ¿çšãããŸãã AWS Glue ã³ã³ãœãŒã«ã«ç§»åããŸãã ããã²ãŒã·ã§ã³ãã€ã³ã® ETL Jobs ã®äžã«ãã Visual ETL ãéžæããŸãã Material Master Job ãéžæãã Actions ãéžãã§ Clone job ãéžæããŸãã ãžã§ãã®ååã Material Master Job Delta ã«å€æŽãã Script ã¿ããéžæããŸãã åãžã§ãå®è¡æã®ãã«ã¿ããŒã¯ã³ã®ä¿åãšååŸãè¡ãããã®è¿œå ã® Python ã©ã€ãã©ãªã远å ããå¿
èŠããããŸãããããè¡ãã«ã¯ã Job Details ã¿ãã«ç§»åããäžã«ã¹ã¯ããŒã«ã㊠Advanced Properties ã»ã¯ã·ã§ã³ãå±éããŸãã Python library path ã«æ¬¡ã®ãã¹ã远å ããŸãã s3://aws-blogs-artifacts-public/artifacts/BDB-4789/sap_odata_state_management.zip 次㫠Script ã¿ããéžæããå³äžã® Edit script ãéžæããŸããConfirm ãéžæããŠããžã§ããã¹ã¯ãªããã®ã¿ã§ããããšã確èªããŸãã ãã«ã¿ããŒã¯ã³ãæå¹ã«ããã«ã¯ãã¹ã¯ãªããã«æ¬¡ã®å€æŽãå ããŠãã ããã ã¹ããã 5 ã§è¿œå ãã SAP OData ã¹ããŒã管çã©ã€ãã©ãªã¯ã©ã¹ãã€ã³ããŒããããããæ¬¡ã®ã³ãŒãã 8 è¡ç®ã«è¿œå ããŸãã from sap_odata_state_management.state_manager import StateManagerFactory, StateManagerType, StateType æ¬¡ã®æ°ã¹ãããã§ã¯ããã«ã¿ããŒã¯ã³ããžã§ãã¿ã°ã«ååŸããŠæ°žç¶åããåŸç¶ã®ãžã§ãå®è¡ããã¢ã¯ã»ã¹ã§ããããã«ããŸãããã«ã¿ããŒã¯ã³ã¯ SAP ãœãŒã¹ãžã®èŠæ±ã«è¿œå ãããå¢å倿ŽãæœåºãããŸãã ããŒã¯ã³ãæž¡ãããªãå ŽåãããŒãã¯ååããŒããšããŠå®è¡ãããããŒã¯ã³ã次åã®å®è¡çšã«æ°žç¶åãããŸãããã®æ¬¡ã®å®è¡ã¯ãã«ã¿ããŒãã«ãªããŸãã sap_odata_state_management ã©ã€ãã©ãªãåæåãããããæ¥ç¶ãªãã·ã§ã³ã倿°ã§å®çŸ©ããŠãã¹ããŒããããŒãžã£ãŒã§ãã®å€æ°ã®å€ãå®çŸ©ããŸãããããè¡ãã«ã¯ã job.init ã¹ããŒãã¡ã³ãã®åŸ 16 è¡ç®ã«æ¬¡ã®ã³ãŒãã远å ããŸãã <key of MaterialMasterAttributes node> ãš <entityName for Material Attribute> ã¯ãæ¢åã®çæãããã¹ã¯ãªããã® # Script generated for node Material Master Attributes ã³ãŒãã«å€ãå
¥ã£ãŠããã®ã§ããããã³ããŒã§ããŸããé©åãªå€ã«çœ®ãæããŠãã ããã key = "<key of MaterialMasterAttributes node>" state_manager = StateManagerFactory.create_manager( manager_type=StateManagerType.JOB_TAG, state_type=StateType.DELTA_TOKEN, options={"job_name": args['JOB_NAME'], "logger": glueContext.get_logger()} ) options = { "connectionName": "GlueSAPOData", "entityName": "<entityName for Material Attribute>", "ENABLE_CDC": "true" } connector_options = state_manager.get_connector_options(key) options.update(connector_options) Material Master Attributes ããŒãã«çæãããæ¢åã®ã¹ã¯ãªããã®åã« # ãä»ãå ããŠã³ã¡ã³ãã¢ãŠãããæ¬¡ã®çœ®æã¹ããããã远å ããŸãã <key of MaterialMasterAttributes node> = glueContext.create_dynamic_frame.from_options(connection_type="sapodata", connection_options=options, transformation_ctx="<key of MaterialMasterAttributes node>") ãã€ãããã¯ãã¬ãŒã ãããã«ã¿ããŒã¯ã³ãèªã¿åã£ãŠããžã§ãã®ã¿ã°ã«ä¿åããã«ã¯ãã¹ã¯ãªããã®æåŸã®è¡ job.commit() ã®åã«æ¬¡ã®ã³ãŒãã¹ããããã远å ããŸãã state_manager.update_state(key, <key of MaterialMasterAttributes node>.toDF()) æçµçã«ã¹ã¯ãªããã¯æ¬¡ã®ããã«ã«ãªãã€ã¡ãŒãžã§ãã import sys from awsglue.transforms import * from awsglue.utils import getResolvedOptions from pyspark.context import SparkContext from awsglue.context import GlueContext from awsglue.job import Job from awsglue.dynamicframe import DynamicFrame from sap_odata_state_management.state_manager import StateManagerFactory, StateManagerType, StateType args = getResolvedOptions(sys.argv, ['JOB_NAME']) sc = SparkContext() glueContext = GlueContext(sc) spark = glueContext.spark_session job = Job(glueContext) job.init(args['JOB_NAME'], args) key = "MaterialMasterAttributes_node1730873953236" state_manager = StateManagerFactory.create_manager( manager_type=StateManagerType.JOB_TAG, state_type=StateType.DELTA_TOKEN, options={"job_name": args['JOB_NAME'], "logger": glueContext.get_logger()} ) options = { "connectionName": "GlueSAPOData", "entityName": "/sap/opu/odata/sap/ZMATERIAL_ATTR_SRV/EntityOf0MATERIAL_ATTR", "ENABLE_CDC": "true" } # Script generated for node Material Group Text MaterialGroupText_node1730874412841 = glueContext.create_dynamic_frame.from_options(connection_type="sapodata", connection_options={"ENABLE_CDC": "false", "connectionName": "GlueSAPOData", "FILTER_PREDICATE": "SPRAS = 'E'", "ENTITY_NAME": "/sap/opu/odata/sap/ZMATL_GROUP_SRV/EntityOf0MATL_GROUP_TEXT"}, transformation_ctx="MaterialGroupText_node1730874412841") # Script generated for node Material Master Attributes #MaterialMasterAttributes_node1730873953236 = glueContext.create_dynamic_frame.from_options(connection_type="sapodata", connection_options={"ENABLE_CDC": "true", "connectionName": "GlueSAPOdata", "FILTER_PREDICATE": "limit 100", "SELECTED_FIELDS": "MATNR,MTART,MATKL,BISMT,ERSDA,DML_STATUS,DELTA_TOKEN,GLUE_FETCH_SQ", "ENTITY_NAME": "/sap/opu/odata/sap/ZMATERIAL_ATTR_SRV/EntityOf0MATERIAL_ATTR"}, transformation_ctx="MaterialMasterAttributes_node1732755261264") MaterialMasterAttributes_node1730873953236 = glueContext.create_dynamic_frame.from_options(connection_type="sapodata", connection_options=options, transformation_ctx="MaterialMasterAttributes_node1730873953236") # Script generated for node Change Schema ChangeSchema_node1730875214894 = ApplyMapping.apply(frame=MaterialGroupText_node1730874412841, mappings=[("matkl", "string", "matkl2", "string"), ("txtsh", "string", "txtsh", "string")], transformation_ctx="ChangeSchema_node1730875214894") # Script generated for node Join MaterialMasterAttributes_node1730873953236DF = MaterialMasterAttributes_node1730873953236.toDF() ChangeSchema_node1730875214894DF = ChangeSchema_node1730875214894.toDF() Join_node1730874996674 = DynamicFrame.fromDF(MaterialMasterAttributes_node1730873953236DF.join(ChangeSchema_node1730875214894DF, (MaterialMasterAttributes_node1730873953236DF['matkl'] == ChangeSchema_node1730875214894DF['matkl2']), "left"), glueContext, "Join_node1730874996674") # Script generated for node Amazon S3 AmazonS3_node1730875848117 = glueContext.write_dynamic_frame.from_options(frame=Join_node1730874996674, connection_type="s3", format="json", connection_options={"path": "s3://sapglueodatabucket", "compression": "snappy", "partitionKeys": []}, transformation_ctx="AmazonS3_node1730875848117") state_manager.update_state(key, MaterialMasterAttributes_node1730873953236.toDF()) job.commit() 倿Žãä¿åããã«ã¯ã ä¿å ãã¿ã³ãéžæããŸãã å®è¡ ãã¯ãªãã¯ããŠãžã§ããå®è¡ããŸãããã®æç¹ã¯ããžã§ãã®è©³çްã¿ãã«ã¯ã¿ã°ããããŸããã ãžã§ãã®å®è¡ãæ£åžžã«å®äºãããŸã§åŸ
ã¡ãŸããã¹ããŒã¿ã¹ã¯ å®è¡ ã¿ãã§ç¢ºèªã§ããŸãã ãžã§ãã®å®è¡ãå®äºãããšã ãžã§ãã®è©³çŽ°ã® ã¿ãã§ã¿ã°ã远å ãããããšãããããŸããæ¬¡ã®ãžã§ãã®å®è¡ã§ã¯ããã®ããŒã¯ã³ãèªã¿åãããã«ã¿ããŒããå®è¡ãããŸãã SAP ããã®ããŒã¿ã«å¯Ÿããã¯ãšãª AWS Glue ãžã§ãã®å®è¡ã«ãããData Catalog ã«ãšã³ããªãäœæãããã®ã§ãããã«ããŒã¿ãã¯ãšãªããããšãã§ããŸãã Amazon Athena ã³ã³ãœãŒã«ãéããŸãã Launch Query Editor ãéžæããŸãã é©åãªã¯ãŒã¯ã°ã«ãŒããå²ãåœãŠãããŠããããšã確èªããããå¿
èŠã«å¿ã㊠ã¯ãŒã¯ã°ã«ãŒããäœæ ããŸãã sapgluedatabase ãéžæããæ¬¡ã®ãããªã¯ãšãªãå®è¡ããŠããŒã¿ã®åæãéå§ããŸãã select matkl, txtsh, count(*) from materialmaster group by 1, 2 order by 1, 2 ; ã¯ãªãŒã³ã¢ãã è¿œå æéãçºçããªãããããã®ããã°ã§äœ¿çšãã AWS ãªãœãŒã¹ãåé€ããŸããåé€ãããªãœãŒã¹ã¯ãAWS Glue ãžã§ããSAP OData æ¥ç¶å
ãGlue ããŒã¿ã«ã¿ãã°ãšã³ããªãSecrets Manager ã®ã·ãŒã¯ã¬ãããIAM ããŒã«ãS3 ãã±ããã«ä¿åããŠãããã¡ã€ã«ãS3 ãã±ããã§ãã çµè« ãã®ããã°ã§ã¯ããµãŒããŒã¬ã¹ã§è€æ°ã® SAP ããŒã¿ãœãŒã¹ããã®ããŒã¿æœåºãšå·®åæœåºããã»ã¹ãäœæããæ¹æ³ã説æããŸããããã®æ¹æ³ã§ã¯ãAWS Glue ã䜿çšã㊠SAP ODP ãã«ã¿ããŒã¯ã³ãå©çšããSAP ãœãŒã¹ããããŒã¿ãå·®åã§æœåºããAmazon S3 ã«ããŒã¿ãããŒãããŸããã AWS Glue ã¯ãµãŒããŒã¬ã¹ã®ãµãŒãã¹ãªã®ã§ãã€ã³ãã©ã¹ãã©ã¯ãã£ã®ç®¡çã¯äžèŠã§ããžã§ãã®å®è¡äžã«äœ¿çšããããªãœãŒã¹ã«æéãçºçããŸã (ããŒã¿ä¿åå
ã®ã¹ãã¬ãŒãžã³ã¹ããçºçããŸã)ãçµç¹ããŸããŸãããŒã¿ããªãã³ã«ãªãã«ã€ãããã® SAP ã³ãã¯ã¿ã¯ãSAP ãœãŒã¹ããŒã¿ãããã°ããŒã¿ã®åæãã³ã¹ãã«å¹æçã§é«æ§èœãäžã€ã»ãã¥ã¢ã«åã蟌ãããšãã§ããŸãã詳现㯠AWS Glue ãã芧ãã ããã 翻蚳㯠Specialist SA ãã¥ã¢ã³ãæ
åœããŸãããåæã¯ ãã¡ã ã§ãã èè
ã«ã€ã㊠Allison Quinn ã¯ã¡ã«ãã«ã³ (ãªãŒã¹ãã©ãªã¢) ã«æ ç¹ã眮ã Sr. ANZ Analytics Specialist Solutions Architect ã§ãåå°åã®éèãµãŒãã¹é¡§å®¢ãšå¯æ¥ã«ååããŠããŸããAllison 㯠SAP 補åã§ 15 幎以äžã®çµéšããããçŸåšã¯ AWS ãã€ãã£ããµãŒãã¹ã«åææè¡ã®å°éæ§ãéäžãããŠããŸããããŒã¿ã«é¢ãããã¹ãŠã®ããšã«æ
ç±ãæã¡ãããããçš®é¡ã®é¡§å®¢ãããžãã¹äžã®å©çãåŸãããããããŒã¿ã®æ°äž»åãç®æããŠããŸãã Pavol ã¯ãAWS ã®ã€ãããŒã·ã§ã³ãœãªã¥ãŒã·ã§ã³ã¢ãŒããã¯ãã§ãEMEA å°åã«ããã SAP ã¯ã©ãŠãå°å
¥ãå°éãšããŠããŸãã20 幎以äžã®çµéšãæã¡ãã°ããŒãã«é¡§å®¢ã® AWS ãžã® SAP ã·ã¹ãã ã®ç§»è¡ãšæé©åãæ¯æŽããŠããŸããPavol ã¯ãAWS ã®ã¢ãžãªãã£ãã¬ãžãªãšã³ã·ãŒãããã©ãŒãã³ã¹ã掻çšããŠãSAP ç°å¢ãã¯ã©ãŠãã«ç§»è¡ããããã®æŠç¥ãç«æ¡ããŠããŸãã顧客ã«å¯ŸããŠãAWS ã® AI/MLãããŒã¿åæãã¢ããªã±ãŒã·ã§ã³ãµãŒãã¹ãå©çšã㊠SAP ã©ã³ãã¹ã±ãŒããè¿ä»£åããã€ã³ããªãžã§ã³ã¹ãèªååãããã©ãŒãã³ã¹ãåäžãããã®ãæ¯æŽããŠããŸãã Partha Pratim Sanyal ã¯ãã«ããã®ãã³ã¯ãŒããŒã«ãã AWS Glue ã®ãœãããŠã§ã¢éçºãšã³ãžãã¢ã§ãããŒã¿çµ±åãåæãæ¥ç¶ã«ç¹åããŠããŸããè±å¯ãªããã¯ãšã³ãéçºã®å°éç¥èãæã¡ã顧客äžå¿ã®ã€ã³ãã¯ãã®ããåªãããœãªã¥ãŒã·ã§ã³ãäœãããšã«å°œåããŠããŸãããŠãŒã¶ãŒãããŒã¿ãç°¡åã«åæã»çè§£ã§ãããããªæ©èœãæ§ç¯ããããšã«éç¹ã眮ããŠããŸããPartha ã¯ãè€éãªãŠãŒã¶ãŒããŒãºã«å¯ŸåŠããããŒã¿ã®ã¢ã¯ã»ã·ããªãã£ãšæŽå¯åãé«ããããã®çŽæçã§äŸ¡å€ã®ããäœéšãåµåºããããšã«ç±å¿ã§ãã Diego ã¯ãSAP ãã¯ãããžãŒã« 20 幎以äžã®çµéšãæã€ãšã³ã¿ãŒãã©ã€ãºãœãªã¥ãŒã·ã§ã³ã¢ãŒããã¯ãã§ãSAP ã®ã€ãããŒã·ã§ã³ãããŒã¿åæã«ç¹åããŠããŸããããŒãããŒãšããŠã顧客ãšããŠãåããçµéšããããã·ã¹ãã ãšçµç¹ã販売ãå®è£
ãéçšããããã«å¿
èŠãªããšãå®å
šã«çè§£ããŠããŸãããã¯ãããžãŒãšã€ãããŒã·ã§ã³ã«æ
ç±ãæã¡ãé¡§å®¢ã®ææãšããžãã¹äŸ¡å€ã®æäŸã«éç¹ã眮ããŠããŸãã Luis Alberto Herrera Gomez ã¯ããã³ã¯ãŒããŒã® AWS Glue ã§ãœãããŠã§ã¢éçºãšã³ãžãã¢ãåããŠããŸããããã¯ãšã³ããšã³ãžãã¢ãªã³ã°ããã€ã¯ããµãŒãã¹ãã¯ã©ãŠãã³ã³ãã¥ãŒãã£ã³ã°ã«ç¹åããŠããŸããAmazon ãš AWS ã«å
¥ç€Ÿããåã«è€æ°ã®ã¹ã¿ãŒãã¢ããã§ããã¯ãšã³ãããã³ãã«ã¹ã¿ãã¯éçºè
ãçµéšãã7 ã 8 幎ã®çµéšããããŸããLuis ã¯ãã¹ã±ãŒã©ãã«ã§å¹ççãªã¯ã©ãŠãããŒã¹ã®ã¢ããªã±ãŒã·ã§ã³ã®éçºã«æ³šåããŠããŸããAWS ãã¯ãããžãŒã«é¢ããå°éç¥èãæŽ»ãããè€éãªããŒã¿åŠçã¿ã¹ã¯ãåŠçã§ãã髿§èœã·ã¹ãã ãèšèšããŠããŸããLuis ã¯ãã¯ã©ãŠãã³ã³ãã¥ãŒãã£ã³ã°ã掻çšããŠãé£ããããžãã¹èª²é¡ã解決ããããšã«æ
ç±ãæã£ãŠããŸãã