.table-of-contents ul ul { display: none; } ã¯ããã« ããã«ã¡ã¯ãããŒã¿ã·ã¹ãã éšMLOpsãããã¯ã® å²¡æ¬ ã§ãã MLOpsãããã¯ã§ã¯WEAR by ZOZOïŒä»¥äžWEARïŒãZOZOTOWNã®ã¬ã³ã¡ã³ãã·ã¹ãã ãéçºã»éçšããŠããŸãã WEARã®ã³ãŒãã£ããŒã詳现ç»é¢ã«ã¯ã衚瀺ããŠããã³ãŒãã£ããŒãã«é¢é£æ§ãé«ãã³ãŒãã£ããŒãã衚瀺ããé¢é£æ ããããŸããä»åãWEARã®ã³ãŒãã£ããŒã詳现ç»é¢ã®é¢é£æ ã«ãããã³ãŒãã£ããŒãã®è¡šç€ºããžãã¯ããã«ãŒã«ããŒã¹ããMLã䜿ã£ãããžãã¯ã«çœ®ãæããŸãããæ°ãã«éçºããMLã䜿ã£ãããžãã¯ãã以äžã§ã¯é¢é£ã³ãŒãã¬ã³ã¡ã³ãæ©èœãšåŒãã§ããŸãã説æã®ããã以äžã§ã¯é¢é£ã³ãŒãã¬ã³ã¡ã³ãæ©èœãæ¬æ©èœãšèšèŒããŸãã æ¬æ©èœã®éçºã§ã¯ãã¯ãã«æ€çŽ¢æè¡ãå©çšããŠããŸãããã¯ãã«æ€çŽ¢ã®å®è£
ã«ã¯ãGoogle CloudãæäŸãããããŒãžããªãã¯ãã«æ€çŽ¢ãµãŒãã¹ã§ãã Vertex AI Vector Search ãå©çšããŸããã Vertex AI Vector Searchã§ãã¯ãã«æ€çŽ¢æ©èœãå®è£
ããéã«ãGoogle Cloudã®å
¬åŒããã¥ã¡ã³ãã®èª¬æã ãã§ã¯çè§£ãã«ããã£ãç¹ãªã©ããã€ãèŠåŽããããŸããããŸãæ¬çªç°å¢ã®ã·ã¹ãã ã®äžéšãšããŠéçšããééãã課é¡ããã®è§£æ±ºçã«ã€ããŠãåŠã³ããããŸããã æ¬èšäºã§ã¯ãWEARã®é¢é£ã³ãŒãã¬ã³ã¡ã³ããããžã§ã¯ãã®ãã¯ãã«æ€çŽ¢ã®éçºã«Vertex AI Vector Searchãå°å
¥ããå®è£
ã»æ¬çªéçšã§åŸãããç¥èŠã説æããŸããVertex AI Vector Searchã䜿ã£ãŠãã¯ãã«æ€çŽ¢æ©èœãå®è£
ããéã®åèã«ãªãã°å¹žãã§ãã èšäºã®å
容ã¯2025幎7ææç¹ã§ã®æ
å ±ã§ããããšã«ãçæãã ããã ç®æ¬¡ ã¯ããã« ç®æ¬¡ ãããžã§ã¯ã説æãšVertex AI Vector Searchã®å°å
¥èæ¯ é¢é£ã³ãŒãã¬ã³ã¡ã³ããããžã§ã¯ãã®æŠèŠ ã¬ã³ã¡ã³ãããžãã¯ã®æŠèŠ ã°ã©ããã¥ãŒã©ã«ãããã¯ãŒã¯ã¢ãã«ã«ããEmbeddingã®æœåº 2段éã®ãã¯ãã«æ€çŽ¢ Vertex AI Vector Searchã®å°å
¥èæ¯ éå»ãããžã§ã¯ãã«ããããã¯ãã«æ€çŽ¢æ©èœã®å®è£
Vertex AI Vector Searchã®å°å
¥ã¢ãããŒã·ã§ã³ ãã¯ãã«æ€çŽ¢æ©èœãæã€ãããŒãžããµãŒãã¹ã®æ¯èŒ Vertex AI Vector SearchãšScaNNã¢ã«ãŽãªãºã ã®æŠèŠ Vertex AI Vector Searchã®æŠèŠ ScaNNã¢ã«ãŽãªãºã ã®æŠèŠ Indexæ§ç¯ã®å®è£
ãšãã©ã¡ãŒã¿å€ã®æ±ºå® Indexã®äœæã»æŽæ°ãã㌠Indexæ§ç¯ã®å®è£
Indexäœæã®å®è£
Index Endpointã®äœæãšIndexã®ããã〠Indexæ§æãã©ã¡ãŒã¿ã®æ±ºå® ã¯ãšãªå®è¡æã«äžæžãããŠãããã©ã¡ãŒã¿ Indexæ§ææã«æå®ããŠãããã©ã¡ãŒã¿ Indexãµã€ãºã®èŠç©ãããšã·ã£ãŒããµã€ãºã®æ±ºå® Indexã®ãããã€æŠç¥ãšEmbeddingã®æŽæ°ã»å逿¹æ³ Indexã®ãããã€æŠç¥ Indexã®æŽæ°ã»åé€ Indexã®æŽæ°æ¹æ³ Indexã®å逿¹æ³ ãã¯ãã«æ€çŽ¢ã¯ãšãªå®è¡ãšãã£ã«ã¿ãªã³ã°ã®å®è£
ã¯ãšãªå®è¡ã®å®è£
ãã£ã«ã¿ãªã³ã°ã®å®è£
Vertex AI Vector Searchã®æ¬çªéçšãšèª²é¡ã®æ¹å ã¢ãã¿ãªã³ã°éçš éçšããŠæããã¡ãªãã éçšã»ä¿å®ã®å®¹æã å®å®ããããã©ãŒãã³ã¹ éçšããŠèŠã€ãã£ã課é¡ãšæ¹å Indexã®æŽæ°æéãé·ã Indexã®ããŒã¿æŽæ°è²»çšã®å¢å EmbeddingæŽæ°å¯Ÿè±¡ã®ããŒã¿æ°ãåæž Indexã®ããŒã¿æŽæ°è²»çšã®èŠç©ãã ãŸãšã ãããžã§ã¯ã説æãšVertex AI Vector Searchã®å°å
¥èæ¯ é¢é£ã³ãŒãã¬ã³ã¡ã³ããããžã§ã¯ãã®æŠèŠ WEARã®é¢é£ã³ãŒãã¬ã³ã¡ã³ããããžã§ã¯ãã§ã¯ãã³ãŒãã£ããŒã詳现ç»é¢ã®é¢é£æ ã«è¡šç€ºããã³ãŒãã£ããŒãã®éžåºããžãã¯ãã«ãŒã«ããŒã¹ããMLã«çœ®ãæããããšãç®çãšããŸããã äžå³ã®å·ŠåŽã®ç»åã¯ãWEARã®ã³ãŒãã£ããŒã詳现ç»é¢ã®é¢é£æ ã®UIã§ãå³åŽã®ç»åãã³ãŒãã£ããŒã詳现ã®ãããã§ããé¢é£æ ã¯ã³ãŒãã£ããŒã詳现ç»é¢ã®äžéšã«è¡šç€ºããããŠãŒã¶ãŒãé²èЧããŠããã³ãŒãã£ããŒãã«é¢é£ããä»ã®ã³ãŒãã£ããŒããã³ã³ãã³ããšããŠäžèŠ§è¡šç€ºããŸãã ã«ãŒã«ããŒã¹ã®ããžãã¯ã§ã¯ãã³ãŒãã£ããŒãã«çŽã¥ãã¡ã¿ããŒã¿ã䜿ã£ãŠé¢é£æ ã«è¡šç€ºããã³ãŒãã£ããŒããéžåºããŠããŸãããã«ãŒã«ããŒã¹ã«ããé¢é£æ ã®è¡šç€ºã«ã¯æ¬¡ã®èª²é¡ããããŸããã èŠèŠçãªé¡äŒŒæ§ãèæ
®ã§ããŠããªã ãã°ã€ã³ãŠãŒã¶ãŒã®å奜ã«åºã¥ããé¢é£æ§ãèæ
®ã§ããŠããªã æ¬ãããžã§ã¯ãã§ã¯æ¢åã®ã«ãŒã«ããŒã¹ã«ããã³ãŒãã£ããŒãéžåºã§ã®äžèš2ã€ã®èª²é¡ã解決ãããããMLã䜿ã£ãææ³ã§ã³ãŒãã£ããŒããéžåºããŸããããããŠãŒã¶ãŒãèå³ãæã€ã³ãŒãã£ããŒããææ¡ããããšãç®æããé¢é£æ ã§ã®ã€ã³ãã¬ãã·ã§ã³æ°ãã¯ãªãã¯æ°ãšãã£ãKPIã®æ¹åãå³ããŸããã ã¬ã³ã¡ã³ãããžãã¯ã®æŠèŠ ã°ã©ããã¥ãŒã©ã«ãããã¯ãŒã¯ã¢ãã«ã«ããEmbeddingã®æœåº æ¬æ©èœã§ã¯ãŠãŒã¶ãŒã»ã³ãŒãã£ããŒãã®ãã¯ãã«è¡šçŸã§ããEmbeddingãæœåºãããã¯ãã«æ€çŽ¢ã«å©çšããããšã§ãŠãŒã¶ãŒããšã«ããŒãœãã©ã€ãºãããé¡äŒŒã³ãŒãã£ããŒããéžåºããŸããã Embeddingã®æœåºã«ã¯ã°ã©ããã¥ãŒã©ã«ãããã¯ãŒã¯ïŒä»¥äžGNNïŒãçšããŠããŸããGNNã¯æ·±å±€åŠç¿ã®ææ³ã§ããããããã¯ãŒã¯æ§é ãæã€ããŒã¿ã«å¯ŸããŠå¹ççã«ç¹åŸŽãåŠç¿ã»è¡šçŸããŸãããããã¯ãŒã¯ã¯åèŠçŽ ã衚ãNodeãšNodeéãçµã¶Edgeããæ§æãããŸããæ¬æ©èœã§ã¯WEARã«ããããŠãŒã¶ãŒãšã³ãŒãã£ããŒãã®é¢ä¿ããããã¯ãŒã¯ãšããŠè¡šçŸããGNNã¢ãã«ã§ã³ãŒãã£ããŒããšãŠãŒã¶ãŒã®é¢ä¿æ§ãåŠç¿ããŸãããWEARã«ããããŠãŒã¶ãŒãšã³ãŒãã£ããŒãã®é¢ä¿ã¯æ¬¡ã®ãããã¯ãŒã¯æ§é ã§è¡šçŸããŸããã Node ãŠãŒã¶ãŒ ã³ãŒãã£ããŒã Edge ãŠãŒã¶ãŒè¡å ã¯ãªã㯠ããã etc... WEARã®ããŒã¿ãçšããŠåŠç¿ããGNNã¢ãã«ã䜿ãããšã§ããããã¯ãŒã¯ã®æã€æ§é çãªç¹åŸŽãåæ ããEmbeddingãæœåºã§ããŸãããŸãGNNã¢ãã«ãšã¯å¥ã®ã¢ãã«ã§ãŠãŒã¶ãŒã»ã³ãŒãã£ããŒãã®å
éšç¹æ§ã衚ãç¹åŸŽéãæœåºããŠãããGNNã¢ãã«ã§æœåºããEmbeddingãšéçŽããããšã§æçµçãªEmbeddingãäœæããŠããŸããããããŠãŠãŒã¶ãŒã»ã³ãŒãã£ããŒãã«ã€ããŠæ§é çãã€å±æ§ãåæ ããEmbeddingãåŸãŠããŸãã ãã®ããã«åŸãããã³ãŒãã£ããŒãã®EmbeddingãçšããŠããã¯ãã«æ€çŽ¢ã®Indexãæ§ç¯ããŠããŸããæ§ç¯ããIndexã察象ã«ãã¯ãã«æ€çŽ¢ã¯ãšãªãå®è¡ããããšã§ã³ãŒãã£ããŒã矀ãååŸã§ããŸãããŠãŒã¶ãŒã®Embeddingã¯ãªã©ã³ãã³ã°ã«å©çšããŠããããªã©ã³ãã³ã°ã«ã€ããŠã¯åŸç¯ïŒ 2段éã®ãã¯ãã«æ€çŽ¢ ïŒã§èª¬æããŸãã 2段éã®ãã¯ãã«æ€çŽ¢ æ¬æ©èœã§ã¯ãŠãŒã¶ãŒããšã«ããŒãœãã©ã€ãºãããé¡äŒŒã³ãŒãã£ããŒã矀ãååŸããããã2段éã§ãã¯ãã«æ€çŽ¢ããŠããŸãã é¡äŒŒã³ãŒãã£ããŒã矀ã®ååŸ ãŠãŒã¶ãŒå奜ã«åºã¥ãããŒãœãã©ã€ãºïŒãªã©ã³ãã³ã°ïŒ 1段éç®ã®ãã¯ãã«æ€çŽ¢ã¯ãšãªå®è¡ã§ã¯ãã³ãŒãã£ããŒãEmbeddingãå
¥åãšããŠèŠèŠçã«é¡äŒŒããã³ãŒãã£ããŒã矀ãååŸããŸãã ç¶ããŠ2段éç®ã®ãã¯ãã«æ€çŽ¢ã¯ãšãªå®è¡ã§ã¯ããŠãŒã¶ãŒEmbeddingãå
¥åãšããŠ1段éç®ã§ååŸããé¡äŒŒã³ãŒãã£ããŒã矀ããªã©ã³ãã³ã°ããŸãã ãªã©ã³ãã³ã°ã«ããé¡äŒŒã³ãŒãã£ããŒã矀ã¯ãŠãŒã¶ãŒã®å奜ã«åºã¥ããŠäžŠã³æ¿ããããŸããããã«ããé¢é£æ ã«è¡šç€ºããæçµçãªã³ãŒãã£ããŒããæ±ºå®ããŸãã Vertex AI Vector Searchã®å°å
¥èæ¯ éå»ãããžã§ã¯ãã«ããããã¯ãã«æ€çŽ¢æ©èœã®å®è£
MLOpsãããã¯ã§ã¯é¢é£ã³ãŒãã¬ã³ã¡ã³ããããžã§ã¯ã以åã«ãããã¯ãã«æ€çŽ¢ãå©çšããä»ã®ã¬ã³ã¡ã³ãã·ã¹ãã ãæ§ç¯ã»éçšããŠããŸãããäžæ¹ã§éå»ã®ã·ã¹ãã å®è£
æã«ã¯ãã¯ãã«æ€çŽ¢ã®ãããŒãžããµãŒãã¹ãå
å®ããŠããããèªåã§ãã¯ãã«æ€çŽ¢æ©èœãå®è£
ã»éçšããŠããŸããã éå»ã«èªåã§ãã¯ãã«æ€çŽ¢æ©èœãå®è£
ããéçšãç¶ããŠããã·ã¹ãã ã®1ã€ãšããŠã ZOZOTOWNã®é¡äŒŒã¢ã€ãã æ€çŽ¢æ©èœ ããããŸãã é¡äŒŒã¢ã€ãã æ€çŽ¢æ©èœã¯ZOZOTOWNã®åå詳现ç»é¢ããå©çšã§ããæ©èœã§ããŠãŒã¶ãŒãé²èЧããŠããååã«é¡äŒŒããååãæ€çŽ¢ããŸããé¡äŒŒã¢ã€ãã æ€çŽ¢ã§ã¯ãååç»åããæœåºããç¹åŸŽéã«ãããã¯ãã«æ€çŽ¢Indexãæ§ç¯ããŠããŸããæ§ç¯ããIndexã«å¯ŸããŠãã¯ãã«æ€çŽ¢ã¯ãšãªãå®è¡ããé¡äŒŒååãååŸããŠããŸãã é¡äŒŒã¢ã€ãã æ€çŽ¢ã®ãã¯ãã«æ€çŽ¢ã·ã¹ãã ã¯äž»ã«æ¬¡ã®æ©èœãå«ãã§ããããããã®å®è£
ã»éçšã«ãããå·¥æ°é¢ã§ã®èª²é¡ããããŸããã ãããåŠç Indexã®æ§ç¯ Indexã®æŽæ°ïŒæ°èŠååã®è¿œå ã»åé€ïŒ ãã¯ãã«æ€çŽ¢API IndexãçšããŠãã¯ãã«æ€çŽ¢ã¯ãšãªãå®è¡ Indexã®ãªãªãŒã¹ç®¡ç ãããã§äœæãããã¯ãã«æ€çŽ¢Indexã®æŽæ°ããã¯ãã«æ€çŽ¢APIã«åæ ç¹ã«Indexã®ãªãªãŒã¹ç®¡çã¯ã·ã¹ãã çã«ãããåŠçãšãã¯ãã«æ€çŽ¢APIã«è·šãéšåã§ãããå®è£
ã ãã§ãªãéçšé¢ã®è€éããå¢ãèŠå ãšãªãèª²é¡æããããŸãããããããšãã¯ãã«æ€çŽ¢APIéã®é£æºã«ã€ããŠä»¥äžã§èª¬æããŸãã åæãšããŠMLOpsãããã¯ã§ã¯APIã®åäœã€ã³ãã©ãšããŠGoogle Kubernetes EngineïŒä»¥äžGKEïŒãå©çšããŠããŸããé¡äŒŒã¢ã€ãã æ€çŽ¢ã®ãã¯ãã«æ€çŽ¢APIãGKEäžã«PodãšããŠãããã€ãããŠããŸãã ãã¯ãã«æ€çŽ¢APIã®ã³ã³ããã¯ãã¯ãã«æ€çŽ¢Indexãã¡ã¢ãªã«æã¡ãã¯ãšãªå®è¡æã¯Indexãåç
§ããŠãã¯ãã«æ€çŽ¢ããŸããAPIèµ·åæã«Indexãã¡ã¢ãªãžèªã¿èŸŒã¿ãä¿æããŠããŸãããã®ããIndexã®æŽæ°ããã¯ãã«æ€çŽ¢APIãžåæ ããã«ã¯Podã®åèµ·åãå¿
èŠã§ãã IndexæŽæ°æã¯åæçã«ãã¯ãã«æ€çŽ¢APIãžIndexæŽæ°ãåæ ãããããIndexæŽæ°ãšãã¯ãã«æ€çŽ¢APIã®Podã®åèµ·åãäž¡æ¹ãããåŠçã§å®è¡ããŠããŸããã äžèšã®æ§æã§ã¯éçšé¢ã§å
·äœçã«æ¬¡ã®èª²é¡ããããŸããã ãã¯ãã«æ€çŽ¢APIã®å埩æ§ãæªã Indexã®ããŒã¿éã倧ããããã¯ãã«æ€çŽ¢APIã®Podèµ·åã«æéãããã Podã®å
¥ãæ¿ããé »åºŠãé«ãããšã«ããé害ãªã¹ã¯ã®å¢å å
¥ãæ¿ããæã«ã¯ãšã©ãŒçºçã®ãªã¹ã¯ãé«ã å
¥ãæ¿ããæã®é害ãªã¹ã¯ãé²ãããã«ã¯ããªã¯ãšã¹ãã®ãã¬ã€ã³ãProbeãªã©ç¹ã«æ
éã«èæ
®ããå¿
èŠããããå®è£
é£æåºŠãé«ã ããããšAPIéã®äŸåã«ããã·ã¹ãã ã®è€éæ§ã®é«ã ãã¯ãã«æ€çŽ¢APIãžã®IndexæŽæ°ã®åæ ã¿ã€ãã³ã°ãéçºè
åŽã§å¶åŸ¡ã§ããªã ãããåŠçãå®è£
ããäžã§ããã¯ãã«æ€çŽ¢APIã®å®è£
ãæ§æãèæ
®ããå¿
èŠããã ãã®ããã«èªåã§ãã¯ãã«æ€çŽ¢æ©èœãå®è£
ã»éçšããå Žåã¯å·¥æ°ã»ã·ã¹ãã ã®è€éæ§ã®é¢ã§ããã€ãã®èª²é¡ããããŸããã Vertex AI Vector Searchã®å°å
¥ã¢ãããŒã·ã§ã³ æ¬ã·ã¹ãã ã®éçºã§Vertex AI Vector Searchã®å°å
¥ã«è³ã£ãã¢ãããŒã·ã§ã³ã¯æ¬¡ã«ãªããŸãã ãããŒãžããµãŒãã¹ã§ãã ãã¯ãã«æ€çŽ¢æ©èœã®å®è£
ã»éçšã»ä¿å®ã®å·¥æ°ãæãããã äžåºŠIndexããããã€ããã°Indexã®EmbeddingæŽæ°ã¯Vertex AI Vector SearchåŽã§èªååæ ãã é«éãªã¯ãšãªå®è¡ãå¯èœ ScaNN ã¢ã«ãŽãªãºã ãå©çšããŠé«éãªè¿äŒŒæè¿åæ¢çŽ¢ãå¯èœ å°æ¥éèŠã«åããç¥èŠç²åŸ ä»åŸç€Ÿå
ã§ãã¯ãã«æ€çŽ¢ãå©çšããã¬ã³ã¡ã³ãã·ã¹ãã ã®éçºãå¢ãããšäºæ³ãããããããããŒãžããµãŒãã¹ã§ã®ãã¯ãã«æ€çŽ¢æ©èœã®éçºã»éçšäºäŸãç²åŸãã ãããŒãžããµãŒãã¹ã§ããVertex AI Vector Searchãå©çšããããšã§ããã¯ãã«æ€çŽ¢æ©èœãèªåã§å®è£
ã»éçšããå Žåã®å·¥æ°ãã·ã¹ãã ã®è€éæ§ã®èª²é¡ã解決ã§ãããšèããŸããããŸãä»åŸç€Ÿå
ã§ãã¯ãã«æ€çŽ¢ãå©çšããã¬ã³ã¡ã³ãã·ã¹ãã ã®éçºãå¢ããããšãäºæ³ãããŠãããVertex AI Vector Searchãå©çšããéçºã»æ¬çªéçšã®ç¥èŠãç²åŸãããçãããããŸãããWEARã¯ZOZOTOWNãšæ¯èŒãããšãã©ãã£ãã¯è² è·ã¯å°ãããããVertex AI Vector Searchãæ¬çªç°å¢ã§å©çšããç¥èŠãåŸãã«ã¯é©ãããããžã§ã¯ãã§ãããšå€æããŸããã Vertex AI Vector Searchã§ã¯ããã¯ãã«æ€çŽ¢ã®ã¢ã«ãŽãªãºã ã«ScaNNã¢ã«ãŽãªãºã ãå©çšããŠããè¿äŒŒæè¿åæ¢çŽ¢ãé«éã«å®è¡ã§ããŸããåè¿°ããé¡äŒŒã¢ã€ãã æ€çŽ¢ã®ãã¯ãã«æ€çŽ¢æ©èœã§ã¯ Annoy ã¢ã«ãŽãªãºã ãå©çšããŠããŸãããåŸçºã®ææ³ã§ããScaNNã¢ã«ãŽãªãºã ã®æ§èœã詊ãè¯ãæ©äŒã§ãããšèããŸããã ãã¯ãã«æ€çŽ¢æ©èœãæã€ãããŒãžããµãŒãã¹ã®æ¯èŒ Vertex AI Vector Searchãå°å
¥ããåã«ãGoogle Cloudã®ãã®ä»ã®ãã¯ãã«æ€çŽ¢ãµãŒãã¹ãšæ¯èŒããŸããã Google Cloudã§ã¯è€æ°ã®ãããŒãžãããŒã¿ããŒã¹ãµãŒãã¹ã§ãã¯ãã«æ€çŽ¢æ©èœãæäŸããŠããŸãããã®ãã¡ AlloyDB for PostgreSQL ïŒä»¥äžAlloyDBïŒã§ãScaNNã¢ã«ãŽãªãºã ãå©çšããŠããŸãããããŒãžããµãŒãã¹ã§ããç¹ãScaNNã¢ã«ãŽãªãºã ãå©çšããŠããç¹ã§ãAlloyDBã®ãã¯ãã«æ€çŽ¢æ©èœã¯Vertex AI Vector Searchãšé¡äŒŒããŠããŸããäžæ¹ã§æ¬çªã·ã¹ãã 以å€ã®ãŠãŒã¹ã±ãŒã¹ã§ã¯ããµãŒãã¹æ§ç¯ãä»ãµãŒãã¹ãšã®é£æºã®å®¹ææ§ããµãŒãã¹éžå®ã®äžã§éèŠãªèŠçŽ ãšèããŸããã MLOpsãããã¯ãé¢ãããããžã§ã¯ãã®æ©èœéçºã§ã¯ãæ¬çªã·ã¹ãã éçºåæã®æ®µéã§å®éšã»PoCãè¡ããŸãããã®æ®µéã§ã¯äž»ã«MLãšã³ãžãã¢ãããŒã¿ãµã€ãšã³ãã£ã¹ããMLã¢ãã«ã®ç²ŸåºŠãåºåã®è©äŸ¡ã»æ¹åãããŸããæ¬ã·ã¹ãã ã®æçµçãªåºåã¯ãGNNã¢ãã«ã§æœåºããEmbeddingã§ã¯ãªããã¯ãã«æ€çŽ¢ã¯ãšãªå®è¡ã®çµæã§ãããã®ããå®éšã»PoCã®æ®µéã§ãGNNã¢ãã«ã§æœåºãããŠãŒã¶ãŒã»ã³ãŒãã£ããŒãã®Embeddingãå
ã«ãã¯ãã«æ€çŽ¢Indexãæ§ç¯ããã¯ãšãªãå®è¡ããŠçµæãæ¯èŒã»ç¢ºèªå¯èœã«ããŸãã æ¬çªã·ã¹ãã ã®éçºã§ã¯ãµãŒãã¹éžå®æã«ã·ã¹ãã ã®å
ç¢æ§ãå®å®æ§ãéèŠããŸããäžæ¹ã§éçºåæã®å®éšã»PoC段éã§ã®éçºã§ã¯æ§ç¯å®¹ææ§ãšæè»æ§ãéèŠããŠããŸããå®éšã»PoC段éã§ã®å©çšãèæ
®ã«å
¥ãããšãVertex AI Vector Searchã«ã¯AlloyDBãšæ¯èŒããŠæ¬¡ã®åªäœæ§ãæããŸããã ç°å¢æ§ç¯ã容æ ããŒã¿ããŒã¹ã®æ§ç¯ãäžèŠ Google Cloud Storage ïŒä»¥äžGCSïŒã®ããŒã¿ãå
ã«Indexãäœæå¯èœ Indexã®åãæ¿ãã容æ Vertex AI Pipelines ãšåãããŠå©çšãããã AlloyDBã§ãã¯ãã«æ€çŽ¢æ©èœãå©çšããã«ã¯ã PostgreSQL ã®ããŒã¿ããŒã¹ã®æ§ç¯ãå¿
èŠã§ããããã«å¯ŸããŠVertex AI Vector Searchã§ã¯GCSã®ããŒã¿ãå
ã«Indexãäœæã§ãããããããŒã¿ããŒã¹ã®æ§ç¯ãäžèŠã§ãœãŒã¹ããŒã¿ãçšæããããç°å¢æ§ç¯ã容æã§ãããŸãVertex AI Vector Searchã§ã¯ã¢ã¯ã»ã¹å
ãšãªãIndex EndpointãšIndexãåé¢ãããŠãããIndexãåãæ¿ããŠãã¯ãã«æ€çŽ¢çµæãæ¯èŒå¯èœã§ãã ããã«Vertex AI Vector Searchã¯ç€Ÿå
ã®MLç³»ã®ãµãŒãã¹ã®ãã€ãã©ã€ã³éçºã§å€ãå©çšããŠããVertex AI PipelinesãšãåãããŠå©çšããããã§ããVertex AI Pipelines㯠Kubeflow Pipelines ã®ãã¬ãŒã ã¯ãŒã¯ã䜿çšããŠãµãŒãã¬ã¹ã§MLã¯ãŒã¯ãããŒããªãŒã±ã¹ãã¬ãŒã·ã§ã³ã§ãããµãŒãã¹ã§ããKubeflow Pipelinesã¯Kubernetesäžã§æ©æ¢°åŠç¿ã¯ãŒã¯ãããŒã管çããããã®ãªãŒãã³ãœãŒã¹ããŒã«ã§ãã Vertex AI Pipelinesã§ã¯ãã€ãã©ã€ã³ã®ææç©ã§ããArtifactãGCSã«ä¿åããŸããVertex AI Vector Searchã§ã¯GCSã®ãªããžã§ã¯ããå
ã«IndexãäœæããŸãããã®ããVertex AI Pipelinesã§ãã€ãã©ã€ã³ã®ææç©ãšããŠä¿åããGCSã®ããŒã¿ãçŽæ¥åç
§ããŠIndexãäœæã»æŽæ°ã§ããŸãããã€ãã©ã€ã³ã§ã®Indexã®äœæã»æŽæ°ã«ã€ããŠã¯åŸç¯ïŒ Indexã®äœæã»æŽæ°ãã㌠ïŒã§èª¬æããŸãã ãããã®èгç¹ãããã¯ãã«æ€çŽ¢æ©èœã®ãããŒãžããµãŒãã¹ãšããŠVertex AI Vector Searchãéžå®ããŸããã Vertex AI Vector SearchãšScaNNã¢ã«ãŽãªãºã ã®æŠèŠ Vertex AI Vector Searchã®æŠèŠ Vertex AI Vector Searchã¯ãGoogle CloudãæäŸãããããŒãžããªãã¯ãã«æ€çŽ¢ãµãŒãã¹ã§ãã以åã¯Vertex AI Matching Engineãšããåç§°ã§æäŸãããŠããŸããã Vertex AI Vector Searchã¯äž»ã«æ¬¡ã®ã³ã³ããŒãã³ãã§æ§æãããŠããŸãã Vector Search Index Vector Search Index EndpointïŒä»¥äžIndex EndpointïŒ Vector Search Indexã¯ããã¯ãã«æ€çŽ¢ã®Indexã衚çŸããã³ã³ããŒãã³ãã§ãGCSã«ä¿åãããããŒã¿ãå
ã«æ§ç¯ãããŸããIndex Endpointã¯ãæ§ç¯ããIndexã®ãããã€å
ã§ãããã¯ãšãªãåãä»ãããšã³ããã€ã³ãã§ãã åã³ã³ããŒãã³ãã®é¢ä¿æ§ã¯æ¬¡ã®å³ã§è¡šããŸãã éçºè
ã¯äºåã«Index EndpointãšIndexãäœæããŸããIndexãæå®ããIndex Endpointã«ãããã€ããããšã§ãã¯ãã«æ€çŽ¢ã¯ãšãªãå®è¡ã§ããŸããVertex AI Vector Searchã®åã³ã³ããŒãã³ãããã®ããã«æ§æããããšã§ãã¯ãã«æ€çŽ¢æ©èœãå®è£
ã§ããŸãã ScaNNã¢ã«ãŽãªãºã ã®æŠèŠ Vertex AI Vector Searchãå©çšããäžã§ãScaNNã¢ã«ãŽãªãºã ã®ç¹åŸŽã®çè§£ã¯æçšã§ãã ScaNNã¢ã«ãŽãªãºã ã¯æ¬¡ã®ç¹åŸŽãæã€è¿äŒŒæè¿åæ¢çŽ¢ã®ææ³ã§ãã ããŒã¿ãäºåã«ããŒãã£ã·ã§ãã³ã°ããæ€çŽ¢æã«ã¯å
šããŒã¿ã察象ãšããã«è¿ãããŒãã£ã·ã§ã³ã®ã¿ãæ€çŽ¢ããããšã§èšç®ã³ã¹ããåæžãã ãã¯ãã«éååææ³ã®äžçš®ã§ããProduct QuantizationïŒPQïŒã«ãã髿¬¡å
ãã¯ãã«ãå§çž®ããããã¡ã¢ãªå¹çãè¯ã ScaNNã¢ã«ãŽãªãºã ã§ã¯æ¬¡ã®3段éã®åŠçã§ãã¯ãã«æ€çŽ¢ãããŸãã ããŒãã£ã·ã§ãã³ã°ïŒããŒã¿ãäºåã«ããŒãã£ã·ã§ãã³ã°ããã¯ãšãªå®è¡æã«äžäœã®ããŒãã£ã·ã§ã³ãéžæããŠã2段éç®ã®ã¹ã³ã¢ãªã³ã°ã«é²ã ã¹ã³ã¢ãªã³ã°ïŒæ€çŽ¢å¯Ÿè±¡ã®ããŒãã£ã·ã§ã³å
ã®å
šããŒã¿ãã€ã³ããŸã§ã®è·é¢ãè¿äŒŒèšç®ããã¹ã³ã¢ãªã³ã°ãã åã¹ã³ã¢ãªã³ã°ïŒ2段éç®ã®ã¹ã³ã¢ãªã³ã°ã®çµæããäžäœk'åã®ããŒã¿ãã€ã³ããéžæãããæ£ç¢ºãªè·é¢ãåèšç®ããäžã§äžäœkåã®ããŒã¿ãã€ã³ããéžæããããšã§åã¹ã³ã¢ãªã³ã°ããïŒk'ã¯kãã倧ããªå€ãåãïŒ 1ã»2段éç®ã®åŠçã§ã¯èšç®ã³ã¹ããåæžãã3段éç®ã®åã¹ã³ã¢ãªã³ã°ã§ç²ŸåºŠãåäžãããŸããããã«ããé床ãšç²ŸåºŠã®ãã¬ãŒããªããäž¡ç«ããŠãããå€§èŠæš¡ãªããŒã¿ã»ããã«å¯ŸããŠãé«éãªè¿äŒŒæè¿åæ¢çŽ¢ãå®çŸããŠããŸãã Vertex AI Vector Searchã§æäŸãããŠããTreeAH Indexã¯ScaNNã¢ã«ãŽãªãºã ã䜿çšãããã¯ãã«Indexã®1ã€ã§ãã TreeAHã§ã¯1段éç®ã®åŠçã§ããŒã¿ãããªãŒæ§é ã«ããŒãã£ã·ã§ãã³ã°ããŸããããŒãã£ã·ã§ãã³ã°ãè¡ããªãå Žåã¯å
šããŒã¿ã察象ã«è¿äŒŒèšç®ãããã«ãŒããã©ãŒã¹æ€çŽ¢ãšãªããŸãã TreeAHã®2段éç®ã®åŠçã§ã¯ãPQã®äžã§ãç¹ã«4-bit PQãå©çšããŠé«æ¬¡å
ãã¯ãã«ãå§çž®ïŒéååïŒããŸãã4-bit PQãå©çšããããšã§SIMDã¬ãžã¹ã¿ã«åãŸããµã€ãºã®ãã¯ãã«ãæ±ãããšãã§ããŸãã SIMDãšã¯Single Instruction, Multiple Dataã®ç¥ã§1ã€ã®åœä»€ã§è€æ°ã®ããŒã¿ã䞊ååããåæã«åŠçãã圢æ
ãæããŸãã4-bit PQã§ã¯CPUã®SIMDåœä»€ãå©çšããããšã§é«éãªè¿äŒŒæè¿åæ¢çŽ¢ãå¯èœã«ãªã£ãŠããŸããTreeAHã® AH ã¯Asymmetric Hashingã®ç¥ã§ããã¯ãšãªãã¯ãã«ã¯éååãããªãçãã¯ãã«ã§ããäžæ¹ãåè£ããŒã¿ã®ãã¯ãã«ã¯äºåã«éååãããŠããããšãAsymmetric Hashingã®æä»¥ã§ãã ScaNNã¢ã«ãŽãªãºã ã®ããã¥ã¡ã³ãã§ã¯çµéšåãšããŠããŒã¿æ°ã«å¿ããŠæ¬¡ã®æ§æãæšå¥šãããŠããŸãã ããŒã¿èŠæš¡ æšå¥šææ³ 20,000 ä»¶æªæº ãã«ãŒããã©ãŒã¹ã䜿ã 100,000 ä»¶æªæº Asymmetric HashingïŒAHïŒã§ã¹ã³ã¢ãªã³ã°ãããã®åŸåã¹ã³ã¢ãªã³ã°ãã 100,000 ä»¶ä»¥äž ããŒãã£ã·ã§ãã³ã°ãè¡ããAH ã§ã¹ã³ã¢ãªã³ã°ãããã®åŸåã¹ã³ã¢ãªã³ã°ãã ScaNNã¢ã«ãŽãªãºã ãTreeAHã®çè§£ã«ããã£ãŠã¯æ¬¡ã®è³æãåèã«ãããŠããã ããŸããã Accelerating Large-Scale Inference with Anisotropic Vector Quantization ScaNN Algorithms and Configuration ãŸãè¿äŒŒæè¿åæ¢çŽ¢ã®ã¢ã«ãŽãªãºã ã4-bit PQã«ã€ããŠã¯æ¬¡ã®è³æãåèã«ãããŠããã ããŸããã è¿äŒŒæè¿åæ¢çŽ¢ã®æåç· 4-bit PQã®è§£èª¬ ã¢ã«ãŽãªãºã ã®è©³çްã«ã€ããŠã¯äžèšã®è³æããåç
§ãã ããã Indexæ§ç¯ã®å®è£
ãšãã©ã¡ãŒã¿å€ã®æ±ºå® Indexã®äœæã»æŽæ°ãã㌠é¢é£ã³ãŒãã¬ã³ã¡ã³ãã®ã·ã¹ãã ã§ã¯ãEmbeddingã®æœåºããIndexã®äœæã»æŽæ°ã»ãããã€ãŸã§ã®äžé£ã®åŠçããã€ãã©ã€ã³ã§å®è¡ããŠããŸãããã€ãã©ã€ã³ã®ã€ã³ãã©ã«ã¯Vertex AI Pipelinesãå©çšããŠããŸãã æ¬ã·ã¹ãã ã®ãã€ãã©ã€ã³ã¯æ¥æ¬¡å®è¡ããŠããããããã€æžã¿ã®IndexãæŽæ°ããŠããŸããIndexæŽæ°æã¯æ°èŠã³ãŒãã£ããŒãã®Embeddingã®è¿œå ããæ¢åã®ã³ãŒãã£ããŒãã®Embeddingãææ°ã®ãŠãŒã¶ãŒã®è¡åãåæ ããEmbeddingãžæŽæ°ããŸããããã«å逿žã¿ã®ã³ãŒãã£ããŒãã®Embeddingãåé€ããããšã§Indexãææ°åããŸãã äžå³ã¯æ¬ã·ã¹ãã ã§Vector Search Indexãäœæã»æŽæ°ã»ãããã€ãããã€ãã©ã€ã³ã®åŠçã®ãããŒã瀺ããŠããŸãã説æã®é¢ä¿ã§Vertex AI Vector Searchã«çŽæ¥é¢ä¿ããªãåŠçã¯çç¥ããŠããŸãã Vertex AI Pipelinesã§å®è¡ãããåã¿ã¹ã¯éã®ããŒã¿ã®åãæž¡ãã«ã¯Artifactãå©çšããŠããŸããArtifactã¯Kubeflowã§ããŒã¿ãä¿åããããã®ä»çµã¿ã§ããVertex AI Pipelinesã«ãããŠArtifactã¯GCSã®ãªããžã§ã¯ããšããŠä¿åãããŸãã ãã€ãã©ã€ã³ã®ã¿ã¹ã¯ã§æœåºããã³ãŒãã£ããŒãã®Embeddingã¯Indexã®ãœãŒã¹ããŒã¿åœ¢åŒã«åãããŠæåœ¢ããArtifactãšããŠGCSã«ä¿åããŸããVertex AI Vector Searchã§ã¯GCSã®ããŒã¿ãå
ã«Indexãäœæã»æŽæ°ããŸãããã€ãã©ã€ã³ã®ã¿ã¹ã¯ã§ä¿åããArtifactããã®ãŸãŸIndexã®ããŒã¿ãœãŒã¹ãšããŠåç
§ããVertex AI Vector Searchã®Indexãäœæã»æŽæ°ããŠããŸãã ã·ã¹ãã ã®æ§æã«ããã£ãŠã¯GKEã¯ã©ã¹ã¿ãšVertex AI Vector SearchãVertex AI Pipelinesã§å©çšãããªãŒãžã§ã³ãæããããšã«æ³šæããŸããã ZOZOã§ã¯DWHãšããŠBigQueryãå©çšããŠããããã€ãã©ã€ã³ã§æ±ãå€ãã®ããŒã¿ã¯USãã«ããªãŒãžã§ã³ã«ä¿åãããŠããŸãããã®ããéåžžã¯ãVertex AI Pipelinesã®å®è¡ãªãŒãžã§ã³ã¯BigQueryã®ã¯ãšãªè²»çšãæããããã«USç³»ã®ãªãŒãžã§ã³ãå©çšããããšãå€ãã§ãã äžæ¹ã§æ¬ã·ã¹ãã ã§ã¯Vertex AI Vector Searchã®åŒã³åºãå
ãšãªãAPI㯠asia-northeast1 ãªãŒãžã§ã³ã®GKEã¯ã©ã¹ã¿ã§åäœããŠããŸãã以äžã§ã¯åŒã³åºãå
ãšãªãAPIãã¢ããªã±ãŒã·ã§ã³APIãšèšèŒããŸãããã¯ãã«æ€çŽ¢ã¯ãšãªå®è¡æã®ã¬ã€ãã³ã·ãæããããIndex Endpointã¯ã¢ããªã±ãŒã·ã§ã³APIãšåã asia-northeast1 ãªãŒãžã§ã³ã«äœæããŠããŸãããŸãVertex AI Vector Searchã§ã¯Indexãããã€æã®å¶çŽãšããŠããããã€å
ã®Index EndpointãšIndexã®ãªãŒãžã§ã³ãäžèŽãããå¿
èŠããããŸãããã®å¶çŽã«ããIndexã®ãªãŒãžã§ã³ã¯Index Endpointã«åãã㊠asia-northeast1 ãªãŒãžã§ã³ã«äœæããŸããã Indexã®ãªãŒãžã§ã³å¶çŽãšããŠã¯ããœãŒã¹ããŒã¿ã«æå®ããGCSã®ãã±ãããšIndexã®ãªãŒãžã§ã³ãäžèŽãããå¿
èŠããããŸããæ¬ã·ã¹ãã ã§ã¯Vertex AI Pipelinesã®ArtifactãIndexã®ãœãŒã¹ããŒã¿ãšããŠçŽæ¥åç
§ããŠããŸãããã®ããArtifactãä¿åããGCSã®ãã±ããã asia-northeast1 ãªãŒãžã§ã³ã«äœæããŸããã ãŸãVertex AI Pipelinesã§ã¯ãArtifactã®ä¿åå
ã®GCSãã±ããã®ãªãŒãžã§ã³ãšãã€ãã©ã€ã³ãå®è¡ãããªãŒãžã§ã³ãäžèŽãããå¿
èŠããããŸãããã®ãããã€ãã©ã€ã³ã®å®è¡ã asia-northeast1 ãªãŒãžã§ã³ã§è¡ãããã«ããŸããã ãªãŒãžã§ã³å¶çŽã¯èŠèœãšããã¡ã§ãããããã·ã¹ãã èšèšæã«ç¢ºèªããŠããããšãããããããŸãã Indexæ§ç¯ã®å®è£
Indexäœæã®å®è£
ãã€ãã©ã€ã³ã§ã®Vertex AI Vector Search IndexãIndex Endpointã«é¢ããã¿ã¹ã¯å®è£
ã«ã¯Vertex AI SDK for Pythonãå©çšããŸããã Indexäœæã®å®è£
ã¯æ¬¡ã®éãã§ãã MatchingEngineIndex ã¯ã©ã¹ã® create_tree_ah_index ã¡ãœãããåŒã¶ããšã§Indexãäœæã§ããŸãã create_tree_ah_index ã¡ãœããã®åŒæ°ã®ãã¡ dimensions 以äžã®åŒæ°ã¯Indexæ§æãã©ã¡ãŒã¿ã§ããIndexæ§æãã©ã¡ãŒã¿ã«ã€ããŠã¯åŸç¯ïŒ Indexæ§æãã©ã¡ãŒã¿ã®æ±ºå® ïŒã§å¥é説æããŸãã from google.cloud.aiplatform.matching_engine import MatchingEngineIndex created_index = MatchingEngineIndex.create_tree_ah_index( project=project_id, location=location, display_name=index_display_name, index_update_method= "BATCH_UPDATE" , contents_delta_uri=gcs_uri, ####### 以äžã¯Indexæ§æãã©ã¡ãŒã¿ ######## dimensions=dimensions, approximate_neighbors_count=approximate_neighbors_count, shard_size=shard_size, leaf_node_embedding_count=leaf_node_embedding_count, distance_measure_type=distance_measure_type, feature_norm_type=feature_norm_type, ) with open (output_index_name, "w" ) as f: f.write(created_index.name) åŒæ°ã® contents_delta_uri ã«ã¯Indexäœæã®ãœãŒã¹ããŒã¿ãšããŠåç
§ããGCSã®URIïŒ gs:// ã§å§ãŸããã®ïŒãæå®ããŸãããœãŒã¹ããŒã¿ãšããŠåç
§ãããã©ã«ãã¯æ¬¡ã®æ§æã«åŸãå¿
èŠããããŸãããã©ã«ãåããã¡ã€ã«åã¯ä»»æã§ãããã¡ã€ã«ã®æ¡åŒµå㯠.csv ã .json ã .avro ã®ããããã§ããå¿
èŠããããŸãã batch_root/ feature_file_1.json feature_file_2.json åãã¡ã€ã«ã¯EmbeddingããŒã¿ãå«ã¿ãããŒã¿ã®åœ¢åŒã¯æ±ºãŸã£ãŠããŸããå¯Embeddingãã¹ããŒã¹Embeddingããã€ããªããEmbeddingã®ã©ãã䜿çšãããã«ãã£ãŠããã¡ã€ã«å
ã®ããŒã¿åœ¢åŒã決ãŸã£ãŠããããæ³šæãå¿
èŠã§ãã詳现ã¯å
¬åŒããã¥ã¡ã³ãã® å
¥åããŒã¿åŠç ãåç
§ããŠãã ãããæ¬ã·ã¹ãã ã§ã¯å¯Embeddingã䜿çšããŠãããåãã¡ã€ã«ã¯æ¬¡ã®äŸã®åœ¢åŒã§ä¿åããŠããŸããäŸã«èšèŒã®idãembeddingã®å€ã¯ãèªèº«ã®ç°å¢ã«åãããŠçœ®ãæããŠãã ããã { " id ": " 1 ", " embedding ": [ 0.1 , 0.05 , 0.3 ] } { " id ": " 2 ", " embedding ": [ 0.2 , 0.01 , 0.02 ]} Indexå
ã®åEmbeddingããšã®ã¬ã³ãŒãã以äžã§ã¯ããŒã¿ãã€ã³ããšåŒã³ãŸããããŒã¿ãã€ã³ãã®idã¯Indexå
ã§äžæã«ãªãããéè€ããªãããã«æ³šæããŠãã ãããæ¬ã·ã¹ãã ã§ã¯ã³ãŒãã£ããŒããäžæã®idãæã£ãŠããããã³ãŒãã£ããŒãã®idããã®ãŸãŸããŒã¿ãã€ã³ãã®idãšããŠå©çšããŸããã create_tree_ah_index ã¡ãœããã®æ»ãå€ã¯ MatchingEngineIndex ã¯ã©ã¹ã®ã€ã³ã¹ã¿ã³ã¹ã§ããIndexã®ã¡ã¿ããŒã¿ãå«ã¿ãŸããã€ã³ã¹ã¿ã³ã¹ã® name ããããã£ãåç
§ããããšã§å®å
šä¿®é£ŸãããIndexã®idãååŸã§ããŸããIndexã®idã¯ãããã€æã«å¿
èŠã§ããããã created_index.name ã®å€ãæåŸã«ã¿ã¹ã¯ã®åºåãã©ã¡ãŒã¿ãšããŠæžã蟌ãã§ããŸãã Index Endpointã®äœæãšIndexã®ãããã€ äœæããIndexããããã€ããåã«ãããã€å
ãšãªãIndex Endpointã®äœæãå¿
èŠã§ãã Index Endpointã¯ãã€ãã©ã€ã³ãšã¯å¥ã§ Terraform ã«ããäœæããŸãããTerraformã¯IaCããŒã«ã®1ã€ã§ã€ã³ãã©ã®æ§æã宣èšçã«èšè¿°ã§ãããœãŒã¹ã³ãŒããšããŠç®¡çã§ããŸããMLOpsãããã¯ã§ã¯ååãšããŠGoogle Cloudã®ã€ã³ãã©æ§æãTerraformã§å®çŸ©ãã管çããŠããŸãã æ¬¡ã«ç€ºãã®ã¯Index EndpointãäœæããTerraformå®è£
ã§ããTerraform Providerã®èšå®ã¯çç¥ããŠããŸããã google Provider ãå©çšããŠäœæå¯èœã§ãã resource "google_vertex_ai_index_endpoint" "user_to_snap_index_endpoint" { display_name = "user_to_snap_index_endpoint" region = "asia-northeast1" network = "projects/1234/global/networks/dummy-vpc-network" public_endpoint_enabled = false } Vertex AI Vector Searchãžã®ãªã¯ãšã¹ãã¯ã»ãã¥ãªãã£ãã¬ã€ãã³ã·ã®èгç¹ããå
éšéä¿¡ã®ã¿ã«å¶éããŠããŸããIndex Endpointã®äœææã«VPCãããã¯ãŒã¯ãæå®ããããšã§ããµãŒãã¹ãããã¥ãŒãµãŒåŽã®ãããã¯ãŒã¯ãšãŠãŒã¶ãŒãäœæããVPCãããã¯ãŒã¯éã§VPC Peeringãæ§æã§ããŸããVPC Peeringãæ§æããããšã§2ã€ã®VPCãããã¯ãŒã¯ãæ¥ç¶ã§ããåãããã¯ãŒã¯å
ã®ãªãœãŒã¹ãçžäºã«å
éšéä¿¡ã§ããŸããVertex AI Vector Searchã®VPC Peeringæ§æã®è©³çްã«ã€ããŠã¯å
¬åŒããã¥ã¡ã³ãã® VPC ãããã¯ãŒã¯ ãã¢ãªã³ã°æ¥ç¶ãèšå®ãã ãåãããŠãåç
§ãã ããã ãŸãVertex AI Vector Searchã§ã¯Google Cloudã®ãããã¯ãŒãã³ã°æ©èœã®1ã€ã§ãã Private Service Connect ããµããŒãããŠããŸããPrivate Service Connectãå©çšããå Žåã¯å
¬åŒããã¥ã¡ã³ãã® Private Service Connect ã§ãã¯ãã«æ€çŽ¢ãèšå®ãã ããåç
§ãã ããã æ¬¡ã«ç€ºãã®ã¯ãã€ãã©ã€ã³ã§Indexããããã€ããå®è£
äŸã§ããTerraformã§äºåäœæããIndex Endpointã®idãæå®ã㊠MatchingEngineIndexEndpoint ã¯ã©ã¹ã®ã€ã³ã¹ã¿ã³ã¹ãäœæããŸããäœæããã€ã³ã¹ã¿ã³ã¹ã® deploy_index ã¡ãœãããåŒã¶ããšã§Indexããããã€ã§ããŸãã import time from google.cloud.aiplatform.matching_engine import MatchingEngineIndex, MatchingEngineIndexEndpoint index_endpoint_name = ( "projects/<YOUR_PROJECT_NAME>/locations/asia-northeast1/indexEndpoints/<YOUR_INDEX_ENDPOINT_ID>" ) index_name = "<YOUR_INDEX_ID>" machine_type = "e2-standard-16" min_replica_count = 2 max_replica_count = 5 index_endpoint = MatchingEngineIndexEndpoint( index_endpoint_name= str (index_endpoint_name), ) index = MatchingEngineIndex( index_name= str (index_name), ) current_timestamp_sec = int (time.time()) deployed_index_id = f "{index.display_name}_{index.name}_{current_timestamp_sec}" deployed_index_endpoint = index_endpoint.deploy_index( index=index, deployed_index_id=deployed_index_id, display_name=index_endpoint_display_name, deploy_request_timeout=timeout, ####以äžã§ã¯ãã·ã³ã¿ã€ããã¬ããªã«æ°ãæå®##### machine_type=machine_type, min_replica_count=min_replica_count, max_replica_count=max_replica_count, ) with open (output_index_endpoint_name, "w" ) as f: f.write( str (deployed_index_endpoint.name)) Indexã®ãããã€åŸã¯Indexã®idãšã¯å¥ã«ãããã€æžã¿Indexã®idãä»äžãããŸãããããã€æžã¿Indexã¯Google Cloudã®ã³ã³ãœãŒã«äžã§ç¢ºèªã§ããŸããæ¬¡ã®ç»åã«å«ãŸãã ID ã®å€ããããã€æžã¿Indexã®idã§ãã æ¬ã·ã¹ãã ã§ã¯ããããã€å¯Ÿè±¡ã®Indexã®idãšäœææã®ã¿ã€ã ã¹ã¿ã³ããçµã¿åãããŠãããã€æžã¿Indexã®idãçæããŠããŸããã¿ã€ã ã¹ã¿ã³ããå«ããã®ã¯ãåãIndexãå床ãããã€ããå Žåã«ãããã€æžã¿Indexã®idãéè€ããªãããã«ããæå³ã§ãã Indexã®ãããã€æã«ã¯ãã·ã³ã¿ã€ããããŒãã®æå°ã»æå€§ã¬ããªã«æ°ãæå®ã§ããŸããå©çšå¯èœãªãã·ã³ã¿ã€ãã¯åŸè¿°ããIndexæ§æãã©ã¡ãŒã¿ã®1ã€ã§ããã·ã£ãŒããµã€ãºããšã«éå®ãããŸãããã·ã³ã¿ã€ãã¯Indexã®ãããã€åŸã«å€æŽã§ããªããããã·ã£ãŒããµã€ãºãèŠç©ãã£ãäžã§ãã·ã³ã¿ã€ãã®éžå®ãå¿
èŠã§ããIndexãµã€ãºã®èŠç©ãããšã·ã£ãŒããµã€ãºã®æ±ºå®ã«ã€ããŠã¯åŸç¯ïŒ Indexãµã€ãºã®èŠç©ãããšã·ã£ãŒããµã€ãºã®æ±ºå® ïŒã§èª¬æããŸãã ãŸããã·ã³ã¿ã€ãã«ã¯ãããããããã©ãŒãã³ã¹ãšè²»çšã®ãã¬ãŒããªãããããŸããããããèæ
®ããäžã§ãã·ã¹ãã ã®èŠä»¶ã«åãããŠãã·ã³ã¿ã€ããéžå®ããããšãããããããŸãããã·ã³ã¿ã€ãããšã®ããã©ãŒãã³ã¹ã«ã€ããŠã¯å
¬åŒããã¥ã¡ã³ãã® ããã©ãŒãã³ã¹ã«åœ±é¿ãããããã€èšå® ããåç
§ãã ããã ã¬ããªã«æ°ã¯Indexã®ã·ã£ãŒãããšã®ããŒãå°æ°ãæå®ãããã©ã¡ãŒã¿ã§ããã€ãŸã ã·ã£ãŒãæ°xã¬ããªã«æ° ãããŒãå°æ°ã®åèšã«ãªããŸããã¬ããªã«æ°ã¯æå°ã»æå€§ã¬ããªã«æ°ã®ç¯å²ã§ãªãŒãã¹ã±ãŒãªã³ã°ããŸãããŸãæå°ã¬ããªã«æ°ã2æªæºã«èšå®ããå Žåã¯Vertex AIãµãŒãã¹ã¬ãã«å¥çŽã®å¯Ÿè±¡å€ãšãªãããæ³šæããŠãã ããã Indexããããã€ãããšã¢ã¯ã»ã¹å
ãšãªãgRPCã¢ãã¬ã¹ãå²ãåœãŠãããŸãããã¯ãã«æ€çŽ¢ã¯ãšãªå®è¡æã¯ãã®gRPCã¢ãã¬ã¹ãæå®ããŠãªã¯ãšã¹ããéä¿¡ããŸãããªã¯ãšã¹ãå
ãšãªãã¢ãã¬ã¹ã®ç¯å²ã¯ deploy_index ã¡ãœããã® reserved_ip_ranges åŒæ°ã§æå®å¯èœã§ããæå®ããªãå Žåã¯VPCå
ã§VPC Peeringçšã«å²ãåœãŠãŠããã¢ãã¬ã¹ç¯å²ããå²ãåœãŠãããŸããããŒãã®ã¬ããªã«æ°ã倿Žããå Žåããªã¯ãšã¹ãå
ã®gRPCã¢ãã¬ã¹ã¯äžå€ã§ãã Indexã®ãããã€ã«é¢ãã詳现ã¯å
¬åŒããã¥ã¡ã³ãã® VPC ãããã¯ãŒã¯ã«ã€ã³ããã¯ã¹ ãšã³ããã€ã³ãããããã€ããŠç®¡çãã ãåç
§ããŠãã ããããŸã deploy_index ã¡ãœããã®è©³çްã«ã€ããŠã¯ Python SDKã®ããã¥ã¡ã³ã ããåç
§ãã ããã Indexæ§æãã©ã¡ãŒã¿ã®æ±ºå® Indexã®äœææã«ã¯Indexæ§æãã©ã¡ãŒã¿ãæå®ããŸãããã©ã¡ãŒã¿ã®å€ãã¯Indexäœææã«ç¢ºå®ããŸãããã¯ãšãªå®è¡æã«èšå®å€ãäžæžãããããšã§åçã«èª¿æŽã§ãããã®ããããŸããããã§ã¯ãIndexäœææã«æå®ãããã©ã¡ãŒã¿ãšã¯ãšãªå®è¡æã«åçã«èª¿æŽããŠãããã©ã¡ãŒã¿ã«ã€ããŠãæ¬ã·ã¹ãã ãæ§æããéã®å€ã®æ±ºãæ¹ã説æããŸãã ãã¯ãã«æ€çŽ¢ã§å©çšããã¢ã«ãŽãªãºã ã¯TreeAHãšBruteForceã®2çš®é¡ããéžæã§ããŸããåç¯ïŒ ScaNNã¢ã«ãŽãªãºã ã®æŠèŠ ïŒã§è¿°ã¹ãScaNNã¢ã«ãŽãªãºã ã®ããã¥ã¡ã³ãã®çµéšåã§ã¯100,000件以äžã®ããŒã¿èŠæš¡ã§ã¯TreeAHã®å©çšãæšå¥šãããŠããŸãããæ¬ã·ã¹ãã ã®Indexã«å«ãŸããEmbeddingç·æ°ã¯çŽ1,400äžä»¶ã§ãããããTreeAHãå©çšããããšã«ããŸãããæ¬èšäºã§ã®å
容ã¯ç¹ã«æãããªãéãã¯TreeAHã¢ã«ãŽãªãºã ã®å©çšãåæãšããŠããããšããçæãã ããã Indexæ§æãã©ã¡ãŒã¿ã«ã€ããŠã¯å
¬åŒããã¥ã¡ã³ãã® ã€ã³ããã¯ã¹æ§æãã©ã¡ãŒã¿ ãæ¬ç¯ã®å
容ãšåãããŠãåç
§ãã ããã ã¯ãšãªå®è¡æã«äžæžãããŠãããã©ã¡ãŒã¿ æ¬ã·ã¹ãã ã§ã¯ã¯ãšãªå®è¡æã次ã®Indexæ§æãã©ã¡ãŒã¿ãäžæžãããããšã§åçã«èª¿æŽããŠããŸãã ãã£ãŒã«ã 説æ approximateNeighborsCount åã¹ã³ã¢ãªã³ã°å®è¡åã®è¿äŒŒæ¢çŽ¢ã«ãã£ãŠããã©ã«ãã§æ¢çŽ¢ãããè¿åæ° fractionLeafNodesToSearch ã¯ãšãªãæ€çŽ¢ããããªãŒãããŒãã®ããã©ã«ãã®å²å ãããã®å€ã®å€§å°ã¯åçŸçãšã¬ã€ãã³ã·éã®ãã¬ãŒããªãã«ãªããŸããå€ã倧ããããã»ã©æ€çŽ¢ç¯å²ãåºããããåçŸçã¯åäžããäžæ¹ã§ã¬ã€ãã³ã·ã¯å¢å ããŸãã ãŸãIndexã®æ§æãã©ã¡ãŒã¿ã§ã¯ãããŸãããããã¯ãã«æ€çŽ¢ã¯ãšãªå®è¡æã«ã¯ã¯ãšãªã§è¿ãçµæã®æ°ã§ãã setNeighborCount ãæå®ã§ããŸãããã®å€ã倧ãããªãã»ã©ã¬ã€ãã³ã·ã¯æªåããŸãã åç¯ïŒ 2段éã®ãã¯ãã«æ€çŽ¢ ïŒã®èª¬æã®éããæ¬ã·ã¹ãã ã¯1段éç®ã®ãã¯ãã«æ€çŽ¢ã§ååŸããé¡äŒŒã³ãŒãã£ããŒã矀ã2段éç®ã®ãã¯ãã«æ€çŽ¢ã§ãªã©ã³ãã³ã°ãæçµçãªçµæãè¿ããŸããããã§æçµçã«è¿ãã¹ãä»¶æ°ã®äžéã60ä»¶ãšããèŠä»¶ã¯æ±ºãŸã£ãŠããããããã®å€ãå
ã«ãã®ä»ã®ãã©ã¡ãŒã¿å€ã決å®ããŸããã ãŸã1段éç®ã®ãã¯ãã«æ€çŽ¢ã«ããããã©ã¡ãŒã¿å€ã®æ±ºå®ã«ã€ããŠèª¬æããŸãã1段éç®ã¯ãç²ãéããå€ãã«æŸãããæèããŠæ±ºå®ããŸããã 2段éç®ã«1段éç®ã§ååŸããçµæããªã©ã³ãã³ã°ããããšãéã¿ããš1段éç®ã§ã¯ãªãã¹ãå€ãã®çµæãè¿ããããšèããŸããããã®ãã2段éç®ã§è¿ã60ä»¶ãããåå倧ããå€ã1段éç®ã§è¿ãçµææ°ã®äžéãšãã setNeighborCount ã®å€ã決å®ããŸããã å
¬åŒããã¥ã¡ã³ãïŒ ããã©ãŒãã³ã¹ã«åœ±é¿ããã¯ãšãªæéã®èšå® ïŒã§ã¯ setNeighborCount ã®å€ãå°ããå Žåã approximateNeighborsCount ã®å€ã10åã«ããããšãæšå¥šãããŠããŸãã approximateNeighborsCount ã®å€ã¯ãã®æšå¥šã«åŸã£ãŠæ±ºå®ããŸããã æ¬ã·ã¹ãã ã®ããžãã¹èŠä»¶ã§ã¯ããã»ã©é«ãåçŸçã¯èŠæ±ãããŠããŸããã§ããããã®ããé¡äŒŒã³ãŒãã£ããŒã矀ã®ååŸã«ãããŠã¯ãåçŸçãããã¬ã€ãã³ã·ãåªå
ã㊠fractionLeafNodesToSearch ã¯æ¯èŒçå°ããå€ã§ããããã©ã«ãå€ããã®ãŸãŸæ¡çšããŸããã äžèšãèžãŸããŠ1段éç®ã®ã¯ãšãªå®è¡æã«äžæžããããã©ã¡ãŒã¿ãšå€ã¯æ¬¡ã®éãã§ããåèãŸã§ã«æ¬ã·ã¹ãã ã®ãã¯ãã«æ€çŽ¢Indexã®Embeddingç·æ°ã¯çŽ1,400äžä»¶ã§ãã ãã£ãŒã«ã å€ approximateNeighborsCount 10,000 fractionLeafNodesToSearch 0.05 setNeighborCount 1,000 ç¶ããŠ2段éç®ã®ãã¯ãã«æ€çŽ¢ã«ããããã©ã¡ãŒã¿å€ã®æ±ºå®ã«ã€ããŠèª¬æããŸãã2段éç®ã¯ã粟床ã®éèŠããæèããŠæ±ºå®ããŸããã 2段éç®ã®ãã¯ãã«æ€çŽ¢ã§ã¯1段éç®ã®ãã¯ãã«æ€çŽ¢çµæã«çµã蟌ãã§ãã¯ãã«æ€çŽ¢ããããæ€çŽ¢å¯Ÿè±¡ã¯1,000ä»¶ã«çµãããŠããŸãã approximateNeighborsCount ã fractionLeafNodesToSearch ã®å€ã倧ãããããšã¬ã€ãã³ã·ã®å¢å ãæžå¿µãããŸããäžæ¹ã§ããã§ã¯æ€çŽ¢å¯Ÿè±¡æ°ãçµãããŠããããããã§ã®ã¬ã€ãã³ã·æªåã®æžå¿µã¯å°ãªããšèããŸããããã®ãã2段éç®ã§ã¯åçŸçãéèŠã㊠approximateNeighborsCount ã fractionLeafNodesToSearch ãšãã«åå倧ããªå€ãæå®ããŸãããå
·äœçã«æå®ããå€ã¯æ¬¡ã®éãã§ãã ãã£ãŒã«ã å€ approximateNeighborsCount 1,000 fractionLeafNodesToSearch 0.99 setNeighborCount 60 ãã®ããã«æ±ºããå€ã§ãã¯ãã«æ€çŽ¢ã¯ãšãªãå®è¡ããã¬ã€ãã³ã·ã®èгç¹ã§ç®æšãæºãããããšã確èªããŸããããŸããã¯ãã«æ€çŽ¢çµæã«ã€ããŠå®æ§çã«æºè¶³ã§ããçµæãåŸãããããšã確èªããŸãããéçºã¹ã±ãžã¥ãŒã«ãå³ããã£ãããšããããå³å¯ãªæ¯èŒèª¿æŽãªã©ã®ãã¥ãŒãã³ã°ã¯è¡ããäžèšã®å€ã§éçšãéå§ããŸããã ã¯ãšãªå®è¡æã«æå®å¯èœãªãã©ã¡ãŒã¿ãšããã©ãŒãã³ã¹åœ±é¿ã«ã€ããŠã¯ãå
¬åŒããã¥ã¡ã³ãã® ããã©ãŒãã³ã¹ã«åœ±é¿ããã¯ãšãªæéã®èšå® ã§èª¬æãããŠããããåãããŠãåç
§ãã ããã Indexæ§ææã«æå®ããŠãããã©ã¡ãŒã¿ Indexã®æ§æãã©ã¡ãŒã¿ã®ãã¡ãåç¯ïŒ ã¯ãšãªå®è¡æã«äžæžãããŠãããã©ã¡ãŒã¿ ïŒã§èª¬æãããã©ã¡ãŒã¿ä»¥å€ã¯IndexäœæåŸã®å€æŽã¯ã§ããŸããããããã®å€ã倿Žããå Žåã¯Indexã®åäœæãå¿
èŠã§ãã æ¬¡ã®ãã©ã¡ãŒã¿ã¯Indexäœææã«ç¢ºå®ãããã©ã¡ãŒã¿ã§ãããããã®ãã©ã¡ãŒã¿ã«ã€ããŠæ¬ã·ã¹ãã ã§ã®å€ã®æ±ºãæ¹ã説æããŸãã ãã£ãŒã«ã 説æ dimensions å
¥åãã¯ãã«ã®æ¬¡å
æ° ShardSize åã·ã£ãŒãã®ãµã€ãº leafNodeEmbeddingCount åãªãŒãããŒãã«å¯ŸããEmbeddingã®æ° distanceMeasureType æè¿åæ¢çŽ¢ã§äœ¿çšãããè·é¢å°ºåºŠ featureNormType åãã¯ãã«ã«å¯ŸããŠå®è¡ãããæ£èŠåã®ã¿ã€ã dimensions ã¯å
¥åãã¯ãã«ã®æ¬¡å
æ°ãæå®ãããã©ã¡ãŒã¿ã§ããEmbeddingã®æ¬¡å
æ°ãå
ã«å€ã決å®ããŸããæ¬ã·ã¹ãã ã§ã¯GNNã¢ãã«ã§æœåºããEmbeddingã®æ¬¡å
æ°ã¯256次å
ã§ãããã256ãæå®ããŸããã ShardSize ã¯æ¬¡ã®3çš®é¡ããéžæã§ããŸãã çš®é¡ èª¬æ SHARD_SIZE_SMALL ã·ã£ãŒãããã 2 GiB SHARD_SIZE_MEDIUM ã·ã£ãŒãããã 20 GiB SHARD_SIZE_LARGE ã·ã£ãŒãããã 50 GiB ShardSize ã¯åããŒãäžã®ããŒã¿éãå¶åŸ¡ãããã©ã¡ãŒã¿ã§ããæ¬ã·ã¹ãã ã§ã¯ SHARD_SIZE_MEDIUM ãéžæããŸãããæ¬ã·ã¹ãã ã§ã®ã·ã£ãŒããµã€ãºã®æ±ºå®æ¹æ³ã«ã€ããŠã¯åŸç¯ïŒ Indexãµã€ãºã®èŠç©ãããšã·ã£ãŒããµã€ãºã®æ±ºå® ïŒã§å¥é詳现ã«èª¬æããŸãã leafNodeEmbeddingCount ã¯åãªãŒãããŒãã®Embeddingã®æ°ãæå®ãããã©ã¡ãŒã¿ã§ããåç¯ïŒ ScaNNã¢ã«ãŽãªãºã ã®æŠèŠ ïŒã§èª¬æããããŒãã£ã·ã§ãã³ã°æ°ã«é¢ãããŸããå€ã®å€§å°ã¯åçŸçãšã¬ã€ãã³ã·éã®ãã¬ãŒããªãã«ãªããŸããå€ã倧ããããã»ã©æ€çŽ¢å¯Ÿè±¡ãæžãã¬ã€ãã³ã·ã¯çããªãäžæ¹ã§åçŸçã¯äœäžããŸããå
¬åŒããã¥ã¡ã³ãïŒ ããã©ãŒãã³ã¹ã«åœ±é¿ããæ§æãã©ã¡ãŒã¿ ïŒã§ã¯ãã»ãšãã©ã®ãŠãŒã¹ã±ãŒã¹ã§å€ã15,000ãè¶
ããªãéãã¬ã€ãã³ã·ã¯çããªããšèª¬æãããŠããŸããæ¬ã·ã¹ãã ã§ã¯ã¬ã€ãã³ã·ãåªå
ããŠ15,000ãæå®ããŸããã leafNodeEmbeddingCount ã«ã€ããŠã¯ããã©ã¡ãŒã¿å€ã決å®ããåã«ãããŒã§100ãšããŠãããšããã¯ãšãªå®è¡ã«æ°ç§ãããåé¡ããããŸããã15,000ã«å€æŽãããšããã¯ãšãªå®è¡ã®æéã¯æ°10ããªç§ã«æ¹åããŸãããIndexå
ã®Embeddingæ°ãå€ãå Žåã¯æ³šæã㊠leafNodeEmbeddingCount ã®å€ã決å®ããããšãããããããŸãã distanceMeasureType ãš featureNormType ã¯å
¬åŒããã¥ã¡ã³ãïŒ ã€ã³ããã¯ã¹æ§æãã©ã¡ãŒã¿ ïŒã§ã®æšå¥šå€ã«åŸã DOT_PRODUCT_DISTANCE + UNIT_L2_NORM ãæå®ããŸããã以äžããŸãšãããšæ¬ã·ã¹ãã ã§ã®Indexæ§ææã«æå®ããŠãããã©ã¡ãŒã¿å€ã¯æ¬¡ã®éãã§ãã ãã£ãŒã«ã å€ dimensions 256 ShardSize SHARD_SIZE_MEDIUM leafNodeEmbeddingCount 15,000 distanceMeasureType DOT_PRODUCT_DISTANCE featureNormType UNIT_L2_NORM Indexãµã€ãºã®èŠç©ãããšã·ã£ãŒããµã€ãºã®æ±ºå® Indexæ§æãã©ã¡ãŒã¿ã®ãã¡ã ShardSize ã¯åããŒãäžã®ããŒã¿éãå¶åŸ¡ãããã©ã¡ãŒã¿ã§ãããæå®ãã ShardSize ããšã«ãµããŒããããããŒã¿ãµã€ãºã決ãŸã£ãŠããŸããã·ã£ãŒãæ°ã¯æå®ãã ShardSize ãšIndexã®ãµã€ãºã«å¿ããŠèªåã§æ±ºå®ãããŸãã ShardSize ã¯IndexäœæåŸã«å€æŽã§ããªããããäºåã«å€ã®èŠç©ãããå¿
èŠã§ãããŸãVertex AI Vector Searchã®åèšããŒãå°æ°ã¯ ã·ã£ãŒãæ°xã¬ããªã«æ° ã§æ±ºãŸãããã ShardSize ã®èŠç©ããã¯è²»çšã«ã倧ãã圱é¿ããŸãã æé©ãª ShardSize ãéžæããããã«ãäœæããIndexãµã€ãºã®ç¢ºèªãå¿
èŠã§ããäžæ¹ã§äœæããIndexã®ãµã€ãºãçŽæ¥çã«ç¢ºèªããæ¹æ³ã¯ãªãããæŠç®ããŸãã Indexã®ãµã€ãºã¯æ¬¡åŒã§æŠç®ããŸãããããã§åŒã«å«ãŸããRestrictsã¯EmbeddingããŒã¿ã«å±æ§æ
å ±ãä»äžããããã®ãã£ãŒã«ãã§ããããã¯ãã«æ€çŽ¢æã®ãã£ã«ã¿ãªã³ã°ã«å©çšã§ããŸããæ¬ã·ã¹ãã ã§ã¯2段éç®ã®ãã¯ãã«æ€çŽ¢ã¯ãšãªå®è¡æãæ€çŽ¢å¯Ÿè±¡ã1段éç®ã®ãã¯ãã«æ€çŽ¢çµæãžçµã蟌ãããã«ãã£ã«ã¿ãªã³ã°ãå©çšããŠããŸããRestrictsã«ã€ããŠã¯åŸç¯ïŒ ãã£ã«ã¿ãªã³ã°ã®å®è£
ïŒã§è©³ãã説æããŸãã Indexãµã€ãºïŒBytesïŒ = Embeddingæ° Ã ïŒ»(次å
æ° Ã 1次å
ãããã®ãµã€ãº) + Restrictsã®ãµã€ãºïŒœ ScaNNã¢ã«ãŽãªãºã ã§ã¯Embeddingãå§çž®ããŸããã ShardSize ã®èŠç©ããã«ããã£ãŠã¯å§çž®åŸã®ãµã€ãºã§ã¯ãªããå§çž®åã®ãµã€ãºãå
ã«èšç®ããå¿
èŠãããããšã«æ³šæããŠãã ããã æ¬ã·ã¹ãã ã§ã¯Embeddingæ°ã¯çŽ1,400äžä»¶ã次å
æ°ã¯256次å
ã1次å
ãããã®ãµã€ãºã¯4 BytesïŒfloat32ïŒã§ããEmbeddingããŒã¿ã®ãµã€ãºã¯ 14,000,000 à (256 à 4) ã§èšç®ã§ããçŽ13.35 GiBãšãªããŸããRestrictsã®ãµã€ãºãIndexã®ãµã€ãºã«å¯ŸããŠçŽ15ïŒ
ãšèŠç©ãããšIndexã®å
šäœãµã€ãºã¯çŽ15.35 GiBãšæŠç®ã§ããŸãããæŠç®ã®ããRestrictsã®ãµã€ãºã¯å€§ããã«èŠç©ãããŸããã Indexã®å
šäœãµã€ãºã15.35 GiBã§ããå Žåã SHARD_SIZE_SMALL ãš SHARD_SIZE_MEDIUM ã®2çš®é¡ã®ã·ã£ãŒããµã€ãºã§ã¯å¿
èŠãªã·ã£ãŒãæ°ã¯æ¬¡ã®ããã«èŠç©ããã§ããŸãã çš®é¡ ãµããŒãããããµã€ãº ã·ã£ãŒãæ°ã®èŠç©ãã SHARD_SIZE_SMALL ã·ã£ãŒãããã 2 GiB 8~9 SHARD_SIZE_MEDIUM ã·ã£ãŒãããã 20 GiB 1 äžèšã®Embeddingæ°ã§ã·ã£ãŒããµã€ãºã« SHARD_SIZE_SMALL ãéžæãããšå®éã®ã·ã£ãŒãæ°ã¯8ãšãªããŸãããåè¿°ã®éãIndexã®ãµã€ãºãçŽæ¥ç¢ºèªããæ¹æ³ã¯ãªããããå®éã«æ§æãããŸã§ã¯ã·ã£ãŒãæ°ã®èŠç©ããã®åŠ¥åœæ§ãå³å¯ã«ç¢ºèªã§ããŸãããäžæ¹ã§èŠç©ãããšå®éã®å€ã®èª€å·®ã2 GiBçšåºŠã§äžèŽããããšãããæŠç®å€ã®åŠ¥åœæ§ã¯ããçšåºŠæ
ä¿ãããŠãããšå€æã§ããŸãããã®ããå®éã«Indexãäœæããã·ã£ãŒããµã€ãºã倿ŽããŠç¢ºèªãããšãã·ã£ãŒããµã€ãºã決å®ã§ããŸãã ã·ã£ãŒããµã€ãºãšã·ã£ãŒãæ°ã®å€§å°ã¯åçŸçãšã¬ã€ãã³ã·éã®ãã¬ãŒããªãã«ãªããŸããã·ã£ãŒããµã€ãºãå°ããã»ã©ã·ã£ãŒãæ°ã¯å¢ãåçŸçãåäžããŸããã¬ã€ãã³ã·ãæªåãããŸããã·ã£ãŒããµã€ãºã倧ããã»ã©ã·ã£ãŒãæ°ã¯æžãã¬ã€ãã³ã·ãæ¹åããŸããåçŸçãäœäžãããŸããæ¬ã·ã¹ãã ã§ã¯ã¬ã€ãã³ã·ãåªå
ã㊠SHARD_SIZE_MEDIUM ãéžæããŸããã Indexã®ãããã€ã«äœ¿çšã§ãããã·ã³ã¿ã€ã㯠ShardSize ã«ãã£ãŠç°ãªããŸããäŸãã° e2-standard-2 ãã·ã³ã¿ã€ã㯠SHARD_SIZE_SMALL ã®ã·ã£ãŒããµã€ãºã§ã®ã¿å©çšå¯èœã§ãã SHARD_SIZE_MEDIUM ãéžæããå Žåã«å©çšã§ããæãå®äŸ¡ãªãã·ã³ã¿ã€ã㯠e2-standard-16 ã«ãªããŸãã ãã·ã³ã¿ã€ã SHARD_SIZE_SMALL SHARD_SIZE_MEDIUM SHARD_SIZE_LARGE n1-standard-16 ⯠⯠à n1-standard-32 ⯠⯠⯠e2-standard-2 ⯠à à e2-standard-16 ⯠⯠à e2-highmem-16 ⯠⯠⯠n2d-standard-32 ⯠⯠⯠æ¬ã·ã¹ãã ã®Indexãµã€ãºãå
ã«ã SHARD_SIZE_SMALL ãéžæããå Žåãš SHARD_SIZE_MEDIUM ãéžæããå Žåã®è²»çšãæ¯èŒãããšæ¬¡ã®éãã§ããèšç®ã«åœãã£ãŠã¯ããããã® ShardSize ããšã«äœ¿çšã§ããæãå®äŸ¡ãªãã·ã³ã¿ã€ãã§æ¯èŒããŠããŸãããªãŒãžã§ã³ã¯ asia-northeast1 ãæ³å®ããŠããããã·ã³ã¿ã€ãã®è²»çšã¯ å
¬åŒããã¥ã¡ã³ã ã®2025幎7ææç¹ã®äŸ¡æ Œãå
ã«èšç®ããŠããŸããã¬ããªã«æ°ã¯æ¬ã·ã¹ãã ã§ã®æå°ã¬ããªã«æ°ã§ãã2ãæ³å®ããŠããŸãã çš®é¡ ã·ã£ãŒãæ° ãã·ã³ã¿ã€ã 1ããŒãæéãããã®è²»çšïŒUSDïŒ ããŒãå°æ°ïŒã·ã£ãŒãæ° Ã ã¬ããªã«æ°ïŒ æé¡è²»çšïŒUSDïŒ SHARD_SIZE_SMALL 8 e2-standard-2 0.12 16 1382.4 SHARD_SIZE_MEDIUM 1 e2-standard-16 0.963 2 1386.72 èšç®äŸã§ã¯ã SHARD_SIZE_SMALL ãéžæããŠå°ããªãã·ã³ãå©çšããã®ãš SHARD_SIZE_MEDIUM ãéžæãããã倧ããªãã·ã³ãå©çšããã®ã§ã¯è²»çšã«å€§ããªå·®ã¯ãªãããšãããããŸãã SHARD_SIZE_SMALL ã®å ŽåããªãªãŒã¹åŸã®æ©ã段éã§Embeddingæ°ãå¢ããŠã·ã£ãŒãæ°ã9ã«ãªãå¯èœæ§ããããŸããã·ã£ãŒãæ°9ã ãš SHARD_SIZE_SMALL ã®å Žå㯠SHARD_SIZE_MEDIUM ãšæ¯èŒããŠè¥å¹²è²»çšãé«ããªããŸãããã®ããã«è²»çšèгç¹ã§ã SHARD_SIZE_MEDIUM ãéžæããããšã劥åœã§ãããšå€æããŸããã ShardSize ã®éžææã¯ä»åŸã®Embeddingæ°ã®å¢å ãèæ
®ã«å
¥ãããªã©ãã·ã¹ãã èŠä»¶ã«å¿ããŠå€ãéžå®ããããšãããããããŸãã ã·ã£ãŒããµã€ãºã«ã€ããŠã¯å
¬åŒããã¥ã¡ã³ãã® ã€ã³ããã¯ã¹ã®ç®¡ç ãåãããŠãåç
§ãã ããã Indexã®ãããã€æŠç¥ãšEmbeddingã®æŽæ°ã»å逿¹æ³ æ¬çªã·ã¹ãã ã®éçšèгç¹ã«ãããŠIndexã®ãããã€ã»ãªãªãŒã¹ç®¡çãã©ã®ããã«è¡ãããIndexã®æŽæ°ã»å逿ã¯ã©ã®ãããªãããŒã§è¡ãã®ããããŠã³ã¿ã€ã ã¯ããã®ãã¯éèŠãªãã€ã³ããšããŠèæ
®ããŠããŸããé¢é£ã³ãŒãã¬ã³ã¡ã³ãã§ãVertex AI Vector Searchãå©çšããŠã·ã¹ãã ãæ§æããéã«ãããã®ãã€ã³ãã«ã¯æ³šæããŸããã æ¬ç¯ã§ã¯é¢é£ã³ãŒãã¬ã³ã¡ã³ãã®ã·ã¹ãã ã«ãããIndexã®ãããã€æŠç¥ã®ç޹ä»ãšãIndexã®æŽæ°ã»åé€ã®å®è£
æã®æ³šæç¹ã«ã€ããŠèª¬æããŸãã Indexã®ãããã€æŠç¥ æ¬ã·ã¹ãã ã«ãããIndexã®ãªãªãŒã¹ãããŒãã説æããŸãã åç¯ïŒ Index Endpointã®äœæãšIndexã®ããã〠ïŒã§ã®èª¬æã®éãæ¬ã·ã¹ãã ã§ã¯ãIndexã®ãããã€å
ãšãªãIndex EndpointãTerraformã§äœæããŠããŸããIndex Endpointã®äœæåŸã¯ãVertex AI Pipelinesãå©çšããŠIndexããããã€ããŸãã Indexã®ãããã€ã¯Terraformã® google_vertex_ai_index_endpoint_deployed_index ãªãœãŒã¹ã§ãå¯èœã§ããäžæ¹ã§æ¬ã·ã¹ãã ã§ã¯æ°èŠã«Indexãäœæããå ŽåãIndexã®äœæãšãããã€ã®äœæ¥ãåããªãæ¹ãéçºè
ã®æéãå°ãªããšèããŸããããã®ããIndexã®äœæãšãããã€ã¯ã©ã¡ããVertex AI Pipelinesã§è¡ãæ§æã«ããŸããã ãã å®ééçšããŠã¿ããšãIndexã®ãããã€ã«é¢ããŠã¯Vertex AI Pipelinesã§è¡ã£ãŠãTerraformã§è¡ã£ãŠãéçšäžå€§ããªéãã¯ãªããšæããŠããŸãã倧åãªã®ã¯Indexã®ãããã€æã«ã¯å
ã®ã·ã£ãŒããµã€ãºãšãã·ã³ã¿ã€ããã·ã£ãŒãæ°ãšã¬ããªã«æ°ãªã©ãIndexã®æ§æãã©ã¡ãŒã¿ãšã€ã³ãã©ã®èšå®ã坿¥ã«é¢ãããã®ãããç¹ã«ã€ããŠçè§£ããããšã ãšæããŸããäŸãã°IndexãäœæããããŒã ãšãããã€ãæ
åœããã€ã³ãã©ããŒã ãåãããŠããå Žåã¯ãIndexäœæãšãããã€ã«ã€ããŠãäºãã®æ£²ã¿åãã飿ºã«ã€ããŠã®èªèåãããéèŠãšæããŸããã æ¬ã·ã¹ãã ã«ãããIndexã®ãªãªãŒã¹ãããŒã¯æ¬¡å³ã§è¡šããŸãã Vertex AI Vector Searchã§ã®ãã¯ãã«æ€çŽ¢ã¯ãšãªã¯MLOpsãããã¯ã§éçºã»éçšããã¢ããªã±ãŒã·ã§ã³APIããå®è¡ããŠããŸããã¢ããªã±ãŒã·ã§ã³APIã¯GKEäžã§åäœããŠãããã³ã³ããã®ç°å¢å€æ°ã§ã¯ãšãªå®è¡å
ã®Index EndpointãIndexãæå®ããŠããŸãã MLOpsãããã¯ã§ã¯ãGKEäžã§åäœããAPIã®ãããã€ãKubernetesåãã®GitOpsããªããªãŒããŒã«ã§ãã Argo CD ã§è¡ã£ãŠããŸãããŸãé«åºŠãªãããã€æŠç¥ã®å®çŸã®ãããKubernetesã®ã«ããªã¢ãªãªãŒã¹ããã«ãŒã°ãªãŒã³ãããã€ã¡ã³ãããµããŒãããããŒã«ã§ãã Argo Rollouts ãå©çšããŠããŸãã Argo Rolloutsã§ã¯ãããã®ãããã€æŠç¥ã«å ããŠAPIã®SLOåæãçµã¿åãããæ®µéçãªãªãªãŒã¹ã»èªåããŒã«ããã¯ã®ä»çµã¿ãæäŸããŠããŸãããã®ãããªãããã€ææ³ãããã°ã¬ãã·ãããªããªãŒãšåŒã³ãŸããã¢ããªã±ãŒã·ã§ã³APIã®ãããã€ã¯Argo CDã§ç®¡çãããŠãããArgo Rolloutsãå©çšããŠããã°ã¬ãã·ãããªããªãŒãå®çŸããŠããŸãã ã¢ããªã±ãŒã·ã§ã³APIã®ã¡ããªã¯ã¹ã¯ Datadog ã§åéããŠããŸããArgo Rolloutsã®SLOåæã«ã¯Datadogã®ã¡ããªã¯ã¹ãå©çšããŠããŸãããšã©ãŒã¬ãŒããã¬ã€ãã³ã·ã®SLOãèšå®ãããããã®SLOãæºãããªãå Žåã¯èªåã§ããŒã«ããã¯ããããèšå®ããŠããŸãã MLOpsãããã¯ã§å©çšããArgo CDã®è©³çްã¯ãTECH BLOGã®ãMLOpsãã«ãããã³ãã¯ã©ã¹ã¿ãžã®Argo CDã®å°å
¥ãšéçšãããåç
§ãã ããã techblog.zozo.com æ¬ã·ã¹ãã ã®æ§æã§ã¯æ°èŠã«Indexããããã€ããå Žåãã¢ããªã±ãŒã·ã§ã³APIã®ç°å¢å€æ°ã§ã¯ãšãªå®è¡å
ã®Indexã倿ŽããŸãããã®äœæ¥ãæãããšã§ãIndexæ°èŠãããã€æã®å®å
šæ§ã¯ã¢ããªã±ãŒã·ã§ã³APIã®ããã°ã¬ãã·ãããªããªãŒã«ããæ
ä¿ããŠããŸãã ãŸãæ°èŠã«Indexããããã€ããåŸã§åç
§ãããªããªã£ããããã€æžã¿Indexã¯äžèŠãªè²»çšãé²ãããããããã€ãè§£é€ããŸããå³ã§ã¯çç¥ããŠããŸãããã®äœæ¥ã¯æåã§è¡ã£ãŠããŸãã Indexã®æŽæ°ã»åé€ Indexã®æŽæ°æ¹æ³ æ¬ã·ã¹ãã ã§ã¯Vertex AI Pipelinesã®ã¿ã¹ã¯ã§IndexãæŽæ°ããŸããæ¬çªç°å¢ã§ã¯ææ°ã®Indexã§ã¯ãªãéå»ã«äœæããIndexãæŽæ°ãããã±ãŒã¹ã¯ãªããšèããæŽæ°å¯Ÿè±¡ã¯äœææ¥ãæãæ°ããIndexãšããŸããã from google.api_core.client_options import ClientOptions from google.cloud.aiplatform_v1 import IndexServiceClient, ListIndexesRequest from google.cloud.aiplatform_v1.services.index_service import pagers from google.cloud.aiplatform_v1.types import Index from google.cloud.aiplatform.matching_engine import MatchingEngineIndex location = "asia-northeast1" project_id = "<YOUR_PROJECT_ID>" target_index_display_name = "<YOUR_TARGET_INDEX_DISPLAY_NAME>" option = ClientOptions(api_endpoint=f "{location}-aiplatform.googleapis.com" ) client = IndexServiceClient(client_options=option) parent = f "projects/{project_id}/locations/{location}" filter = f 'display_name="{target_index_display_name}"' request = ListIndexesRequest(parent=parent, filter = filter ) page_result: pagers.ListIndexesPager = client.list_indexes(request=request) latest_index: Index = sorted (page_result, key= lambda i: i.create_time, reverse= True )[ 0 ] index_name = latest_index.name index = MatchingEngineIndex(index_name=index_name) gcs_uri = str (contents_delta_jsonl_dir).replace( "/gcs/" , "gs://" ) index.update_embeddings( contents_delta_uri=gcs_uri, is_complete_overwrite=is_complete_overwrite, ) æ¬ã·ã¹ãã ã§ã¯IndexæŽæ°æ¹æ³ãšããŠãããæŽæ°ãå©çšããŠããŸãããããæŽæ°ã®å Žå㯠MatchingEngineIndex.update_embeddings ã¡ãœãããå©çšããŠIndexãæŽæ°ããŸããæŽæ°æã«æå®ãããœãŒã¹ããŒã¿ã¯Indexäœææã®ãœãŒã¹ããŒã¿ãšåã圢åŒã§çšæãã contents_delta_uri åŒæ°ã§æå®ããŸãã is_complete_overwrite åŒæ°ã¯æŽæ°æã«Indexå
šäœãäžæžããããã©ãããæå®ãããã®ã§ããæ¬ã·ã¹ãã ã§ã¯Indexå
šäœãæŽæ°åã§çœ®ãæãããèš³ã§ã¯ãªããéšåçã«IndexãæŽæ°ããããåžžã« False ãæå®ããŠããŸãã ãœãŒã¹ããŒã¿ã«å«ãŸããããŒã¿ãã€ã³ãã®idãæ¢ã«Indexã«ååšããå Žåã¯æ°ããEmbeddingããŒã¿ã§äžæžããããŸãããœãŒã¹ããŒã¿ã«å«ãŸããããŒã¿ãã€ã³ãã®idãIndexã«ååšããªãå Žåã¯æ°èŠã«è¿œå ãããŸãã Indexã®æŽæ°åŸã¯Indexãèªåã§åæ§ç¯ïŒå§çž®ïŒãããåæ§ç¯äžã¯å¢åIndexãäœæãããŸããå¢åIndexãååšããéã¯ãæŽæ°åŸã«ã³ã³ãœãŒã«ãã確èªã§ããIndexã®å¯ãã¯ãã«æ°ãå®éã«Indexã«å«ãŸããidæ°ããå€ããªãããšããããŸãããŸãå¢åIndexãååšããéã¯ãã¯ãã«æ€çŽ¢ã¯ãšãªå®è¡æã«å¢åIndexãåªå
çã«åç
§ãããããããã®éãæ°èŠæŽæ°ããEmbeddingã䜿ã£ãŠãã¯ãã«æ€çŽ¢ããŸãã ãããã€æžã¿ã®IndexãæŽæ°ããå Žåã¯Indexã®æŽæ°åŸã«åæãå¿
èŠã§ããåºæ¬çã«ããã¯ã°ã©ãŠã³ãã§èªååæãããŸãããæŽæ°ããåæã®å®äºãŸã§ã¯è¥å¹²ã®ã©ã°ãããããšã«æ³šæãå¿
èŠã§ããã³ã³ãœãŒã«ãã確èªã§ãããããã€æžã¿Indexã® ååã®åæ ã®å€ã¯Embeddingã®æŽæ°ã«é¢ããã宿çã«æŽæ°ãããŸããããã¥ã¡ã³ãçã«æèšã¯ãªãã£ãããæå
ã§ç¢ºèªããå€ã«ã¯ãªããŸããããã®å€ã¯5åééã§æŽæ°ãããŠããŸããããã®ããçµéšçã«ã¯IndexæŽæ°ã®å®äºåŸ5å以å
ã«åæãå®äºããŸãã ãããã€æžã¿IndexãæŽæ°ããå ŽåãæŽæ°ãåæ§ç¯ãåæã®éã«ããŠã³ã¿ã€ã ã¯ãããŸããã Indexã®æŽæ°ã«é¢ãã詳现ã¯å
¬åŒããã¥ã¡ã³ãã® ã¢ã¯ãã£ã ã€ã³ããã¯ã¹ã®æŽæ°ãšåæ§ç¯ ããåç
§ãã ããã Indexã®å逿¹æ³ Indexå
ã®EmbeddingããŒã¿ãåé€ããå Žåã¯ãIndexã®ãããæŽæ°ãšåãã MatchingEngineIndex.update_embeddings ã¡ãœãããåŒã³åºããŸããããŒã¿æŽæ°ãšç°ãªãç¹ã¯ contents_delta_uri åŒæ°ã§ãœãŒã¹ããŒã¿ã«æå®ãããã©ã«ãã®æ§æã§ããæå®å
ã®ãã©ã«ãå
ã« delete ãšããååã§ãµããã£ã¬ã¯ããªãäœæããåé€å¯Ÿè±¡ã®ããŒã¿ãã€ã³ãã®idãæå®ããããã¹ããã¡ã€ã«ãé
眮ããŸãã batch_root/ delete/ delete_file.txt ããã¹ããã¡ã€ã«ã«ã¯æ¬¡ã®äŸã®ããã«åè¡ã«1ã€ã®idãæå®ããŸãã 1 2 Indexã®æŽæ°ãšåé€ã¯äžåºŠã® MatchingEngineIndex.update_embeddings ã¡ãœããã®åŒã³åºãã§åæã«è¡ããªãããšã«æ³šæããŠãã ãããæå®ãããã©ã«ãå
ã«EmbeddingæŽæ°çšã®ãã¡ã€ã«ãš delete ãµããã£ã¬ã¯ããªå
ã«åé€çšã®ãã¡ã€ã«ã®äž¡æ¹ãååšããå ŽåãIndexããŒã¿ã®åé€ãè¡ãããªããšããäºæ
ããããŸãããIndexã®æŽæ°ãšåé€ãå¥ã
ã®åŠçãšããŠå®è£
ããããšã§è§£æ¶ã§ãããããæ¬ã·ã¹ãã ã§ã¯åŠçãåããŠããŸãã ãã¯ãã«æ€çŽ¢ã¯ãšãªå®è¡ãšãã£ã«ã¿ãªã³ã°ã®å®è£
ã¯ãšãªå®è¡ã®å®è£
Python Vertex AI SDKãšcurlã³ãã³ãã§ã®ãã¯ãã«æ€çŽ¢ã¯ãšãªå®è¡ã®æ¹æ³ã¯å
¬åŒããã¥ã¡ã³ãïŒ ãããªã㯠ã€ã³ããã¯ã¹ãã¯ãšãªããŠæè¿åãååŸãã ïŒã«èšèŒãããŠããŸããå
¬åŒããã¥ã¡ã³ãã ãã§ãªãGoogle Cloudã®Vertex AI Vector Searchã®ã³ã³ãœãŒã«ã§ã grpc_cli ãšPythonã§ã®å®è£
äŸãçšæãããŠããŸããã³ã³ãœãŒã«ã®Pythonå®è£
äŸã¯Vertex AI Pythonã¯ã©ã€ã¢ã³ãã©ã€ãã©ãªãå©çšããå®è£
ã«ãªã£ãŠããŸãã Vertex AI SDKã¯ã¯ã©ã€ã¢ã³ãã©ã€ãã©ãªãããé«ãæœè±¡åã¬ãã«ã§åäœããŸããããé«ãæè»æ§ãå¶åŸ¡çã®ãã¥ãŒãã³ã°ãå¿
èŠãªå Žåã¯Vertex AI Pythonã¯ã©ã€ã¢ã³ãã©ã€ãã©ãªã®å©çšãæ€èšã§ããŸãã æ¬ã·ã¹ãã ã®ã¢ããªã±ãŒã·ã§ã³APIã¯WEARã®ããã¯ãšã³ãAPIããã®ãªã¯ãšã¹ããåããŠãVertex AI Vector SearchãDBãžã¢ã¯ã»ã¹ãã圹å²ãæ
ããŸãããããã®Vertex AI Vector SearchãDBãžã®ãªã¯ãšã¹ãã¯I/OããŠã³ããªåŠçã§ãããI/OåŸ
ã¡ã®éã«ä»ã®åŠçãéåæã«å®è¡ããããšã§APIã®ããã©ãŒãã³ã¹ãåäžã§ããŸãã ã¢ããªã±ãŒã·ã§ã³APIã®å®è£
ã«ã¯Pythonã®Webãã¬ãŒã ã¯ãŒã¯ã§ãã Fast API ãå©çšããŠããããšã³ããã€ã³ãã®å®çŸ©ã§ããPath Operationã«æž¡ã颿°ã¯ async def ãšããŠå®çŸ©ããŠããŸããFast APIã§ã¯ãã®é¢æ°ã async def ã§å®çŸ©ããããšã§ ã€ãã³ãã«ãŒã ãå©çšããéåæåŠçãå¯èœã«ãªããŸããäžæ¹ã§é¢æ°ã def ã§å®çŸ©ãããšã¹ã¬ããããŒã«ãå©çšãã䞊ååŠçãè¡ããŸããã€ãã³ãã«ãŒãã䜿ã£ãæ¹æ³ã¯ã¹ã¬ããããŒã«ã䜿ã£ãæ¹æ³ãããé«ãäžŠè¡æ§ãæã¡ãŸãã Path Operationã«æž¡ã颿°ã async def ã§å®çŸ©ããéã®æ³šæç¹ãšããŠã颿°å
ã«åæåŠçããããšã€ãã³ãã«ãŒãããããã¯ãããŠããŸãããã©ãŒãã³ã¹ã®äœäžã«ã€ãªãããŸãããã®ãã async def ã§é¢æ°ãå®çŸ©ãå¹ççã«äžŠè¡åŠçããã«ã¯ã颿°å
ã®éããåæåŠçãI/OåŸ
ã¡ã®ããåŠçãå
šãŠéåæåŠçã«ããŸãã Vertex AI Vector Searchã§ã®ãã¯ãã«æ€çŽ¢ã¯ãšãªå®è¡ã«ã€ããŠãPython Vertex AI SDKã§çšæãããŠããã¯ã©ã€ã¢ã³ãã¯éåæåŠçã«å¯Ÿå¿ããŠããŸããããã¯ãã«æ€çŽ¢ã¯ãšãªãéåæã«å®è¡ãããããæ¬ã·ã¹ãã ã§ã¯Vertex AI Pythonã¯ã©ã€ã¢ã³ãã©ã€ãã©ãªã§éåæåŠçã«å¯Ÿå¿ããã¯ã©ã€ã¢ã³ããæå®ããŠå®è£
ããŸããã äžè¿°ããã³ã³ãœãŒã«ã®Pythonå®è£
äŸãéåæåŠçã«å¯Ÿå¿ããããšæ¬¡ã®ããã«ãªããŸããæ³šæç¹ãšããŠã grpc.insecure_channel ã grpc.aio.insecure_channel ã¯éä¿¡ãæå·åããŸããããã®ããå®éã®å®è£
ã§ã¯ grpc.aio.secure_channel ãå©çšããŠéä¿¡ãæå·åããããšãæšå¥šããŸãã import grpc from google.cloud.aiplatform_v1beta1 import MatchServiceAsyncClient from google.cloud.aiplatform_v1beta1.services.match_service.transports.grpc_asyncio import ( MatchServiceGrpcAsyncIOTransport, ) index_endpoint_ip_port = "<YOUR_gRPC_ADDRESS>:10000" channel = grpc.aio.insecure_channel(target=index_endpoint_ip_port) transport = MatchServiceGrpcAsyncIOTransport(channel=channel) client = MatchServiceAsyncClient(transport=transport) ãã¯ãã«æ€çŽ¢ã¯ãšãªå®è¡ã®å®è£
äŸã¯æ¬¡ã§ãã from collections.abc import MutableSequence from google.cloud.aiplatform.matching_engine import MatchingEngineIndexEndpoint from google.cloud.aiplatform_v1.types import ( FindNeighborsRequest, FindNeighborsResponse, IndexDatapoint, ) feature_vector: list [ float ] = [ 0.1 , 0.2 , 0.3 , ...] # ã¯ãšãªã«äœ¿çšãããã¯ãã«ïŒâ» å®éã®å€ã«çœ®ãæããŠãã ããïŒ neighbor_count: int = 1000 # ååŸããè¿åã®æ° approximate_neighbor_count: int = 10000 # è¿äŒŒæ¢çŽ¢ã§ååŸããè¿åã®æ° fraction_leaf_nodes_to_search: float = 0.05 # æ€çŽ¢ãããªãŒãããŒãã®å²å index_endpoint_name = ( "projects/<YOUR_PROJECT_NAME>/locations/asia-northeast1/indexEndpoints/<YOUR_INDEX_ENDPOINT_ID>" ) deployed_index_id = "<YOUR_DEPLOYED_INDEX_ID>" # ãããã€æžã¿Indexã®id datapoint = IndexDatapoint( feature_vector=feature_vector, ) vector_search_endpoint = MatchingEngineIndexEndpoint( index_endpoint_name=index_endpoint_name, ) query = FindNeighborsRequest.Query( datapoint=datapoint, neighbor_count=neighbor_count, approximate_neighbor_count=approximate_neighbor_count, fraction_leaf_nodes_to_search_override=fraction_leaf_nodes_to_search, ) find_neighbors_req = FindNeighborsRequest( index_endpoint=index_endpoint_name, deployed_index_id=deployed_index_id, queries=[query], return_full_datapoint= False , ) res: FindNeighborsResponse = await client.find_neighbors( request=find_neighbors_req, ) neighbors: MutableSequence[FindNeighborsResponse.NearestNeighbors] = ( res.nearest_neighbors ) å
ã®æ¹æ³ã§åæåããã¯ã©ã€ã¢ã³ãã® find_neighbors ã¡ãœãããåŒã³åºãããšã§ãã¯ãã«æ€çŽ¢ãªã¯ãšã¹ããå®è¡ã§ããŸããããã§ find_neighbors ã¡ãœããã¯éåæã¡ãœããã§ããã await ãä»ããŠåŒã³åºãããšã§ãã¯ãã«æ€çŽ¢ãªã¯ãšã¹ããéåæã«å®è¡ã§ããŸãã find_neighbors ã¡ãœããã®åŒæ°ã® request åŒæ°ã« FindNeighborsRequest ã¯ã©ã¹ ã®ãªããžã§ã¯ããæž¡ãããšã§ããã¯ãã«æ€çŽ¢å
ãã¯ãšãªå®è¡ã®ãã©ã¡ãŒã¿ãæå®ã§ããŸãããŸãèšèŒã¯çããŠããŸããã timeout åŒæ°ã retry åŒæ°ãæå®ããããšã§ã¿ã€ã ã¢ãŠãããªãã©ã€ã®èšå®ãå¯èœã§ãã ã¯ãšãªå®è¡å¯Ÿè±¡ã®Index Endpointãšãããã€æžã¿Index㯠FindNeighborsRequest ã¯ã©ã¹ã®åæåæã«æå®ããŸãã index_endpoint åŒæ°ã§Index Endpointã®ãªãœãŒã¹åãã deployed_index_id åŒæ°ã§ãããã€æžã¿Indexã®idãæå®ããŸãã queries åŒæ°ã« FindNeighborsRequest.Query ã¯ã©ã¹ã®ãªããžã§ã¯ããèŠçŽ ãšããŠæã€é
åãæž¡ããŠãã¯ãã«æ€çŽ¢ã¯ãšãªã®å
容ãæå®ããŸãã queries åŒæ°ã®å€ã¯é
åã§ãããèŠçŽ ãè€æ°ããå Žåã¯ã¯ãšãªãåæã«æå®ã§ããŸããæ¬ã·ã¹ãã ã§ã¯ç¹ã« ãã€ããªããæ€çŽ¢ ã¯å©çšããŠããªãããã¯ãšãªã¯1ã€æå®ããŠããŸãããŸã return_full_datapoint åŒæ°ã§ãã¯ãã«æ€çŽ¢çµæã®ããŒã¿ãã€ã³ããæã€å
šãŠã®æ
å ±ãè¿ããã©ãããæå®ã§ããŸãã return_full_datapoint åŒæ°ã®å€ã True ã«ããå ŽåãããŒã¿ãã€ã³ãã®idã ãã§ãªããEmbeddingã restricts ãã£ãŒã«ãã®å€ãè¿ããŸããæ¬ã·ã¹ãã ã§ã¯ããŒã¿ãã€ã³ãã®idã®ã¿ãããã°ååã§ãããã¬ã¹ãã³ã¹ãµã€ãºãå°ããããããã« False ãæå®ããŠããŸãã FindNeighborsRequest.Query ã¯ã©ã¹ã®åæåæã datapoint åŒæ°ã« IndexDatapoint ã¯ã©ã¹ ã®ãªããžã§ã¯ããæž¡ãããšã§ãã¯ãã«æ€çŽ¢ã®å¯Ÿè±¡ãæå®ããŸããã¯ãšãªå®è¡æã®å
¥åã«ã¯Embeddingã ãã§ãªããIndexã«å«ãŸããããŒã¿ãã€ã³ãã®idãæå®ã§ããŸããæ¬ã·ã¹ãã ã§ã¯ããŒã¿ãã€ã³ãã®idãšããŠã³ãŒãã£ããŒãã®idãå©çšããŠããŸããã¢ããªã±ãŒã·ã§ã³APIã¯WEARã®ããã¯ãšã³ãAPIããã³ãŒãã£ããŒãã®idãåãåãããã®ãŸãŸ IndexDatapoint ã® datapoint_id ãã£ãŒã«ãã«æå®ããŠã¯ãšãªå®è¡ããŠããŸãããŸã IndexDatapoint ã¯ã©ã¹ã«ã¯ restricts ãã£ãŒã«ããæå®å¯èœã§ãããã¡ãã¯åŸç¯ïŒ ãã£ã«ã¿ãªã³ã°ã®å®è£
ïŒã§è©³ãã説æããŸãã äžèšã®äŸã§ã¯ã¯ãšãªå®è¡æã®å
¥åã«Embeddingã䜿çšããå®è£
äŸã§ãããããŒã¿ãã€ã³ãã®idãæå®ããå Žåã¯æ¬¡ã®ããã« IndexDatapoint ãªããžã§ã¯ããäœæã§ããŸãã datapoint_id= "1234" datapoint = IndexDatapoint( datapoint_id=datapoint_id ) åç¯ïŒ ã¯ãšãªå®è¡æã«äžæžãããŠãããã©ã¡ãŒã¿ ïŒã§ã®èª¬æã®éãã approximateNeighborsCount ã» fractionLeafNodesToSearch ã» setNeighborCount ã®å€ã¯ã¯ãšãªå®è¡æã«äžæžãããIndexäœææã®æå®å€ã倿ŽããŠããŸãããããã®å€ã¯ FindNeighborsRequest.Query ã¯ã©ã¹ã®åæåæã«æå®ããŸãã ãã£ã«ã¿ãªã³ã°ã®å®è£
Vertex AI Vector Searchã§ã¯Indexã®ããŒã¿ãã€ã³ããžå±æ§æ
å ±ãä»äžãã restricts ãã£ãŒã«ããå©çšããŠããã¯ãã«æ€çŽ¢ã¯ãšãªã®å¯Ÿè±¡ãIndexã®ãµãã»ããã«å¶éã§ããŸãã åç¯ïŒ 2段éã®ãã¯ãã«æ€çŽ¢ ïŒã§èª¬æããéããæ¬ã·ã¹ãã ã§ã¯2段éã«ãã¯ãã«æ€çŽ¢ãããŸãã restricts ãã£ãŒã«ããå©çšããããšã§ã2段éç®ã®ãã¯ãã«æ€çŽ¢ã¯ãšãªå®è¡æã«ãæ€çŽ¢å¯Ÿè±¡ã1段éç®ã®ãã¯ãã«æ€çŽ¢ã§ååŸãããµãã»ããã«å¶éããŠããŸãã restricts ãã£ãŒã«ããæå®ããå Žåãåå空éïŒ namespace ãã£ãŒã«ãïŒã®æå®ãå¿
èŠã§ãããŸããªãã·ã§ã³ãšããŠããŒã¯ã³ïŒ allow ã» deny ãã£ãŒã«ãã®å€ïŒãæå®ã§ããŸããããŒã¯ã³ã®å€ã¯æååã®é
åã«ãªããŸããã¯ãšãªå®è¡æã« allow_list ã deny_list ãæå®ããããšã§ãããŒã¯ã³ã®å€ã«å¿ããŠæ€çŽ¢å¯Ÿè±¡ãå¶éã§ããŸãã åç¯ïŒ Indexäœæã®å®è£
ïŒã§ç޹ä»ããIndexãœãŒã¹ããŒã¿ã®ãã¡ã€ã«ã®å€ã« restricts ãã£ãŒã«ããæå®ãããšã次ã®äŸã®ããŒã¿åœ¢åŒã«ãªããŸãã { " id ": " 1 ", " embedding ": [ 0.1 , 0.05 , 0.3 ] , " restricts ": [{ " namespace ": " snap_id ", " allow ": [ " 1 " ]}] } { " id ": " 2 ", " embedding ": [ 0.2 , 0.01 , 0.02 ] , " restricts ": [{ " namespace ": " snap_id ", " allow ": [ " 2 " ]}]} åç¯ïŒ ã¯ãšãªå®è¡ã®å®è£
ïŒã«èšèŒã®éããæ¬ã·ã¹ãã ã§ã¯Indexã®ããŒã¿ãã€ã³ãã®idãšããŠã³ãŒãã£ããŒãã®idãå©çšããŠããŸãããã£ã«ã¿ãªã³ã°ã«ã¯ååŸããããŒã¿ãã€ã³ãã®idãæå®ãããããäžèšã®ããã« restricts ãã£ãŒã«ãã®ããŒã¯ã³ã®å€ã«ãã³ãŒãã£ããŒãã®idãå©çšããŠããŸãããŸã allow ã» deny ãã£ãŒã«ãã¯ãªãã·ã§ã³ã§ãããããæå®ããªãå Žåã¯ç©ºã®é
åã«ãªããŸããæ¬ã·ã¹ãã ã§ã¯ allow ãã£ãŒã«ãã®ã¿ãæå®ããŠããŸãã æ¬¡ã«ç€ºãã®ã¯ restricts ãã£ãŒã«ããæå®ãããã¯ãã«æ€çŽ¢ã¯ãšãªå®è¡ã«ããã IndexDatapoint ã¯ã©ã¹ã®ãªããžã§ã¯ãã®åæåã®äŸã§ããæ¬ã·ã¹ãã ã®2段éç®ã®ãã¯ãã«æ€çŽ¢ã¯ãšãªå®è¡ã§ã¯ IndexDatapoint.Restriction ã¯ã©ã¹ã®åŒæ°ã« allow_list ãæå®ããã¯ãšãªå®è¡ã®å¯Ÿè±¡ããµãã»ããã«å¶éããŠããŸããã€ãŸã find_neighbors ã¡ãœããåŒã³åºãã®æ»ãå€ã¯ allow_list ã«æå®ããå€ãæã€ããŒã¿ãã€ã³ãã®ã¿ã«ãªããŸãã from google.cloud.aiplatform_v1.types import IndexDatapoint namespace = "snap_id" # åå空éã®æå® allow_list = [ "1" ] # èš±å¯å¯Ÿè±¡ãšããå€ã®é
å datapoint = IndexDatapoint( feature_vector=feature_vector, restricts=[ IndexDatapoint.Restriction( namespace=namespace, allow_list=allow_list, ) ], ) ãŸãæ¬ã·ã¹ãã ã§ã¯1段éç®ã®ãã¯ãã«æ€çŽ¢ã¯ãšãªå®è¡æã«ã restricts ãã£ãŒã«ããæŽ»çšããŠããŸããVertex AI Vector Searchã§ã¯ããŒã¿ãã€ã³ãã®idãæå®ããŠãã¯ãã«æ€çŽ¢ã¯ãšãªãå®è¡ããå Žåããã¯ãã«æ€çŽ¢çµæã«ã¯èªèº«ã®ããŒã¿ãã€ã³ããå«ãŸããŸããæ¬ã·ã¹ãã ã§ã¯1段éç®ã®ãã¯ãã«æ€çŽ¢ã¯ãšãªå®è¡æã«ããŒã¿ãã€ã³ãã®idãæå®ããŠè¿åãååŸããŠããŸãããã®ããããã©ã«ãã§ã¯æå®ããããŒã¿ãã€ã³ãã®idã find_neighbors ã¡ãœããåŒã³åºãã®æ»ãå€ã«å«ãŸããŸããæ¬ã·ã¹ãã ã§ã¯å
¥åãšãªãã³ãŒãã£ããŒãã®é¡äŒŒã³ãŒãã£ããŒããååŸãããããã find_neighbors ã¡ãœããåŒã³åºãã®æ»ãå€ã«å
¥åãšãªãã³ãŒãã£ããŒãã®idãå«ããããããŸãããå
¥åããããŒã¿ãã€ã³ãã®idãæ€çŽ¢çµæããé€å€ããããã IndexDatapoint.Restriction ã¯ã©ã¹ã®åŒæ°ãž deny_list ãæå®ããŠããŸãã from google.cloud.aiplatform_v1beta1.types import IndexDatapoint namespace = "snap_id" # åå空éã®æå® deny_list = [ "1" ] # é€å€å¯Ÿè±¡ãšããå€ã®é
å datapoint = IndexDatapoint( datapoint_id=datapoint_id, restricts=[ IndexDatapoint.Restriction( namespace=namespace, deny_list=deny_list ) ], ) Restrictsã®å©çšæã«æ³šæãå¿
èŠãªã®ã¯ãIndexäœææã«æå®ãã allow ã» deny ãã£ãŒã«ããšã¯ãšãªå®è¡æã«æå®ãã allow_list ã» deny_list ã¯ããããã®çµã¿åããã§ãã£ã«ã¿ãªã³ã°ãããç¹ã§ãããã£ã«ã¿æ¡ä»¶ãšäžèŽã«ã€ããŠæ¬¡ã®äŸã§èª¬æããŸãããã®äŸã¯å
¬åŒããã¥ã¡ã³ãïŒ ãã¯ãã«äžèŽããã£ã«ã¿ãã ïŒã§ã®èª¬æãå
ã«ããŠããŸãã äŸãã° color åå空éã«ã€ããŠã次㮠allow ã» deny ãã£ãŒã«ããæã€ããŒã¿ãã€ã³ããIndexã«ååšãããšããŸãã A : { " id ": " A ", " embedding ": [ ... ]} B : { " id ": " B ", " embedding ": [ ... ] , " restricts ": [{ " namespace ": " color ", " allow ": [ " red " ] , " deny ": []}]} C : { " id ": " C ", " embedding ": [ ... ] , " restricts ": [{ " namespace ": " color ", " allow ": [ " blue " ] , " deny ": []}]} D : { " id ": " D ", " embedding ": [ ... ] , " restricts ": [{ " namespace ": " color ", " allow ": [ " orange " ] , " deny ": []}]} E : { " id ": " E ", " embedding ": [ ... ] , " restricts ": [{ " namespace ": " color ", " allow ": [ " red ", " blue " ] , " deny ": []}]} F : { " id ": " F ", " embedding ": [ ... ] , " restricts ": [{ " namespace ": " color ", " allow ": [ " red " ] , " deny ": [ " blue " ]}]} G : { " id ": " G ", " embedding ": [ ... ] , " restricts ": [{ " namespace ": " color ", " allow ": [ " red ", " blue " ] , " deny ": [ " blue " ]}]} # å®åäžå¿
èŠæ§ãèãã±ãŒã¹ H : { " id ": " H ", " embedding ": [ ... ] , " restricts ": [{ " namespace ": " color ", " allow ": [] , " deny ": [ " blue " ]}]} äžèšããœãŒã¹ããŒã¿ã«æã€Indexã«å¯ŸããŠã¯ãšãªå®è¡æã«æå®ãã allow_list ã» deny_list ã®ãã¿ãŒã³ãšããŠæ¬¡ã®äŸãèããŸãã (1) {} # restricts ã®æå®ãªã (2) { " namespace ": " color ", " allow_list ": [ " red " ] , " deny_list ": []} (3) { " namespace ": " color ", " allow_list ": [ " blue " ] , " deny_list ": []} (4) { " namespace ": " color ", " allow_list ": [ " orange " ] , " deny_list ": []} (5) { " namespace ": " color ", " allow_list ": [ " red ", " blue " ] , " deny_list ": []} (6) { " namespace ": " color ", " allow_list ": [ " red " ] , " deny_list ": [ " blue " ]} (7) { " namespace ": " color ", " allow_list ": [ " red ", " blue " ] , " deny_list ": [ " blue " ]} # å®åäžå¿
èŠæ§ãèãã±ãŒã¹ (8) { " namespace ": " color ", " allow_list ": [] , " deny_list ": [ " blue " ]} ãã®æãããŒã¿ãã€ã³ãã® allow ã» deny ãã£ãŒã«ããšã¯ãšãªå®è¡æã«æå®ãã allow_list ã» deny_list ã®çµã¿åããã«ãããã£ã«ã¿ãªã³ã°çµæã¯æ¬¡ã«ãªããŸããâ¯ã¯æ¡ä»¶ãäžèŽãããŒã¿ãã€ã³ããååŸã§ããçµã¿åããã§ãã空æ¬ã¯æ¡ä»¶ãäžèŽããããŒã¿ãã€ã³ããååŸã§ããªãçµã¿åããã衚ããŠããŸãã A B C D E F G H (1) ⯠⯠⯠⯠⯠⯠⯠⯠(2) ⯠⯠⯠⯠(3) ⯠⯠(4) ⯠(5) ⯠⯠⯠(6) ⯠⯠(7) ⯠(8) ⯠⯠⯠⯠⯠æ¬ã·ã¹ãã ã§ã¯ããŒã¿ãã€ã³ãã®äœææã allow ãã£ãŒã«ãã«1ã€ã®ããŒã¯ã³ã®å€ãæãããŠãããããäžèšã®äŸã§ã¯ããŒã¿ãã€ã³ã B~D ã®ãã¿ãŒã³ã«è©²åœããããŒã¿ãã€ã³ããIndexã«ååšããŸãã 1段éç®ã®ã¯ãšãªå®è¡ã§ã¯å
¥åãšãªãã³ãŒãã£ããŒãã®idãé€å€ããããã« deny_list ãæå®ããŠãããããã¯äžèšã®äŸã§ã¯ã¯ãšãªã® (8) ã«è©²åœããŸãã (8)ãšB~D ã§äžèŽãèŠããš (8)ãšC 以å€ã®çµã¿åããã§äžèŽããããšãããããŸãããŸã2段éç®ã®ã¯ãšãªå®è¡ã§ã¯1段éç®ã®ãã¯ãã«æ€çŽ¢ã§ååŸããè€æ°ã®ããŒã¿ãã€ã³ãã®idã allow_list ã«æå®ããŠãããäžèšã®äŸã§ã¯ã¯ãšãªã® (5) ã«è©²åœããŸãã (5)ãšB~D ã§äžèŽãèŠããš (5)ãšB ã (5)ãšC ã®çµã¿åããã§äžèŽããããšãããããŸãã ãã®ããã«ããŒã¿ãã€ã³ãã® restricts ãã£ãŒã«ãã® allow ã» deny ãã£ãŒã«ããšã¯ãšãªå®è¡æã«æå®ãã allow_list ã» deny_list ã®çµã¿åããã§è€éãªæ¡ä»¶ã§ã®ãã£ã«ã¿ãªã³ã°ãå¯èœã§ããäžæ¹ã§æ¬ã·ã¹ãã ã®ãããªç°¡åãªã¯ãšãªã§ããã°ããŒã¿ãã€ã³ãã«æãããã®ã¯ allow ãã£ãŒã«ãã®ã¿ã§ååã§ãããŠãŒã¹ã±ãŒã¹ã«ãã£ãŠæå®ãããã£ãŒã«ãã䜿ãåãã€ã€ãäžèšã®ãããªè¡šã§ãã£ã«ã¿ãªã³ã°å¯Ÿè±¡ãæå³ãããã®ã«ãªãã確èªããããšãããããããŸãã Vertex AI Vector Searchã®æ¬çªéçšãšèª²é¡ã®æ¹å ã¢ãã¿ãªã³ã°éçš Vertex AI Vector Searchã§ã¯ã¡ããªã¯ã¹ããã·ã¥ããŒããããã©ã«ãã§æäŸãããŠãããããã·ã¥ããŒãã§ã¯æ¬¡ã®ã¡ããªã¯ã¹ã確èªã§ããŸãã ããŒãæ° ã·ã£ãŒãæ° ç§éã¯ãšãªæ° ã¬ã€ãã³ã·ïŒ50ã»95ã»99ïŒ
tileïŒ CPU䜿çšç ã¡ã¢ãªäœ¿çšç 次ã«ç€ºãã®ã¯å®éã®ããã·ã¥ããŒãã®ç»é¢ã§ãã æ¬ã·ã¹ãã ã§ã¯çŸç¶ãVertex AI Vector Searchã®ã¡ããªã¯ã¹ã察象ãšããã¢ã©ãŒãã®èšå®ã¯è¡ãªã£ãŠããŸãããã¢ããªã±ãŒã·ã§ã³APIãLoad BalancerãDatadogã®ã¡ããªã¯ã¹ã¯ã¢ãã¿ãªã³ã°ã»ã¢ã©ãŒããèšå®ããŠããŸããVertex AI Vector Searchã®ã¡ããªã¯ã¹ããã·ã¥ããŒãã¯ãããã®ã¢ãã¿ãªã³ã°ã§ç°åžžããã£ãå Žåã®èª¿æ»ææ®µãšããŠåç
§ããŠããŸãã çŸç¶äžèšã®ã¢ãã¿ãªã³ã°éçšã§ç¹ã«å°ã£ãç¹ã¯ãããŸãããä»åŸã®éçšã§Vertex AI Vector Searchåäœã§ã®ã¡ããªã¯ã¹æªåãæ€ç¥ãããã±ãŒã¹ãªã©åºãå Žåã¯ãVertex AI Vector Searchã®ã¿ã®ç£èŠãã¢ã©ãŒãã®èšå®ãæ€èšããŸãã éçšããŠæããã¡ãªãã é¢é£ã³ãŒãã¬ã³ã¡ã³ãã§Vertex AI Vector SearchãçŽ4ãæéãæ¬çªéçšããŠæããå©çšã¡ãªããã玹ä»ããŸãã éçšã»ä¿å®ã®å®¹æã åç¯ïŒ Vertex AI Vector Searchã®å°å
¥ã¢ãããŒã·ã§ã³ ïŒã§Vertex AI Vector Searchã®æ¡çšçç±ãšããŠãããŒãžããµãŒãã¹ã§ããããšãæããŠããŸããããããŒãžããµãŒãã¹ã§ãããããªãªãŒã¹åŸã«å¿
èŠãªéçšã»ä¿å®äœæ¥ã¯ç¹ã«ãªãããŠãŒã¶ãŒåŽã§ã¯ãã¯ãã«æ€çŽ¢Indexã®äœæãæŽæ°ãã¯ãšãªå®è¡ã®å®è£
ã«éäžã§ããŠããŸãã ãŸã éå»ãããžã§ã¯ãã«ããããã¯ãã«æ€çŽ¢æ©èœã®å®è£
ã§è¿°ã¹ããããªããããšAPIéã®äŸåããããŸãããåç¯ïŒ Indexã®ãããã€æŠç¥ ïŒã§è¿°ã¹ãããã«Indexã®ãããã€æŠç¥ãã·ã³ãã«ãªæ§æã«å®è£
ã§ããŠããŸããéå»ãããžã§ã¯ãã«ããããã¯ãã«æ€çŽ¢æ©èœãšæ¯èŒãããšã·ã¹ãã èªäœãã·ã³ãã«ã«ã§ããç¹ãéçšè² è·ã®è»œæžã«ã€ãªãããŸããã å®å®ããããã©ãŒãã³ã¹ æ¬ã·ã¹ãã ããªãªãŒã¹åŸãVertex AI Vector Searchèµ·å ã§ã®åé¡ã¯ã»ãšãã©çºçããŠããããå®å®ããããã©ãŒãã³ã¹ãçºæ®ããŠããŸãã æ¬ã·ã¹ãã ã®å¹³åãªã¯ãšã¹ãæ°ã¯çŽ15req/secã§ããŒã¯æã¯çŽ30req/secã§ããVertex AI Vector Searchã®ãã¯ãã«æ€çŽ¢ãªã¯ãšã¹ãã®ã¬ã€ãã³ã·ã¯99ïŒ
tileã§çŽ60ã120msã§ãããç¹ã«ããŒã¯æã®å€§ããªã¬ã€ãã³ã·æªåãèŠãããŸããã Vertex AI Vector Searchã¯ãããŒãžããµãŒãã¹ã§ããããããã¯ãã«æ€çŽ¢APIãšåãGKEã¯ã©ã¹ã¿å
ã«èªåã§APIãæ§ç¯ããå Žåãšæ¯èŒãããšãããã¯ãŒã¯çãªè·é¢ã¯é¢ããŠããŸããã¯ãšãªå®è¡ã®ã¬ã€ãã³ã·ãžãããã¯ãŒã¯èµ·å ã§ã®ã¬ã€ãã³ã·çãäžä¹ããããããšã§ããã¯ãã«æ€çŽ¢ãªã¯ãšã¹ãæã®ã¬ã€ãã³ã·ãå¢å ããå¯èœæ§ãæžå¿µããŠããŸããããçµæãšããŠæ¬ãããžã§ã¯ãã®èŠä»¶ã«å¯ŸããŠã¯ååãªããã©ãŒãã³ã¹ã§ããã ãŸããããŒãžããµãŒãã¹ãžã®ãªã¯ãšã¹ãæãtimeoutãšã©ãŒã®çºçã床ã
ä»ã·ã¹ãã ã§èŠãããŠãããVertex AI Vector Searchã«ã€ããŠãæžå¿µããŠããŸããããããæ¬ã·ã¹ãã ã§ã®Vertex AI Vector Searchãžã®ãªã¯ãšã¹ãã§ã¯timeoutãšã©ãŒã¯1æ¥1ä»¶çºçãããã©ããã®é »åºŠãšãªã£ãŠããå®å®ããŠããŸãã é¢é£ã³ãŒãã¬ã³ã¡ã³ããããžã§ã¯ãã®åºé¢ã§ã®è² è·ã¯æ¯èŒçã«èŠæš¡ãå°ãããããé«è² è·ãªæ¬çªç°å¢ã§ã®Vertex AI Vector Searchã®å©çšå®çžŸã¯ãŸã MLOpsãããã¯ã§ã¯äœããŠããŸããããããè² è·è©Šéšæã«ã¯æ°èŠå°å
¥ããVertex AI Vector Searchã®è² è·éçãææ¡ãããããæ¬çªã®æ³å®ãã©ãã£ãã¯ã倧ããè¶
ãã~200req/secãŸã§æ€èšŒããŸãããæ€èšŒããç¯å²ã§ã¯Vertex AI Vector Searchã®ã¬ã€ãã³ã·ã«å€§ããªå€åã¯èŠãããŸããã§ããããããã£ãå®çžŸããããä»åŸããå€§èŠæš¡ãªã·ã¹ãã ã§ãã¯ãã«æ€çŽ¢æ©èœãæ§ç¯ããéã«ãååéžæè¢ã«ãªãåŸããšèããŠããŸãã éçšããŠèŠã€ãã£ã課é¡ãšæ¹å Vertex AI Vector Searchãå©çšããæ¬ã·ã¹ãã ãæ¬çªéçšããäžã§æ¬¡ã®èª²é¡ãèŠã€ãããŸããã IndexæŽæ°ã«ãããæéãé·ã IndexæŽæ°æ°ãå€ããšããŒã¿æŽæ°ã«ãããè²»çšãå¢å ãã ããããã®èª²é¡ãšæ¬ã·ã¹ãã ã§å®æœããæ¹åå
容ã玹ä»ããŸãã Indexã®æŽæ°æéãé·ã æ¬ã·ã¹ãã ã§ã¯æ¥æ¬¡ã§Indexã®æŽæ°ãè¡ãªã£ãŠããŸãããŸãåç¯ïŒ Indexã®å逿¹æ³ ïŒã§è¿°ã¹ãéã远å ã»æŽæ°ãšåé€ã¯ã¿ã¹ã¯ãåããŠçŽåã«å®è¡ããŠããŸããIndexã®æŽæ°ã«ãããæéã¯è¿œå ã»æŽæ°ãåé€ãåããããšçŽ2ã3æéãšãªã£ãŠããããã€ãã©ã€ã³ã®å
šäœã®å®è¡æéã«å¯ŸããŠçŽååã®å²åãå ããŠããŸããã ããããã®ã¿ã¹ã¯ã«ãããæéãšæŽæ°æ°ã¯æ¬¡ã®éãã§ããåèãŸã§ã«Indexå
šäœã®ããŒã¿æ°ã¯çŽ1,400äžä»¶ã§ãã 远å ã»æŽæ°ïŒ1ã2æé æŽæ°æ°ïŒçŽ3,500,000ä»¶ åé€ïŒçŽ30ã40å æŽæ°æ°ïŒçŽ30ã60ä»¶ æ¬ã·ã¹ãã ã§ã¯ã€ã³ãã©æ§ç¯ã«ãããè²»çšãã·ã¹ãã ã®è€éæ§ãæãããããAPIã§ã®ãªã¢ã«ã¿ã€ã ãªEmbeddingæœåºã¯è¡ãªã£ãŠããŸããããã®ãããããåŠçã§äºåã«EmbeddingæœåºããããŠãŒã¶ãŒã»ã³ãŒãã£ããŒãã®ã¿ãé¢é£ã³ãŒãã¬ã³ã¡ã³ããããžã§ã¯ãã§ã®MLã䜿ã£ãã¬ã³ã¡ã³ãã®å¯Ÿè±¡ãšããŠããŸãããã®æ§æã§ã¯æ°èŠãŠãŒã¶ãŒã»ã³ãŒãã£ããŒãã®Embeddingãããæ©ãæœåºããã¬ã³ã¡ã³ã察象ãšããã«ã¯ãããã®å®è¡é »åºŠãäžããå¿
èŠããããŸãããŸããããã®å®è¡é »åºŠãäžãããšãããçŽè¿ã®è¡åãåæ ããEmbeddingã䜿ã£ãŠãã¯ãã«æ€çŽ¢ãã§ããããã¬ã³ã¡ã³ãã®ãªã¢ã«ã¿ã€ã æ§ãåäžããŸãã æ¬ã·ã¹ãã ã§é«ããªã¢ã«ã¿ã€ã æ§ã¯èŠæ±ãããŠããŸããããä»åŸãããã®å®è¡é »åºŠãå¢ãããŠãªã¢ã«ã¿ã€ã æ§ãåäžãããå ŽåãIndexã®æŽæ°ã«ãããæéãããã«ããã¯ã«ãªãããšãæžå¿µããŸããã Indexã®ããŒã¿æŽæ°è²»çšã®å¢å Vertex AI Vector Searchã®è²»çšã¯äž»ã«æ¬¡ã®2ã€ã®èŠçŽ ã§æ§æãããŠããŸãã ã€ã³ã¹ã¿ã³ã¹ã®è²»çš Indexæ§ç¯ã®è²»çš ããŒã¿æŽæ°ã«ããIndexæ§ç¯ã®è²»çšã¯Indexã®æŽæ°æã«åŠçãããããŒã¿ã®éã«å¿ããŠèª²éãããŸããVertex AI Vector Searchã®ãããæŽæ°ã§ã¯2025幎7æçŸåšããã¹ãŠã®ãªãŒãžã§ã³ã«äžåŸã§åŠçãããããŒã¿éã«å¯ŸããŠ$3.00/GiBã課éãããŸããè²»çšã«ã€ããŠã¯ å
¬åŒããã¥ã¡ã³ã ã§ææ°ã®æ
å ±ãåç
§ããŠãã ããã æ¬ã·ã¹ãã ã§ã®IndexæŽæ°æ°ã¯1æ¥ããã3,500,000ä»¶ã§ãããVertex AI Vector Searchã®IndexæŽæ°ã«äŒŽãIndexæ§ç¯ã®è²»çšã¯1æ¥ãããçŽ$46ãšãªã£ãŠããŸããã asia-northeast1 ãªãŒãžã§ã³ã§ e2-standard-16 ã€ã³ã¹ã¿ã³ã¹ãããŒãæ°2ã§å©çšããå Žåã®1æ¥ãããã®è²»çšã¯çŽ$46ãšãªããŸãããã®ããIndexæŽæ°ã®è²»çšã¯ã€ã³ã¹ã¿ã³ã¹ã®è²»çšãšåçšåºŠãšãªã£ãŠãããããŒã¿æŽæ°ã«äŒŽãè²»çšããã¯ãã«æ€çŽ¢æ©èœã®è²»çšã®ååè¿ããå ããŠããŸããããã¯åœåæ³å®ããŠããããã倧ããªè²»çšã§ãããåé¡ã«ãªããŸããã Indexã®ããŒã¿æŽæ°è²»çšã®ç®åºã«ã€ããŠã¯æ··ä¹±ããéšåããã£ãããåŸç¯ïŒ Indexã®ããŒã¿æŽæ°è²»çšã®èŠç©ãã ïŒã§å¥é説æããŸãã EmbeddingæŽæ°å¯Ÿè±¡ã®ããŒã¿æ°ãåæž æ¬ã·ã¹ãã ã§ã¯åç¯ã®éçšèª²é¡ã«å¯ŸããŠIndexã®æŽæ°å¯Ÿè±¡ãšãªãããŒã¿æ°ãæžããããšã§å¯Ÿå¿ããŸãããæŽæ°å¯Ÿè±¡ãšãªãããŒã¿æ°ãæžããããšã§æ¬¡ã®æ¹åãåŸãããŸããã Indexã®ããŒã¿æŽæ°è²»çšã®å€§å¹
ãªåæž IndexæŽæ°æéã®ççž® åç¯ïŒ ã°ã©ããã¥ãŒã©ã«ãããã¯ãŒã¯ã¢ãã«ã«ããEmbeddingã®æœåº ïŒã§è¿°ã¹ãéããæ¬ã·ã¹ãã ã§ã¯GNNã¢ãã«ãå©çšããŠEmbeddingãæœåºããŠããŸããæ¬¡ã«è©²åœãããŠãŒã¶ãŒã»ã³ãŒãã£ããŒããEmbeddingæœåºå¯Ÿè±¡ãšããIndexãžè¿œå ã»æŽæ°ããŠããŸããããã®ãã¡ããŒã¿æŽæ°æ°ã®å¢å ã«å€§ããé¢ä¿ããŠããã®ã¯3ã®ããŒã¿ã§ããã æ°èŠè¿œå ãããNode èªèº«ã«çŽã¥ããŠãŒã¶ãŒè¡åã®å€åããã£ãNode äžèšã®æŽæ°å¯Ÿè±¡ã«é£æ¥ããNode 2ã«è©²åœãããŠãŒã¶ãŒã»ã³ãŒãã£ããŒããã¢ã¯ãã£ããªãŠãŒã¶ãŒã»ã³ãŒãã£ããŒããšè¡šçŸããŸãã3ãæŽæ°å¯Ÿè±¡ãšããããšã§ã¢ã¯ãã£ããªãŠãŒã¶ãŒã»ã³ãŒãã£ããŒãã«çŽã¥ãéã¢ã¯ãã£ããªãŠãŒã¶ãŒã»ã³ãŒãã£ããŒããæŽæ°å¯Ÿè±¡ãšãªããŸããç¹ã«ã¢ã¯ãã£ããªãŠãŒã¶ãŒã»ã³ãŒãã£ããŒãã¯é£æ¥ããNodeãå€ããããçµæãšããŠæŽæ°æ°ã®ççºçãªå¢å ãžç¹ãã£ãŠããŸããã 3ã«è©²åœãããŠãŒã¶ãŒã»ã³ãŒãã£ããŒãã®EmbeddingæŽæ°ã«ã€ããŠã¯ã¬ã³ã¡ã³ãã®ç²ŸåºŠã«ã¯ããã»ã©å€§ãã圱é¿ããªããšèããŸãããæŽæ°å¯Ÿè±¡ã®ããŒã¿æ°ãæžãããã3ã«è©²åœãããŠãŒã¶ãŒã»ã³ãŒãã£ããŒãã¯Embeddingã®æŽæ°å¯Ÿè±¡ãžå«ããªããã倿ŽããŸãããè£è¶³ãšããŠéã¢ã¯ãã£ããªãŠãŒã¶ãŒã»ã³ãŒãã£ããŒãã«ã€ããŠããæ°èŠã«ãŠãŒã¶ãŒè¡åããã£ãå Žåã«ã¯äžèšã®2ã«è©²åœããããã飿¥Nodeãšã®é¢ä¿æ§ãèæ
®ããŠEmbeddingãæŽæ°ããŠããŸãã倿Žã®é©çšåŸãããžãã¹KPIã«æªåã¯èŠããããã¬ã³ã¡ã³ãã®ç²ŸåºŠã«å€§ãã圱é¿ããããšãªãæŽæ°æ°ãäœæžã§ããŸããã åæžååŸã®EmbeddingæŽæ°æ°ã¯æ¬¡ã®éãã§ãã ã³ãŒãã£ããŒã åæžåïŒçŽ3,500,000ä»¶ åæžåŸïŒçŽ150,000ä»¶ ãŠãŒã¶ãŒ åæžåïŒçŽ2,000,000ä»¶ åæžåŸïŒçŽ10,000ä»¶ æ¥ã«ãã£ãп޿°æ°ã¯å€åããŸãããæŽæ°å¯Ÿè±¡ã®ããŒã¿æ°ã倧ããæžããããšãã§ããŸãããã³ãŒãã£ããŒãã«ã€ããŠã¯åæžåã«æ¯ã¹ãŠçŽ23åã®1ããŠãŒã¶ãŒã«ã€ããŠã¯çŽ200åã®1ã®ããŒã¿æŽæ°æ°ãšãªããŸããã æœåºãããŠãŒã¶ãŒã®Embeddingã¯Google Cloudã®ãµãŒãã¬ã¹NoSQLããã¥ã¡ã³ãããŒã¿ããŒã¹ã§ãã Cloud Firestore ã«ä¿åããŠããŸãããæŽæ°å¯Ÿè±¡ã®ãŠãŒã¶ãŒEmbeddingã®ä»¶æ°ãæ³å®ããå€ãã£ãããCloud Firestoreã®æžã蟌ã¿è²»çšã課é¡ãšãªã£ãŠããŸãããæŽæ°å¯Ÿè±¡ã®ããŒã¿æ°ãåæžããããšã«ãããVertex AI Vector SearchãCloud Firestoreã®ããŒã¿æŽæ°è²»çšã¯å¯Ÿå¿ä»¥åãšæ¯èŒããŠ8å²ä»¥äžåæžã§ããŠããŸãã ãŸãIndexæŽæ°å¯Ÿè±¡ã®ããŒã¿æ°ã倧ããæžã£ãããšã§ãIndexã®æŽæ°ã«ãããæéã倧ããççž®ãããŸãããããŒã¿æŽæ°æã«çŽ1ã2æéããã£ãŠããIndexã®æŽæ°æéã¯çŽ40åã«ççž®ã§ããŸãããããŒã¿åŠç察象ã®ä»¶æ°ãæžã£ãããšã§ãã€ãã©ã€ã³å
ã®Indexã®æŽæ°ä»¥å€ã®ã¿ã¹ã¯ã®å®è¡æéãççž®ããããã€ãã©ã€ã³å
šäœã®å®è¡æéã¯çŽååã«ççž®ãããŸããã äžæ¹ã§Indexãžã®Embeddingã®è¿œå ã»æŽæ°æã®æŽæ°æéã¯æŽæ°ããŒã¿åæžå¯Ÿå¿åŸãEmbeddingã®åé€æã®æŽæ°æéãšåçšåºŠãŸã§ççž®ã§ããŸãããçŽ30ã60ä»¶ããæŽæ°å¯Ÿè±¡ã®ããŒã¿æ°ããªãå逿ãšåçšåºŠãŸã§ççž®ã§ããããšãéã¿ãŠãããã«æŽæ°å¯Ÿè±¡ã®ããŒã¿æ°ãæžããããšã«ããæŽæ°æéã®ççž®ã¯å¹æãèããšèããŠããŸãã ãããæŽæ°æéã®æŽãªãççž®ã®ããã«ã¯Indexã® ã¹ããªãŒãã³ã°æŽæ° ã®å©çšå¯èœæ§ããããŸããå
¬åŒããã¥ã¡ã³ãã«ã¯ã¹ããªãŒãã³ã°æŽæ°ãå©çšããããšã§æ°ç§ä»¥å
ã«IndexãæŽæ°ã§ãããšèª¬æããããŸãã äžæ¹ã§æ¢åã®Indexã§ã¯ãããæŽæ°ãå©çšããŠãããããã¹ããªãŒãã³ã°æŽæ°ãå©çšããã«ã¯Indexã®åäœæãå¿
èŠãšãªããŸãããŸãã¹ããªãŒãã³ã°æŽæ°ã§ã¯æŽæ°ãªã¯ãšã¹ãã®å¶éãå²ãåœãŠã®äžéããããããå€§èŠæš¡ãªããŒã¿æŽæ°ã«ã¯åããªããšããæžå¿µããããŸãããã®ãããªçç±ããã¹ããªãŒãã³ã°æŽæ°ã®å©çšã¯çŸæç¹ã§ã¯æ
éã«æ€èšããŠããŸãã Indexã®ããŒã¿æŽæ°è²»çšã®èŠç©ãã Vertex AI Vector Searchãæ¬çªéçšããäžã§ãèªåãèŠç©ããããŠããIndexã®ããŒã¿æŽæ°æã®è²»çšãš Cloud Billing ã®ã³ã³ãœãŒã«ã§ç¢ºèªã§ããè²»çšã«ä¹é¢ããããŸããã åè¿°ã®éãIndexæ§ç¯ã®è²»çšã¯2025幎7æçŸåšã å
¬åŒããã¥ã¡ã³ã ã§æ¬¡ã®ããã«èª¬æãããŠããŸãã ãã¯ãã«æ€çŽ¢ã§ã¯ããã¹ãŠã®ãªãŒãžã§ã³ã«ãããŠãåŠçãããããŒã¿ 1 GiB ããã $3.00 ã課éãããŸãã åœåèªå㯠åŠçãããããŒã¿ ãæŽæ°å¯Ÿè±¡ã®ããŒã¿ãšè§£éããŠããŸãããã€ãŸãIndexæŽæ°ã®ããŒã¿ãœãŒã¹ãšããŠæå®ãããã¡ã€ã«ã«å«ãŸããEmbeddingããŒã¿ã課é察象ãšåéãããŠããŸããã EmbeddingæŽæ°å¯Ÿè±¡ã®ããŒã¿æ°ãåæžããåã®Indexã®æŽæ°æ°ã¯1æ¥ãããçŽ3,500,000ä»¶ã§ããåç¯ïŒ Indexãµã€ãºã®èŠç©ãããšã·ã£ãŒããµã€ãºã®æ±ºå® ïŒã®åŒã§æŽæ°åã®ããŒã¿éãèšç®ãããš1æ¥ãããçŽ3.8GiBã§ããæŽæ°åã®ããŒã¿éã«å¯ŸããŠèª²éãããå Žåã1æ¥ããã$11.4ãè²»çšãšãªãã¯ãã§ãããããåç¯ïŒ Indexã®ããŒã¿æŽæ°è²»çšã®å¢å ïŒã§è¿°ã¹ãéããCloud Billingã§ç¢ºèªããè²»çšã¯1æ¥ãããçŽ$46ã§ãããæŽæ°åã®ããŒã¿éã§ã®èŠç©ããé¡ãããæããã«å€§ããªéé¡ã§ãã åŠçãããããŒã¿ ãæŽæ°å¯Ÿè±¡ã®ããŒã¿ã ãã§ãªããæ¢åã®Indexã«å«ãŸããããŒã¿ãå«ããããŒã¿å
šäœãæããšèããå Žåãå®éã®è²»çšãšè¿ãéé¡ãç®åºã§ããããšã«æ°ãã€ããŸãããåæžå¯Ÿå¿åã§ã¯ãããŒã¿æŽæ°æ°ãšIndexã®ããŒã¿ç·æ°ãåãããŠçŽ1,400äžä»¶ã§ãããããŒã¿éã¯çŽ15.35GiBã§ãããè²»çšãç®åºãããšçŽ$46ãšãªãããCloud Billingã§ç¢ºèªããè²»çšã«äžèŽããŸãã ãããIndexããŒã¿å
šäœã課é察象ãšèãããšãæŽæ°å¯Ÿè±¡ã®ããŒã¿æ°ãæžããããšã§Indexã®æŽæ°ã«äŒŽãIndexæ§ç¯ã®è²»çšã倧ããåæžã§ããããšã®èª¬æãã€ããŸãããããã§Indexã®å§çž®ã«ããåæ§ç¯ã§åŠçãããããŒã¿éãè²»çšã«é¢ä¿ããŠãããšæšæž¬ããŸããããããã¢ããããŒãã§ã®Indexã®å§çž®ã«ã€ããŠã¯ å
¬åŒããã¥ã¡ã³ã ã§æ¬¡ã®èª¬æããããŸãã å¢åããŒã¿ã»ãã ãµã€ãºãåºæ¬ããŒã¿ã»ãã ãµã€ãºã® 20% ãè¶
ãããšãã«è¡ãããŸãã åæžåã®Indexã®ããŒã¿ç·æ°ã¯çŽ1,380äžä»¶ã§ãããæŽæ°å¯Ÿè±¡ã®ããŒã¿æ°ã¯çŽ350äžä»¶ã§ããã350äžä»¶ã¯1,380äžä»¶ã®çŽ25ïŒ
ã«çžåœããŸããæŽæ°å¯Ÿè±¡ã®ããŒã¿æ°ã20ïŒ
ãè¶
ããããšã§ãæŽæ°ã®åºŠã«Indexã¯å§çž®ã«ããåæ§ç¯ãããããŒã¿å
šäœãåŠçããããšã§èª²é察象ãšãªã£ããšèããŸãããåæžåŸã®æŽæ°å¯Ÿè±¡ã®ããŒã¿æ°ã¯çŽ150,000ä»¶ã§ããã1,380äžä»¶ã®çŽ1ïŒ
ã«çžåœããŸããæŽæ°å¯Ÿè±¡ã®ããŒã¿æ°ã20ïŒ
ãäžåã£ãããIndexã®å§çž®ã«ããåæ§ç¯ã¯ããããåŠçãããããŒã¿éã倧ããæžã£ãããšã§è²»çšãæžå°ãããšæšæž¬ããŸããã æ¬¡å³ã¯æŽæ°åã®ããŒã¿éãIndexã®ããŒã¿éã®20ïŒ
以äžã®å Žåã«èª²é察象ãšãªãããŒã¿ã®ã€ã¡ãŒãžã§ãã æ¬¡å³ã¯æŽæ°åã®ããŒã¿éãIndexã®ããŒã¿éã®20ïŒ
以äžã®å Žåã«èª²é察象ãšãªãããŒã¿ã®ã€ã¡ãŒãžã§ãããã¡ãã¯å
éšæåãæšæž¬ããŠäœæããã€ã¡ãŒãžã§ããããå®éã®æåãšã¯ç°ãªãå¯èœæ§ããããŸãã Cloud Billingã§ç¢ºèªã§ããåæžå¯Ÿå¿ã®ååŸã§ã®Indexæ§ç¯ã®è²»çšã®å€åã¯æ¬¡ã®éãã§ãã æŽæ°å¯Ÿè±¡ããŒã¿ãåæžãã7æ7æ¥ä»¥éã«è²»çšã倧ããæžã£ãŠãããæŽæ°å¯Ÿè±¡ããŒã¿åæžã®å¹æãåºãŠããããšãããããŸãããããè²»çšã倧ããæžã£ã7æ8æ¥ä»¥éã¯è²»çšãç·åœ¢ã«å¢å ããŠããŸããæŽæ°å¯Ÿè±¡ã®ããŒã¿éã¯ã»ãŒäžå®ã®å€ã§ãããããç·åœ¢ã«å¢å ããã®ã¯äžèªç¶ã ãšæããŸããããŸãå
¬åŒããã¥ã¡ã³ãã«èšèŒã®å
容ãããã®ãããªè²»çšå¢å ã®èª¬æã¯èªã¿åããŸããã§ãããããã§Vertex AI Vector Searchã®å
éšæåãæšæž¬ããŠãæŽæ°å¯Ÿè±¡ã®ããŒã¿æ°ã20ïŒ
ãäžåãå Žåã®è²»çšèŠç©ããã®æ¹æ³ãèããŸããã æ¬ç¯ã®ä»¥äžã®å
容ã¯ãå
¬åŒããã¥ã¡ã³ãã«èšèŒã®ãªãå
å®¹ãæšæž¬ã§è£ãã仮説ç«ãŠãã説æã§ããå
¬åŒããã¥ã¡ã³ãã«èšèŒã®å
容ã»å®éã®è²»çšã»ããŒã¿é以å€ã¯æšæž¬ã§ããã確å®ã§ãªããããæ³šæãã ããã ããŒã¿æŽæ°ã«ãããIndexæ§ç¯è²»çšã®ç®åºã¯æ¬ã·ã¹ãã ã®éçšã®äžã§èŠåŽããç¹ã§ãããåæ§ã®èª²é¡ãæ±ããæ¹ã®åèã«ãªãã°ãšæããèšèŒããŠããŸãã æŽæ°å¯Ÿè±¡ããŒã¿ãåæžããåŸã§è²»çšã倧ããæžã£ãããšããã®åŸã§ç·åœ¢å¢å ããããšãèžãŸããŠãIndexã¯å
éšã§ããŒãã£ã·ã§ãã³ã°ãããŠããæŽæ°æã«ããŒãã£ã·ã§ã³ããŒã¿ãåŠçããŠããå¯èœæ§ãæšæž¬ããŸãããäžè¬çã«å€§éã®ããŒã¿ãæ±ãã·ã¹ãã ã§ã¯ãäžéšã®ããŒã¿æŽæ°ã§ããŒã¿å
šäœãåæ§ç¯ãããããŒãã£ã·ã§ã³ããŒã¿ã®ã¿ãæŽæ°ããããšã§ããã©ãŒãã³ã¹ãåäžãããŸããå
¬åŒããã¥ã¡ã³ãå
ã§ã®èšèŒã¯ãªãããããããŸã§ãæšæž¬ã§ãããVertex AI Vector Searchã§ãIndexãããŒãã£ã·ã§ãã³ã°ããŠããå¯èœæ§ã¯ãããŸããããŒãã£ã·ã§ã³ãIndexå
šäœã®20ïŒ
ãè¶
ãããŸã§ã¯ããŒãã£ã·ã§ã³ããŒã¿ãæŽæ°ããããŒãã£ã·ã§ã³ãIndexå
šäœã®20ïŒ
ãè¶
ããå Žåã¯Indexå
šäœãå§çž®ããããšæšæž¬ããŸããããŸãããŒãã£ã·ã§ã³ããŒã¿ã®æŽæ°æã¯çŽ¯ç©æŽæ°åãšããŒãã£ã·ã§ã³åã®ããŒã¿éã«å¯ŸããŠèª²éããããšæšæž¬ããå ŽåãåæžåŸã®è²»çšã®ç·åœ¢å¢å ã説æã§ããŸãã æ¬¡å³ã¯æŽæ°åã®ããŒã¿éãIndexã®ããŒã¿éã®20ïŒ
以äžãã€ããŒãã£ã·ã§ã³ããŒã¿ãæŽæ°ããå Žåã«èª²é察象ãšãªãããŒã¿ã®ã€ã¡ãŒãžã§ãããã¡ããå
éšæåãæšæž¬ããŠäœæããã€ã¡ãŒãžã§ããããå®éã®æåãšã¯ç°ãªãå¯èœæ§ããããŸãã 以äžã®æšæž¬ã«åºã¥ããŠãæŽæ°å¯Ÿè±¡ã®ããŒã¿æ°ãIndexå
šäœã®20ïŒ
以äžã§ããå Žåã®Indexã®ããŒã¿æŽæ°è²»çšã¯æ¬¡ã®åŒã§èšç®ã§ãããšèããŸãããæ¬ã·ã¹ãã ã®Indexæ§ç¯è²»çšã®ç®åºã«æ¬¡åŒãå©çšããŠããŸãã ããŒã¿ãµã€ãºïŒBytesïŒ = (ããŒãã£ã·ã§ã³ã®çŽ¯ç©æŽæ°Embeddingæ°ïŒéè€ããïŒ + æŽæ°åã®ããŒãã£ã·ã§ã³ã®Embeddingæ°ïŒéè€ãªãïŒ) à (次å
æ° Ã 1次å
ãããã®ãµã€ãº) + Restrictsã®ãµã€ãºïŒœ å
¬åŒããã¥ã¡ã³ãã«èšèŒã®è²»çšãšCloud Billingã§ç¢ºèªã§ããè²»çšä»¥å€ã®èŠçŽ ã¯æšæž¬ã§ãããããäžåŒã§ã®æ£ç¢ºãªè²»çšã®ç®åºã¯ã§ããŠããŸãããå®éã®ãšãããéå»ã®ããŒã¿ãçšããŠäžåŒã§èšç®ããè²»çšãšCloud Billingã§ç¢ºèªã§ããè²»çšã«ã¯10ïŒ
ååŸã®ãºã¬ãåºãŠããŸãããããå
šäœãšããŠã®åŸåã¯æããããŠãããåã«æŽæ°åã®ããŒã¿éãIndexå
šäœã®ããŒã¿éã åŠçãããããŒã¿ ãšããŠèŠç©ããããããå®éã®è²»çšãšè¿ãå€ãèšç®ã§ããŠããŸãã è²»çšã®åœ¢æ
ã課éã®ä»çµã¿ã¯ãµãŒãã¹å
éšã®ä»æ§ã«äŸåãããããè²»çšã®åŸåãå€ããå¯èœæ§ã¯ãããŸãããã®ããä»åŸãVertex AI Vector Searchã®Indexæ§ç¯ã®è²»çšã¯ç¶ç¶çã«ç¢ºèªããèŠç©ãããã倧ãããããããšãããã°èŠç©ããæ¹æ³ã®èŠçŽããèããŠããŸãã ãŸãšã æ¬èšäºã§ã¯WEARé¢é£ã³ãŒãã¬ã³ã¡ã³ããããžã§ã¯ããžã®Vertex AI Vector Searchå°å
¥ãšå®è·µã«ã€ããŠç޹ä»ããŸãããå®ãããžã§ã¯ãã§ã®å®è£
ã»éçšçµéšãéããŠåŸãããç¥èŠããããããVertex AI Vector Searchã®å°å
¥ãæ€èšããŠããæ¹ã®åèã«ãªãã°å¹žãã§ãã ä»åŸã¯Vertex AI Vector Searchã®å©çšå®çžŸãå¢ãããããå€§èŠæš¡ãªã·ã¹ãã ã§ã®å©çšçµéšãç©ãã§ãããããšèããŠããŸãããŸããVertex AI Vector Searchã®æ°æ©èœãæ¹åç¹ã«ã€ããŠãåŒãç¶ã泚ç®ããŠãããŸãã ZOZOã§ã¯ãäžç·ã«ãµãŒãã¹ãäœãäžããŠãããæ¹ãåéäžã§ããMLOpsãããã¯ã§ãçµ¶è³æ¡çšãè¡ã£ãŠããããããèå³ããæ¹ã¯ä»¥äžã®ãªã³ã¯ãããã²ãå¿åãã ããã corp.zozo.com