G-gen ã®äœã
æšã§ããåœèšäºã§ã¯ Google Cloud ã®æ©æ¢°åŠç¿ã¯ãŒã¯ãããŒãªãŒã±ã¹ãã¬ãŒã·ã§ã³ããŒã«ã§ãã Vertex AI Pipelines ã解説ããŸãã MLOps ãš ML ãã€ãã©ã€ã³ã®å¿
èŠæ§ Vertex AI Pipelines ãã€ãã©ã€ã³ã®å®çŸ© 2çš®é¡ã®ã€ã³ã¿ãŒãã§ãŒã¹ Kubeflow Pipelines SDK TensorFlow Extended SDK ãã€ãã©ã€ã³ ã³ã³ããŒãã³ã ã³ã³ããŒãã³ãã®åºæ¬ Google Cloud ãã€ãã©ã€ã³ ã³ã³ããŒãã³ã æŠèŠ Google Cloud ãã€ãã©ã€ã³ ã³ã³ããŒãã³ãã®äŸ ãã€ãã©ã€ã³ã®äŸ ã«ã¹ã¿ã ã³ã³ããŒãã³ã ãã€ãã©ã€ã³ ãã³ãã¬ãŒã ãã€ãã©ã€ã³ã®å®è¡ æåå®è¡ scheduler API ã«ããã¹ã±ãžã¥ãŒã«å®è¡ ã¢ãã«ã®å質äœäžãããªã¬ãŒãšããå®è¡ ãã®ä» Vertex AI ããŒã«ãšã®é£æº æé æéã®åºæ¬ billing ID ã䜿çšããã³ã¹ãåæ MLOps ãš ML ãã€ãã©ã€ã³ã®å¿
èŠæ§ MLOpsïŒMachine Learning OperationsïŒ ãšã¯ããµãŒãã¹ã®ããªããªãŒé床ãä¿¡é Œæ§ã®åäžãé¢ä¿è
éã®ãªãŒããŒã·ããæ§ç¯ãç®çãšãã DevOps ããæ©æ¢°åŠç¿ïŒ ML ïŒã·ã¹ãã ã«é©çšããåãçµã¿ã»ææ³ã§ãã ML ã¢ãã«ã¯ãããŒã¿ã®åéãå å·¥ããã¬ãŒãã³ã°ãªã©ã®éçšãçµãŠæ§ç¯ãããæ¬çªç°å¢ã§ãµãŒãã³ã°ãããŸãããã®éçºãµã€ã¯ã«ã¯éåžžã®ãœãããŠã§ã¢éçºãšç°ãªãã以äžã®ãããªç¹æ§ããããŸãã ML ã¢ãã«ã®éçºã«ã¯ããŒã¿å å·¥ã®ææ³ãã¢ã«ãŽãªãºã éžæããã©ã¡ãŒã¿æ§æãªã©ã®è©Šè¡é¯èª€ã䌎ãå®éšçãªæ§è³ªããããã¢ãã«ã®ãã¬ãŒãã³ã°ãšãã¹ã㯠繰ãè¿ããŠè¡ããã ML ã¢ãã«ãäºæž¬å¯Ÿè±¡ãšããäžçã¯çµ¶ããå€åãç¶ããŠãããã¢ãã«ã®äºæž¬ç²ŸåºŠãç¶æããããã«ã¯ãå€åãã¢ãã¿ãªã³ã°ããå¿
èŠã«å¿ããŠæ°ããããŒã¿ã§ã¢ãã«ã åãã¬ãŒãã³ã° ããå¿
èŠããã ML ãã€ãã©ã€ã³ã®å¿
èŠæ§ ãã®ãããML ã¢ãã«ã®éçºã»éçšã«ãããŠã¯ DevOps ã®èŠçŽ ã§ããã CI/CD ã®ã»ãã«ãMLOps ç¹æã®èŠçŽ ã§ãã ç¶ç¶çãªã¢ãã«ã®ãã¬ãŒãã³ã° ïŒ Continuous Training ãCTïŒãå®çŸããå¿
èŠããããŸãã ML ãã€ãã©ã€ã³ ã¯ãML ã¢ãã«ã®éçºãµã€ã¯ã«ã®åèŠçŽ ãäžé£ã®ã¹ããããšãããCT ãå®çŸããããã® åå©çšå¯èœ ãªãã€ãã©ã€ã³ã§ãã ML ã¢ãã«ã®éçºãµã€ã¯ã«ãå
å
ããåå©çšå¯èœãªãã€ãã©ã€ã³ åè : DevOps åè : MLOps: Continuous delivery and automation pipelines in machine learning Vertex AI Pipelines Vertex AI 㯠Google Cloud ã«ããã ML ãµãŒãã¹ã®çµ±åãã©ãããã©ãŒã ã§ãããML ã¯ãŒã¯ããŒããå±éããããã®æ§ã
ãªããŒã«ãæäŸãããŠããŸãã Vertex AI Pipelines 㯠Vertex AI ã§æäŸãããããŒã«ã®1ã€ã§ãããGoogle Cloud äžã§è€é㪠ML ã¯ãŒã¯ãããŒãå®è¡ãããã€ãã©ã€ã³ãæ§ç¯ã»å®è¡ããããšãã§ããŸãã ML ãã€ãã©ã€ã³ã®å®è¡ã¯ãµãŒããŒã¬ã¹ãªç°å¢ã§å®è¡ãããããããŠãŒã¶ãŒåŽã§ã€ã³ãã©ã¹ãã©ã¯ãã£ã®ç®¡çãã¹ã±ãŒãªã³ã°ã®å®è£
ãããå¿
èŠããããŸããã ãŸããä»ã® Vertex AI ããŒã«ãšã®é£æºã«ããããã€ãã©ã€ã³ã®ã¹ã±ãžã¥ãŒã«å®è¡ãããã€ãã©ã€ã³ã§çæãããã¢ãŒãã£ãã¡ã¯ãïŒå å·¥ããããŒã¿ãã¢ãã«ãã¢ãã«ã®è©äŸ¡ãªã©ïŒã®ç®¡çã»åæãéçºããã¢ãã«ã®ãããã€ãªã©ãã·ãŒã ã¬ã¹ã«è¡ãããšãã§ããŸãã åè : Introduction to Vertex AI Pipelines åè : Architecture for MLOps using TensorFlow Extended, Vertex AI Pipelines, and Cloud Build ãã€ãã©ã€ã³ã®å®çŸ© 2çš®é¡ã®ã€ã³ã¿ãŒãã§ãŒã¹ Vertex AI Pipelines ã§ã¯ã DAG ïŒæåéå·¡åã°ã©ãïŒãšããŠã¯ãŒã¯ãããŒãèšè¿°ããYAML 圢åŒã«ã³ã³ãã€ã«ããããã¡ã€ã«ã䜿çšããããšã§ãå®çŸ©ããã ML ãã€ãã©ã€ã³ãå®è¡ããŸãã Vertex AI Pipelines ã«ãã ML ãã€ãã©ã€ã³ã®ã°ã©ãïŒå
¬åŒããã¥ã¡ã³ãããïŒ ãã®ãã€ãã©ã€ã³ãå®çŸ©ããããã® Python SDK ã€ã³ã¿ãŒãã§ãŒã¹ãšããŠã Kubeflow Pipelines SDK ïŒKFPïŒ ãš TensorFlow Extended SDK ïŒTFXïŒã®2çš®é¡ãæäŸãããŠããŸãã TensorFlow Extended SDK ã¯ãã©ãã€ãåäœã®å€§èŠæš¡ãªæ§é åããŒã¿ãŸãã¯ããã¹ãããŒã¿ãåŠçããéã«æšå¥šãããŠããããã以å€ã®ãŠãŒã¹ã±ãŒã¹ã§ã¯ã·ã³ãã«ãªèšè¿°ã§æè»ãªã¿ã¹ã¯ãåŠçã§ãã Kubeflow Pipelines SDK ã®å©çšãæšå¥šãããŠããŸãã åè : Interfaces for Vertex AI Pipelines Kubeflow Pipelines SDK Kubeflow Pipelines SDK ã¯ãä»»æã®ã³ã³ããã€ã¡ãŒãžã颿°ããã€ãã©ã€ã³ã®èŠçŽ ïŒ ã³ã³ããŒãã³ã ïŒãšããŠå®çŸ©ããããšãã§ããã€ã³ã¿ãŒãã§ãŒã¹ã§ãã TensorFlow ã PyTorchãscikit-learn ãªã©ãæ§ã
㪠ML ãã¬ãŒã ã¯ãŒã¯ã䜿çšããç¬èªã®ã³ã³ããŒãã³ãïŒã«ã¹ã¿ã ã³ã³ããŒãã³ãïŒãéçºã管çããããšãã§ããŸãã ãŸããGoogle Cloud ãã€ãã©ã€ã³ ã³ã³ããŒãã³ããšããŠãä»ã® Google Cloud ãµãŒãã¹ãå©çšããäºåå®çŸ©æžã¿ã³ã³ããŒãã³ãããã€ãã©ã€ã³å®çŸ©ã«çµã¿èŸŒãããšãã§ããŸãã åè : Kubeflow Pipelines - Overview TensorFlow Extended SDK TensorFlow Extended SDK ã¯ãã¹ã±ãŒã©ãã«ãã€é«ãããã©ãŒãã³ã¹ãå¿
èŠãšããã¿ã¹ã¯åãã® ML ãã€ãã©ã€ã³ãå®çŸ©ããã€ã³ã¿ãŒãã§ãŒã¹ã§ãã TensorFlowãKeras ã䜿çšããéçºã«æé©åãããŠãããTensorFlow Data ValidationïŒããŒã¿æ€èšŒïŒã TensorFlow TransformïŒååŠçïŒãTensorFlow Model AnalysisïŒã¢ãã«è©äŸ¡ïŒãšãã£ããå
žåç㪠ML ã¿ã¹ã¯ãå®è¡ããæšæºã³ã³ããŒãã³ããå©çšããããšãã§ããŸãã åè : The TFX User Guide ãã€ãã©ã€ã³ ã³ã³ããŒãã³ã ã³ã³ããŒãã³ãã®åºæ¬ Vertex AI Pipelines ã§ã¯ãããŒã¿ã®æœåºãå å·¥ãã¢ãã«ã®ãã¬ãŒãã³ã°ããããã€ãªã©ã® ML ã¯ãŒã¯ãããŒã®åã¹ãããã ã³ã³ããŒãã³ã ãšããŠå®çŸ©ãããããçµã¿åãããããšã§ãã€ãã©ã€ã³ãæ§æããŸãã ã³ã³ããŒãã³ããçµã¿åãããŠãã€ãã©ã€ã³ãå®çŸ©ãã ã³ã³ããŒãã³ãã¯ããŒã¿ã»ãããã¢ãã«ããã€ããŒãã©ã¡ãŒã¿ãªã©ã®å
¥åãåãåã£ãŠåŠçãè¡ãããã®åºåãæ¬¡ã®ã³ã³ããŒãã³ãã®å
¥åãšããŠæž¡ãããšãã§ããŸãã Google Cloud ãã€ãã©ã€ã³ ã³ã³ããŒãã³ã æŠèŠ Google Cloud ãã€ãã©ã€ã³ ã³ã³ããŒãã³ã㯠Google ã«ãã£ãŠäºåå®çŸ©ããããã«ãæžã¿ Kubeflow Pipelines ã³ã³ããŒãã³ãã§ãããVertex AI ã®åçš®ããŒã«ã BigQueryãDataflow ãšãã£ãä»ã® Google Cloud ãµãŒãã¹ã®æäœããã€ãã©ã€ã³ã«çµã¿èŸŒãããšãã§ããŸãã åè : Introduction to Google Cloud Pipeline Components Google Cloud ãã€ãã©ã€ã³ ã³ã³ããŒãã³ãã®äŸ 以äžã¯ãGoogle Cloud ãã€ãã©ã€ã³ ã³ã³ããŒãã³ããšããŠæäŸãããŠããäºåå®çŸ©ã³ã³ããŒãã³ãã®äžäŸã§ãã ã³ã³ããŒãã³ã 説æ BigqueryQueryJobOp BigQuery ã§ã¯ãšãªãå®è¡ããã TabularDatasetCreateOp Cloud Storage ã BigQuery ãããŒã¿ãœãŒã¹ãšããŠãVertex AI ã§äœ¿çšã§ãã衚圢åŒããŒã¿ã»ãããäœæããã AutoMLTabularTrainingJobRunOp AutoML ã§è¡šåœ¢åŒããŒã¿ã»ããã䜿çšãããã¬ãŒãã³ã° ãžã§ããå®è¡ããã BigqueryCreateModelJobOp BigQuery ML ã«ããã¢ãã«ã®äœæãžã§ããå®è¡ããã CustomTrainingJobOp Vertex AI ã®ã«ã¹ã¿ã ãã¬ãŒãã³ã° ãžã§ããå®è¡ããã DataflowPythonJobOp Apache Beam Python SDK ã§èšè¿°ããããžã§ãã Dataflow ã§å®è¡ããã WaitGcpResourcesOp Google Cloud ãªãœãŒã¹ã®å®è¡ãå®äºãããŸã§åŸ
æ©ããïŒ2025幎5æçŸåšãDataflow ãžã§ãã«å¯ŸããŠã®ã¿æå¹ïŒã DataprocSparkBatchOp Apache Spark ã®ããããžã§ãã Dataproc ã§å®è¡ããã ModelUploadOp ã¢ãã«ã Vertex AI Model Registry ã«ã¢ããããŒãããã EndpointCreateOp Vertex AI Endpoints ã®ãšã³ããã€ã³ããäœæããã ModelDeployOp Vertex AI Endpoints ã®ãšã³ããã€ã³ãã«ã¢ãã«ããããã€ããã ModelBatchPredictOp æå®ããã¢ãã«ã䜿çšããŠãVertex AI Batch Predictions ã«ãããããäºæž¬ãè¡ãã åè : Google Cloud Pipeline Components list ãã€ãã©ã€ã³ã®äŸ ãã€ãã©ã€ã³å®çŸ©ã¯ Jupyter Notebook çã®éçºç°å¢ã䜿çšããŠèšè¿°ããŸãã 以äžã¯ãGoogle Cloud ãã€ãã©ã€ã³ ã³ã³ããŒãã³ãã䜿çšãã Kubeflow Pipelines SDK ã«ãã ML ãã€ãã©ã€ã³ã®äŸã§ãã ãã®ãã€ãã©ã€ã³ã§ã¯ãå
¬éãããŠããããŒã¿ãœãŒã¹ããç»åããŒã¿ã»ãããäœæããAutoML ã§ç»ååé¡ã¢ãã«ã®ãã¬ãŒãã³ã°ãè¡ã£ãããšãäœæããã¢ãã«ã Vertex AI Endpoints ã®ãšã³ããã€ã³ãã«ãããã€ããäžé£ã®ã¹ããããå®çŸ©ãããŠããŸãã # Kubeflow Pipelines SDK, Vertex AI Python SDK ã®ã€ã³ããŒã import kfp from google.cloud import aiplatform # Google Cloud ãã€ãã©ã€ã³ ã³ã³ããŒãã³ãã®ã€ã³ããŒã from google_cloud_pipeline_components.v1.dataset import ImageDatasetCreateOp from google_cloud_pipeline_components.v1.automl.training_job import AutoMLImageTrainingJobRunOp from google_cloud_pipeline_components.v1.endpoint import EndpointCreateOp, ModelDeployOp project_id = "myproject" pipeline_root_path = "gs://mybucket/pipeline-root/path/" # ãã€ãã©ã€ã³å®çŸ©ã®äŸ @ kfp.dsl.pipeline ( name= "automl-image-training-v2" , pipeline_root=pipeline_root_path) def pipeline (project_id: str ): # ããŒã¿ã»ãããäœæããã³ã³ããŒãã³ã ds_op = ImageDatasetCreateOp( project=project_id, display_name= "flowers" , gcs_source= "gs://cloud-samples-data/vision/automl_classification/flowers/all_data_v2.csv" , import_schema_uri=aiplatform.schema.dataset.ioformat.image.single_label_classification, ) # AutoML ã«ãããã¬ãŒãã³ã°ãå®è¡ããã³ã³ããŒãã³ã training_job_run_op = AutoMLImageTrainingJobRunOp( project=project_id, display_name= "train-iris-automl-mbsdk-1" , prediction_type= "classification" , model_type= "CLOUD" , dataset=ds_op.outputs[ "dataset" ], model_display_name= "iris-classification-model-mbsdk" , training_fraction_split= 0.6 , validation_fraction_split= 0.2 , test_fraction_split= 0.2 , budget_milli_node_hours= 8000 , ) # Vertex AI Endpoints ã®ãšã³ããã€ã³ããäœæããã³ã³ããŒãã³ã create_endpoint_op = EndpointCreateOp( project=project_id, display_name = "create-endpoint" , ) # ãšã³ããã€ã³ãã«ã¢ãã«ããããã€ããã³ã³ããŒãã³ã model_deploy_op = ModelDeployOp( model=training_job_run_op.outputs[ "model" ], endpoint=create_endpoint_op.outputs[ 'endpoint' ], automatic_resources_min_replica_count= 1 , automatic_resources_max_replica_count= 1 , ) ãã¬ãŒãã³ã°ãšã¢ãã«ã®ãããã€ãè¡ããã€ãã©ã€ã³ã®äŸ ãã€ãã©ã€ã³å®çŸ©ã¯ã以äžã®ããã« YAML 圢åŒã«ã³ã³ãã€ã«ããVertex AI Pipelines ã§å®è¡å¯èœãªç¶æ
ã«ããŸãã from kfp import compiler compiler.Compiler().compile( pipeline_func=pipeline, # å®çŸ©ãããã€ãã©ã€ã³ package_path= 'image_classif_pipeline.yaml' ) YAML ãã¡ã€ã«ã«ã¯ãã€ãã©ã€ã³ã®å®è¡ã«å¿
èŠãªæ
å ±ããã¹ãŠå«ãŸããŸãã以äžã¯ãã®äŸã§äœæã§ãã YAML ãã¡ã€ã«ã®äžéšæç²ã§ãã # PIPELINE DEFINITION # Name: automl-image-training-v2 # Inputs: # project_id: str components : comp-automl-image-training-job : executorLabel : exec-automl-image-training-job inputDefinitions : artifacts : base_model : artifactType : schemaTitle : google.VertexModel schemaVersion : 0.0.1 description : Only permitted for Image Classification models. If it is specified, the new model will be trained based on the `base` model. Otherwise, the new model will be trained from scratch. The `base` model must be in the same Project and Location as the new Model to train, and have the same model_type. isOptional : true dataset : artifactType : schemaTitle : google.VertexDataset schemaVersion : 0.0.1 description : The dataset within the same Project from which data will be used to train the Model. The Dataset must use schema compatible with Model being trained, and what is compatible should be described in the used TrainingPipeline's [ training_task_definition ] [ google.cloud.aiplatform.v1beta1.TrainingPipeline.training_task_definition ] . For tabular Datasets, all their data is exported to training, to pick and choose from. # 以äžçç¥ åè : Build a pipeline ã«ã¹ã¿ã ã³ã³ããŒãã³ã Kubeflow Pipelines SDK ã§ã¯ããŠãŒã¶ãŒãèšè¿°ããåŠçã ã«ã¹ã¿ã ã³ã³ããŒãã³ã ãšããŠãã€ãã©ã€ã³ã«çµã¿èŸŒãããšãã§ããŸãã 以äžã®äŸã§ã¯ãå
¥åããã2ã€ã®æ°å€ãå ç®ããã³ã³ããŒãã³ããšãå
¥åãããæ°å€ããã°ã«åºåããã³ã³ããŒãã³ãã®2ã€ãäœæããããããé ã«å®è¡ãããã€ãã©ã€ã³ãå®çŸ©ããŠããŸãã from kfp import dsl pipeline_root_path = "gs://mybucket/pipeline-root/path/" # å
¥åããã2ã€ã®æ°å€ãå ç®ããã«ã¹ã¿ã ã³ã³ããŒãã³ã @ dsl.component ( base_image= 'python:3.12' , # ã³ã³ããŒãã³ããå®è¡ããã³ã³ããã€ã¡ãŒãž packages_to_install=[ 'google-cloud-storage' ] # å¿
èŠã«å¿ããŠã©ã€ãã©ãªã远å ã§ããïŒãã®äŸã§ã¯äœ¿çšããªãïŒ ) def add_numbers ( num1: float , # å
¥åãã©ã¡ãŒã¿1 num2: float # å
¥åãã©ã¡ãŒã¿2 ) -> float : # 2ã€ã®æ°å€ãå ç®ãã print (f "Adding {num1} and {num2}" ) result = num1 + num2 print (f "Result: {result}" ) return result # èšç®çµæãåºåãšããŠè¿ã # å
¥åãããæ°å€ãåºåããã«ã¹ã¿ã ã³ã³ããŒãã³ã @ dsl.component ( base_image= 'python:3.12' ) def print_result ( result_value: float # å
¥åãã©ã¡ãŒã¿ ): # åãåã£ãæ°å€ã衚瀺ãã print (f "The final result is: {result_value}" ) # ãã€ãã©ã€ã³å®çŸ© @ dsl.pipeline ( name= 'custom-component-pipeline-sample' , description= 'A simple pipeline using custom components.' , pipeline_root=pipeline_root_path ) def my_custom_pipeline ( input1: float = 10.5 , # ãã€ãã©ã€ã³ã®å
¥åãã©ã¡ãŒã¿ (ããã©ã«ãå€) input2: float = 5.2 ): # ã¹ããã 1: add_numbers ã³ã³ããŒãã³ããå®è¡ add_task = add_numbers(num1=input1, num2=input2) # ã¹ããã 2: print_result ã³ã³ããŒãã³ããå®è¡ # åã®ã¹ããã (add_task) ã®åºåããæ¬¡ã®ã¹ãããã®å
¥åãšããŠæž¡ã print_task = print_result(result_value=add_task.output) Build your own pipeline components ãã€ãã©ã€ã³ ãã³ãã¬ãŒã Kubeflow Pipelines SDK ã§å®çŸ©ããããã€ãã©ã€ã³ã¯ã ãã€ãã©ã€ã³ ãã³ãã¬ãŒã ãšã㊠Artifact Registry ãªããžããªã«ã¢ããããŒãããããšãã§ããŸãã ã¢ããããŒããããã³ãã¬ãŒãã¯è€æ°ã®ãŠãŒã¶ãŒããã€ãã©ã€ã³ãåå©çšããããšãã§ããŸãã Kubeflow Pipelines ãã³ãã¬ãŒãã管çãããªããžããªãäœæãã 以äžã®ã³ãŒãã§ã¯ãGoogle Cloud ãã€ãã©ã€ã³ ã³ã³ããŒãã³ãã®äŸã§äœæãã YAML ãã¡ã€ã«ãããã€ãã©ã€ã³ ãã³ãã¬ãŒããšããŠã¢ããããŒãããŠããŸãã from kfp.registry import RegistryClient # ãã³ãã¬ãŒãã®ã¢ããããŒãå
ãªããžããªã®æå® client = RegistryClient(host=f "https://asia-northeast1-kfp.pkg.dev/myproject/vertexai-pipelines-template" ) # ãã³ãã¬ãŒãã®ã¢ããããŒã templateName, versionName = client.upload_pipeline( file_name= "image_classif_pipeline.yaml" , tags=[ "v1" , "latest" ], extra_headers={ "description" : "This is an example pipeline template." }) ãªããžããªã«ã¢ããããŒããããã³ãã¬ãŒãã¯ããã€ãã©ã€ã³ã®å®è¡æã«æå®ããããšãã§ããŸãã ãã³ãã¬ãŒãã䜿çšããŠãã€ãã©ã€ã³ãå®è¡ãã ãŸãã ãã³ãã¬ãŒã ã®ã£ã©ãªãŒ ã«ã¯ Google ãäœæãããã€ãã©ã€ã³ ãã³ãã¬ãŒããçšæãããŠãããäžè¬çãªãŠãŒã¹ã±ãŒã¹ã®ãã€ãã©ã€ã³ããã®ãŸãŸå®è¡ããããç¬èªã®ãã€ãã©ã€ã³ã«çµã¿èŸŒãã§å©çšãããã§ããŸãã Google ãäœæãããã€ãã©ã€ã³ ãã³ãã¬ãŒãã䜿çšãã åè : Create, upload, and use a pipeline template åè : Use a prebuilt template from the Template Gallery ãã€ãã©ã€ã³ã®å®è¡ æåå®è¡ ãã€ãã©ã€ã³ã¯ Vertex AI SDK for Python ã Google Cloud ã³ã³ãœãŒã«ã䜿çšããŠå®è¡ããããšãã§ããŸãã 以äžã®ã³ãŒãã¯ãGoogle Cloud ãã€ãã©ã€ã³ ã³ã³ããŒãã³ãã®äŸã§äœæãããã€ãã©ã€ã³ããJupyter Notebook ããå®è¡ããŠããŸãã import google.cloud.aiplatform as aip aip.init( project= "myproject" , location= "asia-northeast1" , ) # å®è¡ãããã€ãã©ã€ã³ã®èšå® job = aip.PipelineJob( display_name= "automl-image-training-v2" , template_path= "image_classif_pipeline.yaml" , # ã³ã³ãã€ã«ãã YAML ãã¡ã€ã« pipeline_root=pipeline_root_path, parameter_values={ 'project_id' : project_id } ) # ãã€ãã©ã€ã³ã®å®è¡ job.submit() ãã€ãã©ã€ã³ã®å®è¡ã¯ Google Cloud ã³ã³ãœãŒã«ããã¢ãã¿ãªã³ã°ããããšãã§ããŸãã ãã€ãã©ã€ã³ã®ã¢ãã¿ãªã³ã°ç»é¢ scheduler API ã«ããã¹ã±ãžã¥ãŒã«å®è¡ scheduler API ã䜿çšããããšã§ãML ãã€ãã©ã€ã³ã®å®è¡ãã¹ã±ãžã¥ãŒãªã³ã°ããããšãã§ããŸãã ããã«ãããã¢ãã«ã®åãã¬ãŒãã³ã°ã宿çã«å®è¡ããããªããã«ããã¢ãã«ã®å質äœäžãªã©ã®åé¡ã«å¯ŸåŠããããšãã§ããŸãã ãã€ãã©ã€ã³ã®ã¹ã±ãžã¥ãŒã«å®è¡ã¯ã Vertex AI SDK for Python ã Google Cloud ã³ã³ãœãŒã«ã䜿çšããŠèšå®ã§ããŸãã ã³ã³ãœãŒã«ãããã€ãã©ã€ã³ã®ã¹ã±ãžã¥ãŒã«å®è¡ãèšå®ãã åè : Schedule a pipeline run with scheduler API ã¢ãã«ã®å質äœäžãããªã¬ãŒãšããå®è¡ Vertex AI Model Monitoring ã䜿çšãããšãVertex AI Endpoints çã§éçšäžã®ã¢ãã«ã«å¯ŸããŠã¢ãã¿ãªã³ã°ãè¡ãããšãã§ããŸãã ã¢ãã¿ãªã³ã°ã®äŸãšããŠãäºæž¬ã®éã«å
¥åãããç¹åŸŽããŒã¿ããäºæž¬ã®çµæãããŒã¹ã©ã€ã³ã®ããŒã¿ååžãšæ¯èŒããååžã®å€åïŒããªããïŒããã£ãå Žåã«ã¢ã©ãŒãéç¥ãè¡ãããšãã§ããŸãã ååžã®å€åãéŸå€ãè¶
ããå ŽåãCloud Logging ã«ãã°ãšããŠåºåãããŸããããã Pub/SubãCloud Run Functions ã«é£æºããããšã§ãããªããã«ããã¢ãã«ã®å質äœäžãããªã¬ãŒãšãããã€ãã©ã€ã³ã®å®è¡ãå®çŸããããšãã§ããŸãã åè : Introduction to Vertex AI Model Monitoring ãã®ä» Vertex AI ããŒã«ãšã®é£æº Vertex AI ã§æäŸãããŠããåçš®ããŒã«ã¯ãVertex AI Pipelines ãšé£æºããML ãã€ãã©ã€ã³ãäžå¿ãšãã MLOps ã®å®çŸã«åœ¹ç«ãŠãããšãã§ããŸãã Vertex AI ã®åããŒã«ã Vertex AI Pipelines ãšã©ã®ããã«é£æºããããã以äžã«ç€ºããŸãã ããŒã« 説æ Vertex AI Feature Store ML ã¢ãã«ã®ãã¬ãŒãã³ã°ãäºæž¬ã§äœ¿çšãã ç¹åŸŽããŒã¿ãäžå
管ç ããããŒã«ã BigQuery ãããŒã¿ãœãŒã¹ãšããç¹åŸŽããŒã¿ã ML ãã€ãã©ã€ã³ããããã€æžã¿ã®ã¢ãã«ã«å¯ŸããŠå¹ççã«ãµãŒãã³ã°ããããšãã§ããã Vertex ML Metadata ML ãã€ãã©ã€ã³ã§å
¥åããããã©ã¡ãŒã¿ãåºåãããã¢ãŒãã£ãã¡ã¯ããææšãªã©ãèšé²ããããšãã§ããã ãã€ãã©ã€ã³ã®å®è¡ããšã®åæããã¢ãŒãã£ãã¡ã¯ãã® ãªããŒãžã®è¿œè·¡ ã®ã»ããèšé²ããããã©ã¡ãŒã¿ã䜿çšããŠãã€ãã©ã€ã³ãåå®è¡ãããå Žåãªã©ã«äœ¿çšã§ããã Vertex AI Experiments ML ã¢ãã«éçºã«ãããå®éšããã»ã¹ãè¿œè·¡ã»æ¯èŒããããã® å®éšç®¡ç ã®ããã®ããŒã«ã 詊è¡é¯èª€çã«å®è¡ããã ML ãã€ãã©ã€ã³ã远跡ã»èšé²ããããšã§ãæé©ãªã¢ãã«ãäœãåºããã€ãã©ã€ã³ãç¹å®ããããã®åæãè¡ãããšãã§ããã Vertex AI TensorBoard å®éšããã»ã¹ã§èšé²ãããåææšã å¯èŠå ããããã®ããŒã«ã ML ã¢ãã«ã®ãã¬ãŒãã³ã°æã«èšé²ãããæå€±ã粟床ãªã©ã®è©äŸ¡ææšããã¢ãã«ã®éã¿ããã€ã¢ã¹ãªã©ãã©ã¡ãŒã¿ã®å€åããªã¢ã«ã¿ã€ã ã«å¯èŠåããããšãã§ããã Vertex AI Model Registry ML ãã€ãã©ã€ã³ã§æ§ç¯ããã¢ãã«ã ããŒãžã§ã³ç®¡ç ãããªããžããªãšããŠäœ¿çšããã ãªããžããªã§ç®¡çãããŠããã¢ãã«ã¯ãããäºæž¬ã«çŽæ¥äœ¿çšããããVertex AI Endpoints ã«ãããã€ãããã§ããã Vertex AI Batch Predictions ãã¬ãŒãã³ã°æžã¿ã®ã¢ãã«ã䜿çšããŠãéåæã® ãããäºæž¬ ãå®è¡ããããšãã§ããã Vertex AI Endpoints ã¢ãã«ããšã³ããã€ã³ãã«ãããã€ãã ãªã³ã©ã€ã³ïŒãªã¢ã«ã¿ã€ã ïŒäºæž¬ ãå®è¡ã§ããããã«ããã ãšã³ããã€ã³ãã¯ãã€ãã©ã€ã³ããçŽæ¥äœæããããšãã§ããã Vertex AI Model Monitoring éçšäžã®ã¢ãã«ã«å¯Ÿããã¢ãã¿ãªã³ã°ãå®è¡ããããŒã«ãã¢ãã«ã®ãã¬ãŒãã³ã°æã®ããŒã¿ååžãããŒã¹ã©ã€ã³ãšããå
¥åãããããŒã¿ã®ååžãäºæž¬çµæã®ååžãç°ãªã ããªãããæ€åº ã§ããã Cloud LoggingãPub/Sub ãšé£æºããããšã§ã¢ãã«ã®å質äœäžã®ã¢ã©ãŒããçºè¡ããããšãã§ããã Vertex AI Pipelines ãšä»ããŒã«ã®é£æºã€ã¡ãŒãž æé æéã®åºæ¬ Vertex AI Pipelines ã§ã¯ãã¡ã€ã³ã®èª²éèŠçŽ ãšããŠã åã³ã³ããŒãã³ããã䜿çšãããªãœãŒã¹ã®äœ¿çšéã«å¿ããæé ãçºçããŸãããŸããML ãã€ãã©ã€ã³ã®å®è¡ããšã« $0.03 ã®å®è¡æéãçºçããŸãã äŸãã°ããã€ãã©ã€ã³ã®ã³ã³ããŒãã³ããšã㊠AutoML ã®ãã¬ãŒãã³ã°ãå®è¡ããå Žå㯠AutoML ã®æéãçºçããŸããVertex AI 以å€ã®ãµãŒãã¹ã§ããBigQuery ã Dataflow ã®ãžã§ããã³ã³ããŒãã³ãããå®è¡ããå Žåã¯ããžã§ãå®è¡ã«äŒŽãæéãçºçããŸãã ãã®ããã«ããã€ãã©ã€ã³ã®å®è¡ã«äŒŽãæé㯠Vertex AI 以å€ã®è€æ°ã®ãµãŒãã¹ã«å¯ŸããŠçºçããå¯èœæ§ããããCloud Billing ãããã€ãã©ã€ã³å
šäœã®æéãææ¡ããã®ãè¥å¹²é£ããããšã«æ³šæãå¿
èŠã§ãã åè : Vertex AI pricing billing ID ã䜿çšããã³ã¹ãåæ Vertex AI Pipelines ã¯ããã€ãã©ã€ã³å®è¡æã«äžæã® billing ID ãçæããŸãããã® ID ãã³ã³ããŒãã³ããã䜿çšããã Google Cloud ãªãœãŒã¹ã®ã©ãã«ãšããŠçŽã¥ããããšãã§ããŸãã ã©ãã«ãä»äžããããªãœãŒã¹ã®å©çšæéã¯ãCloud Billing ã®èª²éã¬ããŒãã§ãã£ã«ã¿ããŠç¢ºèªããããšãã§ããŸãã ãŸããCloud Billing ããè«æ±æ
å ±ã BigQuery ã«ãšã¯ã¹ããŒãããbilling ID ã䜿çšããŠã¯ãšãªãå®è¡ããããšã§ããã€ãã©ã€ã³å®è¡ããšã®æéã確èªããããšãã§ããŸãã ã©ãã«ã¯ãGoogle Cloud ãã€ãã©ã€ã³ ã³ã³ããŒãã³ãããäœæããããªãœãŒã¹ïŒDataflow ãªãœãŒã¹ãé€ãïŒã«å¯ŸããŠèªåã§ä»äžãããŸããDataflow ãªãœãŒã¹ããã«ã¹ã¿ã ã³ã³ããŒãã³ãããäœæããããªãœãŒã¹ã«å¯ŸããŠã¯ãã©ãã«ãçŽã¥ããããã®èšè¿°ãã³ã³ããŒãã³ãå®çŸ©ã«å«ããå¿
èŠããããŸãã åè : Understand pipeline run costs åè : Resource labeling by Vertex AI Pipelines äœã
æš é§¿å€ª (èšäºäžèЧ) G-genæå端ãåæµ·éåšäœã®ã¯ã©ãŠããœãªã¥ãŒã·ã§ã³éšãšã³ãžã㢠2022幎6æã«G-genã«ãžã§ã€ã³ãGoogle Cloud Partner Top Engineer 2025 Fellowã«éžåºã奜ããªGoogle Cloudãããã¯ãã¯Cloud Runã è¶£å³ã¯ã³ãŒããŒãå°èª¬ïŒSFããã¹ããªïŒãã«ã©ãªã±ãªã©ã Follow @sasashun0805