ããã«ã¡ã¯ãã¿ã€ããŒã®ããŒã¿çµ±æ¬éšããŒã¿ãµã€ãšã³ã¹ïŒä»¥äžDSïŒã°ã«ãŒãæå±ã®èå°ã§ãã ä»åã¯ãã¿ã€ããŒãBIããŒã«ãšããŠå°å
¥ããŠããLookerã§ã®ã H3 ã䜿çšããå¯èŠåãããããã®åãçµã¿ã玹ä»ããããšæããŸãïŒ H3ãšã¯ H3 ãšã¯ãUber瀟ãéçºããŠããã°ãªããã·ã¹ãã ã§ããªãŒãã³ãœãŒã¹ãšããŠæäŸãããŠããŸãã H3 ã§ã¯ãäœçœ®æ
å ±ã«çŽã¥ããã€ãã³ããéå±€çãªå
è§åœ¢ã®é åã«ãã±ããåããããšãã§ãããã±ããåãããåäœã§ããŒã¿ã®éèšãå¯èœã«ãªããŸãã ã¿ã€ããŒã§ã¯ããµãŒãã¹ãæäŸããåéœåžã®éçµŠãæž¬å®ããããã«ïŒå
è§åœ¢åäœã§éèšããããŒã¿ãå¯èŠåãããªã©ãæ§ã
ãªå Žé¢ã§ã®åæã«æŽ»çšããŠãããäŸãã°ä»¥äžã®ãããªå¯èŠåãè¡ãªã£ãŠãããŸãïŒæ°å€ã¯ã©ã³ãã å€ã§ãïŒã H3 ã«ã€ããŠã®è©³çްã¯ã以äžã®ããŒãžãåèã«ãªãããšæããŸãã https://h3geo.org/ https://github.com/uber/h3 https://www.uber.com/en-JP/blog/h3/ åææ¡ä»¶ Lookerã®ããŒã¿ãœãŒã¹ã¯Google Cloud BigQueryãšããŸãã å¯èŠåã«å¿
èŠãªåçš®ãã¡ã€ã«ã®çæã«ã¯PythonãçšããŠããã䜿çšããŠããPythonããŒãžã§ã³ãšãäŸåã©ã€ãã©ãªã®ããŒãžã§ã³ã«ã€ããŠã¯ã以äžã§åäœç¢ºèªãè¡ã£ãŠããŸãã PythonããŒãžã§ã³: 3.11.4 äŸåã©ã€ãã©ãª geojson==3.0.1 h3==3.7.6 numpy==1.25.2 pandas==2.0.3 shapely==2.0.1 topojson==1.5 ãŸãã倧å€ç°¡ç¥åããäŸã§ãããããŒã¿ãœãŒã¹ã®BigQueryãããžã§ã¯ãã»ããŒã¿ã»ããã»ããŒãã«ã¯ã以äžã®æ³å®ãšããŸãã ãããžã§ã¯ãå: sample-project ããŒã¿ã»ããå: sample-dataset ããŒãã« sales : 売äžããŒã¿ãä¿æããŠããããŒãã« sales_at : 売äžã®æ¥æ amount : å£²äž place_id : places ããŒãã«ã®å€éšã㌠places : äœçœ®æ
å ±ïŒç·¯åºŠã»çµåºŠïŒãä¿æããŠããããŒãã« places ããŒãã«ãš sales ããŒãã«ã«ã¯1:nã®ãªã¬ãŒã·ã§ã³ãååš erDiagram places ||--|{ sales: "1:n" places { INTEGER id FLOAT latitude FLOAT longitude } sales { INTEGER id DATETIME sales_at INTEGER amount ããããINTEGER place_id } Lookerã§ã®å¯èŠåãè¡ãããã®æé ä»åã¯ä»¥äžã®æé ã«åŸã£ãŠãäžèšsaleããŒãã«ã®å£²äžãH3å
è§åœ¢ã«ãã±ããåããLookeräžã§å¯èŠåããŸãã 緯床çµåºŠæ
å ±ãä¿æããŠããBigQueryããŒãã«ã«H3å
è§åœ¢IDãä»äžããå¥ããŒãã«ãšããŠä¿å TopoJsonãã¡ã€ã«ã®äœæ äœæããTopoJsonãã¡ã€ã«ãLookerã«è¿œå Lookerã®modelãã¡ã€ã«ã«map_viewãã£ãŒã«ãã远å Lookerã®viewãã¡ã€ã«ã«dimensionã远å Lookerã®modelãã¡ã€ã«ã«exploreã远å Lookerã§ã®å¯èŠå 1. 緯床çµåºŠæ
å ±ãä¿æããŠããBigQueryããŒãã«ã«H3å
è§åœ¢IDãä»äžããå¥ããŒãã«ãšããŠä¿å éèšã®éã«äœ¿çšããã緯床çµåºŠæ
å ±ãä¿æããŠããBigQueryããŒãã«ãïŒããã§ã¯ places ããŒãã«ïŒã«å¯ŸããŠãH3å
è§åœ¢IDïŒä»¥äžH3 hex idãšèšèŒïŒãä»äžããå¥ããŒãã«ãšããŠä¿åããŠãããŸããããã§ã¯ h3_places ããŒãã«ãšããŠä¿åããŠããŸãã äžèšã¯ã places ããŒãã«ãpandas.DataFrameãšããŠèªã¿èŸŒã¿ãH3å
è§åœ¢è§£å床0~15ãŸã§ã®H3 hex idãä»äžããããŒãã«ãšããŠæžãåºãã³ãŒãã®äŸã§ãã H3å
è§åœ¢è§£å床ã¯å€ã倧ãããªãã«ã€ããŠãå°ããªå
è§åœ¢ïŒ=è§£å床ãäžããïŒã«ãªãã詳现ã«ã€ããŠã¯äžèšããã¥ã¡ã³ããåèã«ãªãããšæããŸãã https://h3geo.org/docs/core-library/restable/ import h3 import pandas as pd class BigQueryClient : def __init__ (self): ... def read_table_as_dataframe (self, table_id: str ) -> pd.DataFrame: """BigQueryããŒãã«ãpandas.DataFrameãšããŠèªã¿èŸŒãåŠç""" ... def write_table_from_dataframe (self, df: pd.DataFrame, table_id: str ) -> None : """pandas.DataFrameãBigQueryããŒãã«ãæžã蟌ãåŠç""" ... def make_h3_hex_ids (df: pd.DataFrame) -> pd.DataFrame: _df = df.copy() for resolution in range ( 16 ): # 緯床ã»çµåºŠæ
å ±ãå
ã«ãH3 hex idãä»äž _df[f 'h3_hex_id_res_{resolution}' ] = df.apply( lambda x: h3.geo_to_h3(x[ 'latitude' ], x[ 'longitude' ], resolution), axis= 1 ) return _df if __name__ == '__main__' : ... bq_client = BigQueryClient() # 緯床ïŒlatitudeïŒãçµåºŠïŒlongitudeïŒãä¿æããŠããBigQueryããŒãã«ãDataFrameãšããŠèªã¿èŸŒã df = bq_client.read_table_as_dataframe( 'sample-project.sample-dataset.places' ) # H3 hex idãä»äžãã h3_df = make_h3_hex_ids(df) h3_df.rename(columns= dict ( id = 'place_id' ), inplace= True ) # H3 hex idãä»äžããDataframeãBigQueryããŒãã«ãšããŠæžã蟌㿠bq_client.write_table_from_dataframe(df=h3_df, 'sample-project.sample-dataset.h3_places' ) äŸãšããŠã以äžã®ãããªç·¯åºŠçµåºŠãä¿æããŠãããµã³ãã«ããŒã¿ã«ãH3 hex idãä»äžããå Žåã以äžã®ãããªçµæã«ãªããŸãã import numpy as np np.random.seed( 42 ) tokyo_latitude = 35.6762 tokyo_longitude = 139.6503 df = pd.DataFrame( [[i, np.random.normal(tokyo_latitude, 0.3 ), np.random.normal(tokyo_longitude, 0.3 )] for i in range ( 1 , 11 )], columns=[ 'place_id' , 'latitude' , 'longitude' ] ) h3_df = make_h3_hex_ids(df) h3_df.head( 10 ) 2. TopoJsonãã¡ã€ã«ã®äœæ ã1. 緯床çµåºŠæ
å ±ãä¿æããŠããBigQueryããŒãã«ã«H3 hex idãä»äžãããã§H3 hex idãä»äžããDataFrameãå
ã«ãTopoJsonãã¡ã€ã«ãäœæããŸãã TopoJsonã®è©³çްã«ã€ããŠã¯ãã¡ãã®ã topojson ãGitHubãªããžããªãåç
§ããŠãã ããã äžèšã¯ãTopoJsonãã¡ã€ã«ãäœæããã³ãŒãäŸã§ãã åŠçã®å
容ãšããŠã¯ãGeoJson圢åŒãçµç±ããŠãTopoJsonã«å€æãããã¡ã€ã«ãšããŠåºåãããŠããŸãã TopoJsonãã¡ã€ã«ã¯ã H3è§£å床 å¥ã«äœæããŠããŸãã from pathlib import Path import geojson import h3 import pandas as pd from shapely import geometry import topojson class H3ToGeojson : @ staticmethod def get_h3_geojson_features (h3_hex_ids: list [ str ]) -> list [geojson.Feature]: polygons = h3.h3_set_to_multi_polygon(h3_hex_ids, geo_json= True ) features = [geojson.Feature(geometry=geometry.Polygon(polygon[ 0 ]), properties= dict (h3_hex_id=h3_hex_id)) for polygon, h3_hex_id in zip (polygons, h3_hex_ids)] return features def get_h3_geojson_feature_collection_from_dataframe (self, df: pd.DataFrame, h3_hex_id_column: str ) -> geojson.FeatureCollection: assert df.columns.isin([h3_hex_id_column]).any(), f 'column `{h3_hex_id_column}` is not exists.' unique_h3_hex_ids = df[h3_hex_id_column].unique().tolist() geojson_features = self.get_h3_geojson_features(unique_h3_hex_ids) feature_collection = geojson.FeatureCollection(geojson_features) return feature_collection class H3ToTopojson : def __init__ (self): self.h3_to_geojson = H3ToGeojson() def get_h3_topojson_topology_from_dataframe (self, df: pd.DataFrame, h3_hex_id_column: str ) -> topojson.Topology: feature_collection = self.h3_to_geojson.get_h3_geojson_feature_collection_from_dataframe( df, h3_hex_id_column=h3_hex_id_column ) return topojson.Topology(feature_collection, prequantize= False ) def make_h3_topojson_file_from_dataframe (self, df: pd.DataFrame, h3_hex_id_column: str , save_file_path: Path) -> None : topojson_topology = self.get_h3_topojson_topology_from_dataframe(df=df, h3_hex_id_column=h3_hex_id_column) topojson_topology.to_json(save_file_path) if __name__ == '__main__' : ... h3_to_topojson = H3ToTopojson() save_dir = Path( 'topojson' ) save_dir.mkdir(exist_ok= True ) for resolution in range ( 0 , 16 ): h3_hex_id_column = f 'h3_hex_id_res_{resolution}' h3_to_topojson.make_h3_topojson_file_from_dataframe(df=h3_df, h3_hex_id_column=h3_hex_id_column, save_file_path=save_dir / f '{h3_hex_id_column}.json' ) äŸãšããŠãå
ã»ã©äœæãããµã³ãã«ããŒã¿ã«å¯ŸããŠã resolution=4 ãæå®ããŠTopoJsonãã¡ã€ã«ãšããŠæžãåºãåŠçã¯ä»¥äžã®ããã«ãªããŸãã h3_to_topojson = H3ToTopojson() h3_to_topojson.make_h3_topojson_file_from_dataframe(h3_df, resolution= 4 ) TopoJsonãã¡ã€ã«ã®äžèº«ã¯ä»¥äžã®ããã«ãªããŸãã { " type ":" Topology "," objects ": { " data ": { " geometries ": [{ " properties ": { " h3_hex_id ":" 842f5a3ffffffff " } ," type ":" Polygon "," arcs ": [[ -5 , -2 , 0 ]] ," id ":" feature_0 " } , { " properties ": { " h3_hex_id ":" 842f5bdffffffff " } ," type ":" Polygon "," arcs ": [[ 1 , -4 , 2 ]] ," id ":" feature_1 " } , { " properties ": { " h3_hex_id ":" 842f5abffffffff " } ," type ":" Polygon "," arcs ": [[ 3 , 4 , 5 ]] ," id ":" feature_2 " }] ," type ":" GeometryCollection " }} ," bbox ": [ 139.198358 , 35.267135 , 140.126313 , 36.103519 ] ," arcs ": [[[ 139.44526 , 35.765969 ] , [ 139.458427 , 36.000295 ] , [ 139.695196 , 36.103519 ] , [ 139.918545 , 35.971536 ] , [ 139.903854 , 35.7366 ]] , [[ 139.44526 , 35.765969 ] , [ 139.667342 , 35.634256 ]] , [[ 139.653549 , 35.399825 ] , [ 139.419179 , 35.297723 ] , [ 139.198358 , 35.429167 ] , [ 139.21065 , 35.662982 ] , [ 139.44526 , 35.765969 ]] , [[ 139.653549 , 35.399825 ] , [ 139.667342 , 35.634256 ]] , [[ 139.667342 , 35.634256 ] , [ 139.903854 , 35.7366 ]] , [[ 139.903854 , 35.7366 ] , [ 140.126313 , 35.603627 ] , [ 140.111006 , 35.368594 ] , [ 139.874758 , 35.267135 ] , [ 139.653549 , 35.399825 ]]]} 3. äœæããTopoJsonãã¡ã€ã«ãLookerã«è¿œå Lookerã®FileBrowserãéããŠãå
ã»ã©äœæããTopoJsonãã¡ã€ã«ã远å ããŸãã 远å åŸãé©åãªãã©ã«ãã«ãã¡ã€ã«ãç§»åããŸããããã§ã¯ maps/h3 ãã©ã«ãã«TopoJsonãã¡ã€ã«ãç§»åããŸãã âââ maps   âââ h3   âââ h3_hex_id_res_0.topojson   âââ h3_hex_id_res_1.topojson   âââ h3_hex_id_res_2.topojson ...   âââ h3_hex_id_res_15.topojson 4. Lookerã®modelãã¡ã€ã«ã«map_viewãã£ãŒã«ãã远å äžèšã®ããã«map_layerãèšå®ããŸããmap_layer㯠H3è§£å床 å¥ã«èšå®ããŠããŸãã property_key ã¯ã2. TopoJsonãã¡ã€ã«ã®äœæãã§äœ¿çšããŠãã H3ToGeojson.get_h3_geojson_features ã¡ãœããå
ã® geojson.Feature ã®åŒæ°ã§èšå®ããŠãã properties ã®keyåã§ãã h3_hex_id ãæå®ããŠããŸãã map_layer: h3_hex_id_res_0 { file : "/ maps / h3 /h3_hex_id_res_0. topojson " format : topojson property_key: " h3_hex_id " } map_layer: h3_hex_id_res_1 { file : "/ maps / h3 /h3_hex_id_res_1. topojson " format : topojson property_key: " h3_hex_id " } map_layer: h3_hex_id_res_2 { file : "/ maps / h3 /h3_hex_id_res_2. topojson " format : topojson property_key: " h3_hex_id " } ... map_layer: h3_hex_id_res_15 { file : "/ maps / h3 /h3_hex_id_res_15. topojson " format : topojson property_key: " h3_hex_id " } 5 . Lookerã®viewãã¡ã€ã«ã«dimensionã远å map_layer_nameã¯ãã4. Lookerã®modelãã¡ã€ã«ã«map_viewãã£ãŒã«ãã远èšãã§äœæãããmap_layeråãæå®ããŸãã dimension㯠H3è§£å床 å¥ã«èšå®ããŠããŸãã view : h3_places { sql_table_name: ` sample - project . sample - dataset .h3_places` ;; dimension : h3_hex_id_res_0 { group_label: " H3 " group_item_label: " H3 è§£å床0ã®å
è§åœ¢ ID " type : string sql : $ { TABLE } .h3_hex_id_res_0 ;; map_layer_name: h3_hex_id_res_0 } dimension : h3_hex_id_res_1 { group_label: " H3 " group_item_label: " H3 è§£å床1ã®å
è§åœ¢ ID " type : string sql : $ { TABLE } .h3_hex_id_res_1 ;; map_layer_name: h3_hex_id_res_1 } dimension : h3_hex_id_res_2 { group_label: " H3 " group_item_label: " H3 è§£å床2ã®å
è§åœ¢ ID " type : string sql : $ { TABLE } .h3_hex_id_res_2 ;; map_layer_name: h3_hex_id_res_2 } ... dimension : h3_hex_id_res_15 { group_label: " H3 " group_item_label: " H3 è§£å床15ã®å
è§åœ¢ ID " type : string sql : $ { TABLE } .h3_hex_id_res_15 ;; map_layer_name: h3_hex_id_res_15 } } 6 Lookerã®modelãã¡ã€ã«ã«exploreã远å äžèšã®ããã«exploreã远å ããŸãã explore : sales { label : " sales " ... join : h3_places { view_label: " place " type : inner sql_on: $ { sales .place_id } = $ { h3_places.place_id } ;; relationship : many_to_one } } 7. Lookerã§ã®å¯èŠå äœæããexploreã§ãããã§ã®å¯èŠåãè¡ããšãå°å³äžã«H3å
è§åœ¢ã¡ãã·ã¥ã衚瀺ãããã¡ãã·ã¥æ¯ã«ãã±ããåãããéèšå€ãè²ã§è¡šçŸããããšãã§ããŸãã äžèšã¯æ±äº¬è¿éã®ããŒã¿ã H3è§£å床 7ã®dimensionã䜿çšããŠå¯èŠåããäŸã§ãïŒæ°å€ã¯ã©ã³ãã å€ã§ãïŒã ä»åäœæãã H3è§£å床 dimensionã倿Žããããšã§ãç®çã«åãããŠå
è§åœ¢ã¡ãã·ã¥ã®å€§ããã倿ŽããŠå¯èŠåãè¡ãããšãå¯èœã§ãã ãŸãšã ä»åã¯ãUber瀟ããªãŒãã³ãœãŒã¹ãšããŠæäŸããŠãã H3 ã䜿çšããŠãLookeräžã§å¯èŠåãè¡ãæ¹æ³ã«ã€ããŠè§£èª¬ããŸããã ã¿ã€ããŒã§ã¯ä»å玹ä»ããLookerã§ã®å¯èŠå以å€ã«ããæ©æ¢°åŠç¿ã®ç¹åŸŽéäœææã«äœ¿çšãããªã©ãæ§ã
ãªå Žé¢ã§ H3 ãæŽ»çšããŠããŸãã ä»åŸãå°çæ
å ±ãæŽ»ãããåæãããéã«æŽ»çšããŠãããããšèããŠããŸãã Weâre Hiring! ã¿ã€ããŒã®ããŒã¿çµ±æ¬éšã§ã¯ããšãã«åãã¡ã³ããŒãåéããŠããŸãïŒïŒ çŸåšåéäžã®ããžã·ã§ã³ã¯ ãã¡ã ã§ãïŒ ã話ãèãããããšæãããæ¹ã¯ãæ¯éäžåºŠã«ãžã¥ã¢ã«é¢è«ã§ã話ãããŸãããïŒ mermaid.initialize({startOnLoad: true});