Hi, this is Kageyama, from KINTO Technologies! As is customary every year, I'd like take a moment to reflect on 2024 and share some thoughts on what’s ahead in 2025. Review of 2024 Looking back, the number of challenges we needed to address has increased compared to a year ago. In response, our team grew, and the organization expanded. At the same time, even with that growth, I believe believe we’ve made a conscious effort to preserve the strengths of being an in-house development team. One of the major milestones this year was the full-scale launch of the Dealership DX Project, supporting the digital transformation of Toyota dealerships across Japan. We’ve already rolled out tools aimed at reducing the workload for dealerships for creating estimates, as well as customer management apps. In addition, AI-driven product development has been progressing steadily. As AI adoption and cloud usage advance, the scope of security and governance are also expanding. To that end, we have established a dedicated cloud security unit and began reinforcing security around AI-related projects. Internally, we have also moved steadily incorporating AI. The AI integration with Slack, our primary communication platform, is steadily becoming the norm across the company. There have also been cases where even non-engineering employees have used generative AI to develop an application. We have also started offering generative AI training sessions for our group companies and dealerships. Progress of Key Initiatives In last year's Advent Calendar , I wrote about three key priorities we aimed to improve: Technical proficiency Development productivity Release speed First, on technical proficiency, we have expanded our technichal support for Toyota Motor Corporation and its group companies. We also developed use cases that have been well received by various IT vendors. One highlight has been our Tech Blog which attracted an impressive 120,000 unique users this year. As more people started contributing to it, we have received valuable feedback from both inside and outside the company. Next, in terms of development productivity, we've been using Findy Team+ to quantitatively measure team performance, which has led to initiatives aimed at improving the efficiency of individual processes. The company has also introduced systems that integrate generative AI, such as GitHub Copilot, which has increased employees' awareness around development productivity, sparking new initiatives including efforts to increase overall throughput across the entire development lifecycle, from business planning to release. An finally, regarding of release speed, our main focus has been on streamlining the early stages of development, particularly the process until the requirements definition phase. By having the development teams lead the requirements definition process as well, we’re beginning to steer the business side toward a product development approach that is more feasible and demands less development effort. We are still in the trial-and-error stage, but I would like to further spread this awareness of speed throughout the company. Outlook for 2025 So, what will we focus on next year? I plan to center our efforts around two major "firsts": AI First Release First (Fastest Release) AI First AI first means exactly what it says, and we will be strengthening AI initiatives through: Seamless integration of AI across all products Development of more AI-driven products Leading AI adoption at dealerships and throughout the Toyota Group. Thanks to the efforts of all our employees so far, we are in a strong position to accelerate these efforts in 2025. I am optimistic that we will be able to generate significant output next year. Moving forward, I believe it's important to adopt a mindset of actively incorporating AI in product development; even for products that don't necessarily require AI. I look forward to seeing new ideas and cross-functional collaborations that will help bring our AI-first approach to life. Release First Release First (Fastest Release) is about using our expertise and technology to deliver products as quickly as possible. What it takes to achieve the fastest release can vary depending on the business owner and the nature of the product. This year, our focus was on enhancing productivity in both the requirements definition phase and the implementation process for specific projects and teams. Next year, instead of solely focusing on improving the productivity of specific stages, I’d like each product team to collaborate closely with stakeholders and consider how and where they can shorten the entire release cycle to achieve the fastest possible product releases. I also believe we need to fully embrace the concept of MVP (Minimum Value Product). That doesn’t mean every product will follow the same development or delivery model. Our strength as an in-house development team lies in our ability to thoughtfully assess product value while still aiming for the shortest possible release timelines. This balance is one of our greatest contributions to the business, and we will push even further in this direction next year. User First and Organizational Intensity To support AI First and Release First , we will be reemphasizing two foundational themes in 2025: User First and Organizational Intensity . User First means going beyond serving end users of our services: we also need to proactively engage with our business partners and the customers they serve. By identifying what truly matters to users and customers, we can elevate the value and impact of our work. That is why I see methodologies like user research as essential skills to develop. Intensity is insipired by a Soccer Term . One of KINTO Technologies' key strengths is our team of highly skilled individuals. But to achieve even greater outcomes as an in-house development team, we need to further raise the collective capabilities of our teams and organization. I encourage everyone to reflect on the mindset and actions needed to help KTC become a truly high-intensity organization. Full Throttle While we didn’t explicitly include technical expertise and communication skills in our 2025 themes, these have become deeply embedded in our culture and are now expected. They remain timeless pillars at KTC, and I hope each of us will continue to nurture them and strengthen them. We will continue to give our best next year as well, with no shortcuts. Hitoshi Kageyama, Executive Vice-President, KINTO Technologies

自己紹介 初めまして！ KINTOテクノロジーズのCloud InfrastructureグループでManagerをやってる辻です 2020年5月に入社している地味に古株な人です 新卒からずっとインフラ(オンプレ/クラウド両方)エンジニアしています 概要 こないだVSCodeでのGitHub Copilot Agent Mode(以降Agent)が正式リリースされましたね(2025/4/16現在) ちょうどいいのでVSCode Agentを使って何か便利なもんでも作ってみましょう 前提 私は根っからのインフラエンジニアです フロントエンドどころかバックエンドもほぼワカランです その代わりインフラは(特にAWS)はチョットデキル この記事では深いTechなことには触れません やりたいこと Agentを使ってインフラ以外できないエンジニアがどこまで行けるのか挑戦する 日々の 業務で利用できる実用的なシステムをMVP(Minimum Viable Product) で開発する 絶対にコードは自分で書かない じゃあ何を作ろうか？？ KINTOテクノロジーズのインフラは基本的にすべてIaC(Terraform)で作成/管理されています terraformを使ったことがある人ならわかると思いますが、planの内容って慣れてないと見づらいですよね？ なので今回はterraformのplanが見やすくなるナニカを作りたいと思います 作りたいもの terraform planを見やすくする どうせなら生成AI使って要約とかしたい Agentが作るんだからちょっとかっこいいフロントエンドも作ってもらう 開発環境 今回使用した開発環境は下記です　※ご参考までに 項目 使ったもの OS Linux(WSL2) IDE VSCode(1.99.3) Agent GitHub Copilot Agent 生成AIモデル Claude 3.7 Sonnet インフラ AWS やってみましょう まず最初にやることはアレですね。そうプロンプトを考えましょう 今回は超ざっくりにやりたいことを↓のプロンプトでAgentに指示しました terraform planの結果をブラウザで確認する仕組みを実装したいです AWS構成：Cloudfront+S3+Lambda + Bedrock 案 1.ローカルでterraform plan -out hoge.planを実行し、plan内容をファイルに出力 2.hoge.planをs3に送信 3.lambdaが起動し、terraform show hoge.planを実行し、人が見れる形にしてs3に出力 4.terrafrom showの結果をBedrockが解析し、planの要約を出力 5.ブラウザからアクセスを実施し、要約とplanの内容を人が見る 上記案で実施しようと考えています もっと良い案があれば提案してください。もしなければ上記を作る手伝いをしてください コードを作成する必要があるのであれば、/home/mizuki/配下にディレクトリを作成し、そこに出力してください え？もっといいプロンプトあるって？ 細かいことは置いておきましょう なぜなら私はインフラしかできないエンジニアです そしたら賢いAgent君は私のやりたいことを300%ぐらいパワーアップさせて対応してくれました お客様のご要望は、Terraform planの結果をブラウザで確認するシステムの実装についてですね。ご提案いただいた案は実現可能であり、良く練られています。まずはその案を評価し、いくつかの改善点を提案した後、実装のお手伝いをさせていただきます。 # 現在の案の評価 1. ローカルでterraform plan -out hoge.planを実行し、plan内容をファイルに出力 2. hoge.planをS3に送信 3. LambdaがS3のイベントをトリガーに起動し、terraform show hoge.planを実行して人間が読める形式でS3に出力 4. Bedrockがterraform showの結果を解析し、planの要約を生成 5. ブラウザからアクセスして要約とplanの内容を確認 # 考えられる改善点 1. **セキュリティ強化**: S3へのアクセス制限、IAM権限の適切な設定 2. **差分のビジュアル表示**: planの結果をグラフや色付きテキストで可視化 3. **履歴管理**: 過去のplan結果を保存し比較できる機能 4. **通知機能**: 重要な変更がある場合にSlackやメールで通知 5. **インタラクティブな操作**: ブラウザ上でリソースの詳細を展開/折りたたみできる機能 これらを加味した上で、実装を進めていきましょう。まずはディレクトリ構造を作成します。ディレクトリ構造を作成しました。次に、このプロジェクトの各コンポーネントを実装していきましょう。 まず、Terraformの設定ファイルを作成して、必要なAWSリソースを定義します。ディレクトリ構造の作成が完了しました。これから各コンポーネントを実装していきましょう。 まずはTerraformのインフラストラクチャを定義する`main.tf`を作成します。このファイルにはS3バケット、CloudFront、Lambda、IAMロールなどの必要なリソースを定義します。 : : 略 。。。。君すごいね Agentの目玉機能であるローカルへの操作もどんどんやってくれます これは作業ディレクトリ作って、そこにterraformファイルなどを作成してくれている その後は作業内容を説明してくれながら、これやっていい??って随時確認してくれます 私はPythonのことなんてナンモワカランのでとりあえず全部「accept」ボタンを連打します 最終的にJavaScriptやCSS及びPythonやTerraformファイルだけじゃなく、Lambdaのビルド用シェルまで作ってくれました しかもご丁寧に使い方やデプロイ手順や改良案も添えて デプロイしてみましょう 作ってくれたのはありがたい けどほんとに動くの？？？ とりあえず教えてくれたデプロイ手順を愚直にやってみます terraform applyしてAWSにS3とかCloudfrontとか作って ビルドしてLambda用のzip作って Lambdaをデプロイして JSとかCSSをS3に配置して なんのエラーも出ずにデプロイ完了しました 流石にエラーでました 作ってくれた環境にアクセスしたら、流石にLambdaのエラーが出ました 私はPythonの事わからないのでエラー解析なんてできません なので、LambdaのログをそのままAgentに渡して調べて貰いましょう すると予想通りログを解析して問題点の指摘及び修正までやってくれました 修正されたファイルをもう一度デプロイしたら今度はうまくいきました どんなシステムになったのか？ 出来上がったシステム構成ははこんな感じです！！ 流れとしては下記となっており ユーザーがterraform plan結果をS3に送信 イベント駆動でLambdaが起動 LambdaがBedrockへplan結果の解析/要約を依頼 生成されたplan結果の要約と要約ページのHTMLファイルをS3へ配置 Step FunctionsがCloudFrontのキャッシュを削除 ユーザーはS3へplan結果を配置するだけで全て完了します 実際にアクセスしてみましょう めっちゃかっこいいですね じゃあそもそもの目的であったplan結果の要約はできてるのでしょうか？ 完璧ですね。。。。 ちょっとインデントがおかしい気もしますが、細かいことは気にするのやめましょう 良いポイントとしては 作成/更新/削除などの件数が画面上部に表示され見やすい 変更されるリソースの変更内容が説明されている リスク評価が記載されており、このplanでのリスクが把握できる 注意点として今後改善したほうが良い内容を教えてくれる まさに求めていた内容です 細かい要件追加 その後も細かい修正をAgentに指示してやってもらいました 時間がUTCになってるからJSTにして インデントがずれてるから直して スクショを添えて planが生成されたらCloudFrontのキャッシュをクリアして などなど 上記の細かい要件追加も 簡単 にやってくれました やってみた感想 想像以上に簡単にシステムが作れました 1日でこのシステムを作成しましたが、ほとんどの時間がAgentの処理待ちだったので、実作業時間は1時間程度です。私のようにインフラしかできない人間でも、フロントエンド～バックエンドまで1人で構築できました。 ただしPythonコードを見ると素人目でも不要なコードが存在したりしています まだまだ改善の余地がありますが、これはMVP（Minimum Viable Product）段階での開発としては十分な成果です。今後、必要に応じてコードの最適化や機能の拡張を行っていくことができます。 重要なのは、短時間で機能するシステムを作り上げられたことです。これにより、アイデアを素早く形にし、実際の運用を通じて改善点を見出すことができます。 大事なのは 1人で短時間で簡単に作れた ってことです MVP(Minimum Viable Product)でのファーストステップとしてAgentは最適解だと思います まとめ Agentだけ使ってエンタープライズなシステムを作るにはまだまだ時間がかかりそうですが、プロンプト初心者の私でも簡単にシステム開発できました 何より驚いたのが「accept」ボタンを押すだけでAgentがサクサクとシステムを構築してくれる点ですね 今までは生成AIに提案された内容をコピペしたりコマンドを自分で実行したりしてましたが、今回はほぼ自分でコマンドを打っていません。「accept」ボタン押しまくっただけです ボタン押しただけでみんなの工数削減&ミス防止を可能になるシステムが作れたってことです また、今回作ったシステムは実際に利用しており、運用フローに組み込む予定です さいごに 私の所属するCloud Infrastructureグループでは一緒に働いてくれる仲間を募集しています。クラウドに自信のある方もこれからクラウドを学びたいと思っている若手の方も大歓迎です。お気軽にお問い合わせください 詳しくは、 こちら と こちら をご確認ください また、グループの紹介記事もWantedlyにて掲載されています！ こちら で見ることができるので、ぜひCloud Infrastructureグループについて知ってください！

My name is Ryomm and I am developing the app my route (iOS) at KINTO Technologies. The my route app's library management tool has finally migrated from CocoaPods to Swift Package Manager (SPM)! Introduction The my route app used to use CocoaPods. However, around summer 2024, CocoaPods issued the following announcement. https://blog.cocoapods.org/CocoaPods-Support-Plans/ That was an announcement stating that CocoaPods would be transitioning to maintenance mode. It doesn't mean it will become unusable, and they will still support security issues and other concerns. However, support for GitHub issues and the addition of new features will no longer be provided. If you do this, you may find yourself in a situation where you want to add a new library only to find that CocoaPods is no longer supported, or that you are unable to fix a problem with CocoaPods itself. There had been talk of migrating to SPM at my route for some time, but the announcement of CocoaPods gave us the impetus to finally embark on the migration to SPM. What is Swift Package Manager? SPM is a genuine library management tool provided by Apple. https://github.com/swiftlang/swift-package-manager Speaking of library management tools for iOS, there are other third-party tools such as CocoaPods and Mint. The biggest difference between these and SPM is that SPM is directly integrated into Xcode. This means that by simply updating Package.resolved when necessary, the latest specified version of the library will be imported when you open or build a project. This means that I no longer need to tell my team to run pod install every time I update the version. Migration Complete! It’s simply a matter of transferring everything, but there were a few stumbling points and tips that I encountered, so I’d like to share them with you. The Specified Version of the Library Is Not Downloaded I had a problem where, when switching to the desired version, even after executing "Reset Package Caches" or "Update to Latest Package Versions" in Xcode, the version specified in Package.swift would not be downloaded. Even deleting DerivedData doesn't fix this. This is because residual caches remain in locations such as ~/Library/Caches/org.swift.swiftpm and ~/Library/org.swift.swiftpm . When I delete these, the correct version of the library is downloaded. Build Doesn't Pass Only During Testing My route has multiple schemas, each of which depends on slightly different libraries. So, it has a structure separated from the project's own Package.swift as shown below. We create a Dependencies package in the same workspace as the project file and manage the library within this package so that it becomes a one-to-one product (framework) with each schema. These products are linked to each target's Frameworks, Libraries, and Embedded Content as shown in the screenshot on the right. However, with this configuration, I encountered a problem where xcodebuild build passed, but xcodebuild build-for-testing did not. This command corresponds to Run and Test in the Xcode GUI. Typically, items in a package are tested within that same package. However, with the above configuration, tests are run on the project’s main target instead. This means that the tests are executed outside the package. So yes... this is technically an improper setup... which I do plan to fix, but for now the tests are included in the same package, which allows us to proceed with testing preserving the current configuration. It seems that the cause was the different behavior around the linker between Run and Test. The build product for Run is assumed to be 1), and the build product for Test is assumed to be 2). 1) includes SPM in Dependencies, but 2) does not. Therefore, if a test in the MainTarget depends on a library in Dependencies, the build will fail. A simple solution is to move the libraries required for testing to the SPM in MainTarget, which will be included in the build for both Run and Test. To fundamentally solve the problem, it is a good idea to simplify the tests in one package, as mentioned above. Putting SwiftLint and LicensePlist on SPM Since I want to include SwiftLint and LicensePlist in the project's Build Phase so that they are executed during the build, I create a separate package in a location independent of the workspace. Project/ ├── Test.xcworkspace ├── Test.xcodeproj ├── Test/ │ └── ... ├── ... └── tools // <- this! ├── Package.swift └── Package.resolved I placed the libraries I wanted to include in the Build Phase, such as SwiftLint and LicensePlist, into the newly created "tools" package. Then, I created a shell script, which I executed only the first time to download the libraries locally. SPM_PACKAGE_PATH=tools SPM_BUILD_PATH=tools/.build/release echo "swiftlint" && $SPM_BUILD_PATH/swiftlint --version || swift run -c release --package-path $SPM_PACKAGE_PATH swiftlint --version echo "license-plist" && $SPM_BUILD_PATH/license-plist --version || swift run -c release --package-path $SPM_PACKAGE_PATH license-plist --version Then, it becomes executable at tools/.build/release/swiftlint , so I add this to the Build Phase. The same applies to LicensePlist. Even when executing from platforms like Bitrise, it can be confirmed that LicensePlist is executed by executing the shell first, followed by building the project. Conclusion It's been a few months since the migration, and while there are still some issues, I feel that replacing libraries has become easier. When I want to check whether an issue is caused by a library, I personally find it convenient to quickly check it using the app-version of Praygrounds for a quick test. Now that SPM is integrated into the project, it has also become possible to use Apple libraries that are only available via SPM. In the future, I would like to use these libraries to improve our implementation.

Hello(=ﾟωﾟ)ﾉ I'm Hikaru Takeno @t_hikarutaaaan , from the Talent Acquisition team at KINTO Technologies! I've been at KTC for almost three years, but as an HR professional, I'm still a newbie in my first year. In this article, I’d like to share what I applied and learned as a first-time Event PM , as an HR newbie! Event Overview Event name: Product History Conference Organizer: YOUTRUST Inc. Date: 10:00 a.m. to 8:00 p.m., Saturday, November 30, 2024 Location: TODA HALL & CONFERENCE TOKYO @ card The Beginning With less than three months to go until the event, it has been decided that KTC will join as a gold co-sponsor of the event! When I heard, "Alright, the approval's in!!" I was relieved... But at the same time I thought, "Wait, where do we even start...?" That was when a miraculous helping hand reached out to me °˖✧◝(⁰▿⁰)◜✧˖° Kin-chan , the manager of the Developer Relations Group, shared the Slack channel from another event and advised us, saying, "This is how you should approach it!" ( ;∀;) For someone like me who learns best through hands-on experience, it was the most easy-to-understand information. I'm truly grateful. Next Steps: "Alright, let's take a look at the Slack channel." … (ㆆωㆆ)ｰ "Hmm, not very sure but let's get everyone involved together and have a meeting first!!!" We set up a meeting . Then, we rummaged through the internal Confluence. We found meeting minutes that seemed usable. Copy!!  We copied past materials! Somehow, by rummaging through the internal Confluence and the past event-related Slack channels , we were able to resolve most of the issues! Looking back, I’m really impressed and grateful for our environment where knowledge sharing comes so naturally… It was a lifesaver… On the Day of the Event Since we had finished setting up the booth the day before, we arrived at the venue at 09:30 a.m. .  The booth with simple setup completed After settling down a bit, the time of opening the venue was approaching... and then... Someone said, " Let's form a circle among all the members of the sponsoring companies!! " Σ(･ω･ﾉ)ﾉ! So, I firmly put my arms around the shoulders of the people on both sides of me. Then the event started!　As people gradually started to arrive, there was a crowd at the booth by lunchtime!  Many visitors showed interest! We were talking nonstop the whole time!! We conducted a questionnaire at the booth, and I was really happy to see that there were more people who knew about KTC than we expected. The most common response: "I've seen you at lots of events!!" I was reminded once again of the collective effort of all the employees who have steadily participated in many of them!!  KTC's mascot, Kumobii, is also doing its best Kageyama-san takes the Stage This time, we held a cross-talk session featuring our Executive Vice President Hitoshi Kageyama and Mr. Naomichi Okada, CTO of Luup Inc. ! With the theme "Creating a New Standard of Mobility through the Use of Technology: The Future as Imagined by the Two Companies,” they spoke candidly about the challenges both companies are facing, including: The thoughts behind the founding and establishment of each company The evolution of their development organizations across various phases The challenges and excitement of working on product development amid numerous constraints And more.  Crosstalk What I’ve Learned It is best to set agendas and choose appropriate timings for operation meetings. Perhaps it would be a good idea to try a fixed weekly meeting style? Kumobii's unstoppable popularity Cuteness proved to be unbeatable as expected! Kumobii is the official mascot of KINTO Corporation. https://corp.kinto-jp.com/mascot/ An environment where knowledge naturally accumulates is a huge strength. . Thanks to our open Slack channel culture, it's easy to find and learn from past events. So many incredibly experienced employees in our company Hosting a post-event party was absolutely worth it! . It helped keep the connections going beyond the event itself. About 25 to 30 people joined.  People from other companies also joined the post-event party. Thank you to the participants from X Mile Inc. @xmile_product , Asuene Inc. @asuene_inc_ , and note inc. @note_corp !! What I Want to Carry into the Next Event We should think more about what kind of experience we want people to have when they visit our booth. I want to apply our experience to booth setting and the arrangement of novelties. I want to prepare “express cards” that allow people we’ve directly spoken to skip the selection process . Since this was also an event hosted by YOUTRUST Inc., a career-focused social media, many participants were interested in changing jobs. It might have been a good idea to create a process to ensure talented individuals remain in the selection process without fail. I want to set KPIs. Looking Back Being a bit nervous, started off my role as the event PM thinking, "I have no idea what’s going to happen...!" with my heart racing. But in the end, it was like, "We made it!!" and the event finished without any major problems. This was all thanks to the operations team, who not only helped me prepare in advance by guiding me with questions like, "Shouldn’t we decide on this?" and "What’s the status of this?" but also stayed flexible and supportive on the event day itself! Really, thank you so much!!!  Everyone wearing Kumobii!

This article is the entry for Day 23 in the Developer Relations Group Advent Calendar 2024 🎅🎄 Introduction Hello! I am Rina ( @chimrindayo ). I am an engineer at KINTO Technologies, where I am involved in the development and operation of Mobility Market and on the Developer Relations Group as well. KINTO Technologies launched the Tech Blog in July 2022, and thanks to our writers and readers, we have been able to keep it running to this day. Thank you for reading as always! Our Advent Calendar is one of the biggest events on our blog each year, and this time was no exception 🎄 We proudly reached a total of 100 articles up to Series 4 👏 Amazing! Until we were able to release 100 articles, the Developer Relations Group members worked hard every day, coming up with various new ideas. This time, I would like to introduce some of the journey and initiatives we went through to release 100 Advent Calendar articles. Journey of the KINTO Technologies Advent Calendars The KINTO Technologies Advent Calendars were started by volunteers in 2021, prior to the launch of the Tech Blog. The following year, in April 2022, a small team called the Tech Blog Project (now Developer Relations Group) was formed, and the team launched the Tech Blog in July 2022. Since the launch of the Tech Blog, we have consistently posted an Advent Calendar every year, increasing the number of articles by 25 each year ⤴️ Year URL Number of posts 2021 KINTO Technologies - Toyota car subscription service "KINTO" under development! Advent Calendar 2021 24 2022 April: Tech Blog Operation Team was established July: Tech Blog launched🎉 - 2022 KINTO Technologies Advent Calendar 2022 KINTO Technologies Group Introduction Advent Calendar 2022 50 2023 KINTO Technologies Advent Calendar 2023 75 2024 KINTO Technologies Advent Calendar 2024 100 I will reflect on each year to review the steps we took to increase the number of articles. Since we adapted our approach to fit each stage, from the initial launch to the ongoing operation of the Tech Blog, I hope this article will be helpful for anyone thinking about starting a tech blog or are currently running one but facing challenges to increase the number of contrbutors🙌 2022 Advent Calendar Initiatives First, here are some of the initiatives we implemented during the early stage of the Tech Blog. Looking back, I think a defining feature of this initial phase was that we approached individuals to ask them to write for the Tech Blog. Let me introduce some of the specifics. Making it a Rule to Show Gratitude to Writers The first thing we did was to establish the rule of showing "gratitude toward writers" as the stance of reviewers. This rule was decided by project team led by Nakanishi-san . We have also continued to convey gratitude to writers by requesting managers and leaders, who could serve as reviewers, to do so in company-wide meetings or 1-on-1 sessions, and by expressing our gratitude in the PR (pull request) template for the Tech Blog. Excerpt from the PR template for the Tech Blog Even now, expressing gratitude to writers for their contributions is something that the entire Developer Relations Group values very much. Having All the Managers Write Articles The next step we took was to have all the managers write articles. In the early days of the Tech Blog, many people in the company thought, "What is a tech blog? Why are we doing this?" So our first priority was to make sure each group leader understood the value and purpose of the Tech Blog. We also believed that getting General Managers to contribute articles was key to spreading awareness across the company. After a 30-minute meeting with them, we successfully launched the "KINTO Technologies Group Introduction" series. https://qiita.com/advent-calendar/2022/kinto-technologies-introduction Under the theme letting everyone get to know our people and offices , managers and leaders wrote articles for our Advent Calendar. On Christmas Day, Vice President Kageyama closed out the series with an special post titled "2022 Year in Review & 2023 Outlook”🎅. Since then, his article has become an annual tradition, one that's read by nearly all current and prospective employees. His articles condense key aspects about KINTO Technologies, including: What kind of organization we are What we achieved over the past year What we aim to pursue in the year ahead They serve as a compass for the organization. It’s a once-a-year piece Written with care and dedication. Click here 👇 for his review & outlook article: https://blog.kinto-technologies.com/posts/2022-12-25-LookBack2022/ Studying the Structure of Each Article Together We hold 30-minute sessions to help those who are eager to write but lack experience with technical articles and need guidance on where to begin. During these sessions, members of the Developer Relations Group conduct interviews with the writers, asking about: Projects they've worked on Successes, failures, and the lessons learned along the way Their tech stack and tools of choice As we listen, the Developer Relations team helps the writer shape the article structure in real time, laying a solid foundation they can build on. the Developer Relations team shares the proposed article structure with the writer on the spot for feedback. Most writers respond with surprise, saying things like, “This might be easier than I thought!” When we launched the Tech Blog in 2022, we carried out this process with every single writer. Looking back, it might seem like a bit of a brute-force approach; but we believe that this hands-on, all-in communication style is one of our team’s greatest strengths. When we’ve shared this method with technical PR teams at other companies, it’s often met with praise, with many calling it a “wonderful initiative.” There were several reasons behind this approach. The main goals were to: Help every writer understand the value of the Tech Blog Lower the barrier to contributing Gain visibility into who is responsible for what Build trusting relationships, where writers feel comfortable asking us for advice And more. However, what we consider most important is that each writer reaffirms the value of their own work . In face-to-face interviews with them, many would say, "I can write, but my daily work is just ordinary, so it's not something worth putting on a blog." But that’s exactly the kind of story we want to share. It’s perfectly okay to be ordinary! There’s always someone out there facing the same challenges, and if your story reaches even one of them, then it’s absolutely worth sharing on the Tech Blog. That’s why we kept encouraging each writer: “Let’s write it together!” This way of thinking is still valued by the Developer Relations Group, and it was featured in a blog post that was released the other day.👇 https://blog.kinto-technologies.com/posts/2024-12-11-gekishin/ In this way, not only do we work together on the structure of each article, but we also believe it is important to help each writer feel confident about the content they are about to write. 2023 Advent Calendar Initiative In the early stages of the Tech Blog's launch, we focused on encouraging first-time writers through one-on-one conversations. Next, in 2023, we shifted our focus on sustanining that momentum supporting continued contributions and increase the number of writers. Supporting Inputs One of the initiatives the Developer Relations Group undertook first in 2023 was supporting the introduction of Udemy Business . We believe meaningful output on the Tech Blog starts with strong input, the knowledge and skills the writers bring. To reinforce this mindset, we linked access to Udemy Business with participation in the Tech Blog: registering for an account required contributing at least one article. There are actually team members who have taken Udemy courses and are currently writing articles.👇 https://blog.kinto-technologies.com/posts/2024-08-30-udemy-kotlin-coroutines-and-flow/ In 2024, in addition to the support for the introduction of Udemy, we established a new team within the Developer Relations Group called the "Learning Roadside Station". This team was created to offer comprehensive support for both input and output. Read more about the background of the Learning Roadside Station Team's establishment here.👇 https://blog.kinto-technologies.com/posts/2024-04-23_%E5%AD%A6%E3%81%B3%E3%81%AE%E9%81%93%E3%81%AE%E9%A7%85%E3%81%AF%E3%81%98%E3%82%81%E3%81%BE%E3%81%97%E3%81%9F/ Encouraging Newcomers to Write an Article In October 2023, we launched a new initiative asking all new employees to write a blog entry after joining the company. The goals of this initiative are to: Give readers a feel for our company’s atmosphere Reduce the hesitation around writing for the Tech Blog Encourage connections among new hires By having everyone contribute from the start, we aim to lower the barrier to writing and helping newcomers understand how to approach blog writing, ultimately making it a natural practice. https://blog.kinto-technologies.com/posts/2024-01-01-newcomers-introduction/ Initially, the Developer Relations Group planned for each new hire to write their own entry. But when Ryomm-san , who joined in October, suggested, “I’d like to co-author!” it sparked the creation of a new co-writing process. This is just one example of how we’re building a culture where Tech Blog initiatives aren’t just driven by the Developer Relations Group, but shaped by contributions from employees across the company. Asking Teams to Write Next, we implemented a plan where we asked teams at the project and department-level to write for the Advent Calendar. Five groups, with five members each, participated and together filled 25 days' worth of content. The themes were: New Car Subscription Website Renewal Project Agile Development KINTO FACTORY QA Diversity & Inclusion By encouraging people across teams to write together, we aimed to make the writing process more collaborative and approachable. Having teammates work on shared topics made it easier to ask for feedback and consult each other during the writing process. We also hoped that theme-based series of articles would feel more relatable to internal readers, helping to increase engagement, and ultimately, encouraging more people to read and share the Tech Blog, both internally and on social media. The actual project is introduced in the blog here.👇 https://blog.kinto-technologies.com/posts/2023-11-20-advent-calendar/#2.-%E8%A8%98%E4%BA%8B%E3%83%AA%E3%83%AC%E3%83%BC 2024 Advent Calendar Initiative Finally, let us share what we’ve been working on this year. In 2024, our focus has been on expanding the Tech Blog and improving its quality . Holding a Tech Blog Study Session In August 2024, we held an internal study session led by p2sk-san , a member of the Developer Relations Group. The session, titled "Reviewing the Tech Blog and Best Practices for Enhancing Dissemination Power," involved a comprehensive review of all Tech Blog articles published by KINTO Technologies. He shared insights into what we’re doing well, along with suggestions for improvement. The response from participants was overwhelmingly positive: "My motivation increased!" "Made me want to try writing articles!" "Let's do our best for the Advent Calendar!" The session was and full of valuable insights, but we’ll leave the deep dive for another time. Open Call for Planning & Operations In previous years, the Advent Calendar was planned and led by the Developer Relations Group, but this year, we have recruited volunteers to take part. The goal was to engage members beyond the usual circle and reach new voices we hadn’t connected with before.  While we were hoping for up to five volunteers, only one person raised their hand; but that one person made a big difference. naka_shima san stepped forward and thanks to his involvement, we were able to launch a department-based article series you’ll read about later 👏 Filling the Series by Division In 2024, we structured the Advent Calendar series around articles written by Groups (org. structure). A "series" refers to posting a total of 25 articles one for each day from December 1st to 25th The groups involved were the Mobile App Development Group, where our volunteer coordinator naka_shima-san belongs, and the Developer Relations Group. Mobile App Development Group Developer Relations Group The Mobile App Development Group was chosen first because it was the most actively engaged in information dissemination this year. This group stood out for not only planning and managing major sponsored events like iOSDC 2024 and DroidKaigi but also for regularly contributing to the Tech Blog. With a playful challenge of “Can the Mobile Group fill all 25 days by themselves?” they impressively took on the task and completed an entire series 🎉 Next, the reason why it was decided that the Developer Relations Group would fill one series was that we believed it was important for the very team leading information dissemination to actively demonstrate its output through the Tech Blog. Thanks to these efforts, we released a total of 100 articles, which included: Random topics freely contributed by volunteers Articles by the Mobile App Development Group Articles by the Developer Relations Group Translated articles (English-Japanese) by the Localization Team Future Prospects Today I shared our journey through the Advent Calendars over the past three years: from the Tech Blog’s launch in 2022 up to the present in 2024. Thanks to everyone involved and all the efforts made, we proudly reached the milestone of releasing 100 articles in the 2024 Advent Calendar! Looking ahead to 2025, we plan to focus on initiatives that make writing easier and keep motivation high for our contributors. Thank you for reading and supporting us throughout 2024☺️

Hello ( º∀º )/ My name is Yukachi from the Event Team on the Developers Relations Group. In 2024, we began sponsoring conferences and ramped up our external communication efforts, leading to a big increase in events both inside and outside the company compared to previous years! As we started hosting more events with external guests I realized it's a bit tricky to explain how to get to our office. I thought, ¨Maybe I should use more photos to explain...¨ and then I thought, wait, we have the Tech Blog! That's how this article came to be. About the Office Our company has four locations! Now that I think about it, I actually wrote a post introducing them two years ago! @ card Out of all our locations, we host most of our events at TOKYO JCT in the Muromachi office . It's on the 16th floor of the office building in COREDO Muromachi 2 ** Address: Muromachi Furukawa Mitsui Building (COREDO Muromachi 2), 2-3-1, Nihonbashi Muromachi, Chuo-ku, Tokyo 103-0022 Directly connected to Mitsukoshimae Station on the Tokyo Metro Ginza Line *About a 2-minute walk from the ticket gate. Directly connected to Mitsukoshimae Station on the Tokyo Metro Hanzomon Line *About a 5-minute walk from the ticket gate. Directly connected to Shin-Nihombashi Station on the JR Sobu Rapid Line *About a 5-minute walk from the ticket gate. About a 10-minute walk from Kanda Station on the JR Chuo Line, Keihin Tohoku Line, and Yamanote Line. Let me walk you through how to get here from each station with photos! Shortcut to the Access Guide From Mitsukoshimae Station (Tokyo Metro Ginza Line) From Mitsukoshimae Station (Tokyo Metro Hanzomon Line) From Shin-Nihombashi Station (JR Sobu Rapid Line) From Kanda Station (JR Chuo Line, Keihin-Tohoku Line, Yamanote Line) Mitsukoshimae Station (Tokyo Metro Ginza Line) Let's start with the closest stop, which is the Mituskoshimae Station on the Ginza Line!  Exit from the Mitsukoshi Ticket Gate and turn left.  There is Coredo Muromachi 1, so go through it.  Once you pass through, you'll find the entrance to COREDO Muromachi 2.  Go inside, take the escalator up.  The office area will be right in front of you! :::message During event reception time, our staff will be stationed at the entrance to the office area! Since an access card is required to get in from COREDO Muromachi 2, we'll be keeping an eye out and opening the door for anyone who looks like they're here for the event! ::: For appointments or non-event visits before 6:00 p.m., please use the main office entrance, located just to the left after exiting the building.  Step out to Edozakura-street from here.  This is the main entrance. Please check in at the front desk inside the entrance, then go to the floor your host has indicated. :::message If you arrive after event reception hours, please use the same main entrance and contact the person listed in the event details, either via phone or through our official X account. An event staff member will come to meet you on the first floor. @ card ::: Mitsukoshimae Station (Tokyo Metro Hanzomon Line) It's also directly connected from Mitsukoshimae Station on the Hanzomon Line, but it's a bit of a walk...  Exit from the Nihombashi Gate and go straight to the right.  There is a small staircase, go down and take a slight right.  Go straight and turn left at the end.  Here it is — COREDO Muromachi 2!  Go inside, take the escalator up.  The office area will be right in front of you! :::message During event reception time, our staff will be stationed at the entrance to the office area! Since an access card is required to get in from COREDO Muromachi 2, we'll be keeping an eye out and opening the door for anyone who looks like they're here for the event! ::: For appointments or non-event visits before 6:00 p.m., please use the main office entrance, located just to the left after exiting the building.  Step out to Edozakura-street from here.  This is the main entrance. Please check in at the front desk just inside the entrance, then head to the floor your host has indicated. :::message If you arrive after event reception hours, please use the same main entrance and contact the person listed in the event details, either via phone or through our official X account. A staff member will come to meet you on the first floor. @ card ::: Shin-Nihombashi Station (JR Sobu Rapid Line) It's also directly connected from Shin-Nihombashi Station, but it's a bit of a walk...  After exiting the ticket gate, turn left and go straight towards Mitsukoshimae Station (Shin-Nihombashi Station only has one ticket gate!) *  Turn left towards Mitsukoshimae Station.  Continue straight with the Ginza Line on your right.  You'll see COREDO Muromachi 1 on your left, go inside and walk through it.  Once you pass through, you'll find the entrance to COREDO Muromachi 2.  Go inside, take the escalator up.  The office area will be right in front of you! :::message During event reception time, our staff will be stationed at the entrance to the office area! Since an access card is required to get in from COREDO Muromachi 2, we'll be keeping an eye out and opening the door for anyone who looks like they're here for the event! ::: For appointments or non-event visits before 6:00 p.m., please use the main office entrance, located just to the left after exiting the building.  Step out to Edozakura-street from here.  This is the main entrance. Please check in at the front desk inside the entrance, then go to the floor your host has indicated. :::message If you arrive after event reception hours, please use the same main entrance and contact the person listed in the event details, either via phone or through our official X account. A staff member will come to meet you on the first floor. @ card ::: Kanda Station (JR Chuo Line, Keihin-Tohoku Line, Yamanote Line) Finally, here's how to get to the office from Kanda Station!  Exit from the South Exit (for Nihombashi) and walk along the Starbucks.  Cross the street and go straight to the left.  Cross the street and go straight to the right.  Keep going straight.  Go straight to the left in front of COREDO Muromachi 3.  You'll spot COREDO Muromachi 2, but just keep going straight past it.  You'll see the office entrance on your left. That's the one!  You'll find a staff member holding a sign waiting inside the office area entrance (During event hours). For appointments or non-event visits before 6:00 p.m., please use the main office entrance, located just to the left after exiting the building. :::message If you arrive after event reception hours, please use the same main entrance and contact the person listed in the event details, either via phone or through our official X account. A staff member will come to meet you on the first floor. @ card ::: Conclusion Hopefully, this post makes your visit to the KINTO Technologies Muromachi office a little smoother! Thank you for visiting! We look forward to seeing you again soon! (^_^)/

はじめに こんにちは、分析プロデュースグループの平田です！ アナリストとして働いている私が、毎日取り組んでいるSQL作成業務の効率化について、今回はお話しします。 この記事では、「Github Copilot Agent」と「Python」を使って、数百行に及ぶ複雑なSQLを書き上げる業務をどう効率化したのか、その試行錯誤のプロセスと結果、そして今後の改善について語ります。 【まとめ】 ✔︎ テーブル情報を事前に用意して、生成AIにSQLを作成してもらう ✔︎ 作成したSQLをPythonで自動実行・チェックする仕組みを実装 ✔︎ エラーが発生したら自動で修正してもらい、業務効率をアップ！ 背景：日々のSQL作成業務とその悩み 普段、私が直面しているのはこんな課題です… 生成AIとの煩雑なやり取り 生成AIにテーブルの情報やデータ型、日付フォーマットなどを会話の度に何度も説明する必要があり、その都度手間がかかっていました。 膨大なSQLの作成 マーケティングに使うユーザー抽出や分析用データ作成など、何百行ものSQLを書かなければならず、複雑な加工ロジックが散在している。 試行錯誤の繰り返し（ループ） 作成したSQLをコピーして実行、エラーが返るとまたエラーログを渡して修正依頼…この繰り返しがボトルネックに。 自分で修正すると、Github Copilotが作成した最新版と差分が生まれ、次の修正を依頼したときに先祖返りしてしまうことも。 試行錯誤！生成AIとPythonで自動化フローを構築 そこで、こんな流れで業務の効率化に挑戦しました。 自動化フローの全体像 前提情報の登録 各テーブルの構成、データ型、サンプルデータ、サンプルSQL、加工Tipsをそれぞれのpromptファイルにまとめます。 生成AIでSQL生成 全体フローのプロンプト（「テーブル情報をもとにSQLを生成&保存、実行チェック」）を生成AIに渡して、自動でSQLファイルを作成します。 Pythonで実行・チェック 生成されたSQLをPythonスクリプトで実行。エラーがあればエラーログをフィードバックし、自動修正を促す仕組みにしています。 具体的な取り組みポイント 以下は、私が実際に構築したディレクトリ構成とファイル例です： rules / conversation_rules.prompt.md 基本的な会話のルール。 テーブル情報をもとにSQLを生成&保存、pythonファイルを実行してSQLのが正しいかをチェックすること。また、SQL作成の決まりや会話の好みを記載。 tables / <テーブル名>.prompt.md テーブルの情報を記載。 サンプルデータを載せることでデータの特徴を判断してもらう。 ## テーブル名 users ## 説明 ユーザー情報のテーブル。user_idをキーにorderテーブルと紐付け可能 ## sample SQL ```sql select user_id as "顧客ID", name as "顧客名", address as "住所" from users ``` ## sample data | 顧客ID | 顧客名 | 住所 | | ------ | -------- | -------------------------------- | | 1 | 山田太郎 | 東京都新宿区西新宿 | | 2 | 佐藤花子 | 大阪府大阪市中央区難波 | | 3 | 鈴木次郎 | 愛知県名古屋市中区栄 | sql / 作成されたSQLをファイルで保存してもらう。 utils / query_checker.py SQL作成後このpythonファイルを実行し、SQLが正しいかをチェック。 実際のGithub Copilot Agentの回答例 実際の動きが確認できる簡単な例をお見せします。 [in] 今月何回メール配信したかをユーザー別に抽出するSQLを作成して！ [out] まずスキーマ情報を確認してから、メール配信の分析に関する SQLを作成しましょう。メール配信に関するテーブル情報を探します。 "› Searched for files matching /prompts/tables/**, 14 matches" 配信履歴に関するスキーマ情報が見つかりました！ まずはその内容を確認して、今月の配信回数を抽出するSQLを作成していきましょう。 Read メール配信履歴.prompt.md スキーマ情報を確認できました。[メール配信履歴テーブル]というテーブルに格納されているようです。 今月（2025年4月）のメール配信回数を集計するSQLを作成していきます。 このテーブルには以下の重要な列があります： • email_id： メルマガID • delivery_timestamp： 配信日時 • delivery_status： 配信ステータス それでは、今月のメール配信回数を集計するSQLを作成し、ファイルに保存しましょう。 ファイル名は メール配信集計_今月.sql です。 このSQLでは以下の処理が実行されます： ・ 2025年4月1日から4月30日までの期間に配信されたメールを対象にします。 ・ 「配信完了」ステータスのメールのみを集計します。 ・ ユーザーごとに、配信回数、成功数、失敗数、および総配信数を集計します。 ・ 最後に全体の合計も表示します。 Pythonでクエリを検証してみますか？ 検証のため、先ほど作成したSQLを実行してみましょう。 python3 query_checker.py メール配信集計_今月.sql SQL検証が完了しました！ クエリは正常に検証され、実行可能と確認されました。 テスト実行では構文エラーもなく、正しい実行計画が生成されました。 結果と今後の展望 嬉しいことに、一連のプロセスが無事稼働し、SQL作成の自動化に向けた第一歩となりました！ とはいえ、完璧ではありません。今後取り組みたい課題や工夫は…？ ● 現在の課題 レビュー作業の手間 AIが生成したSQLのレビューは、どうしても自分で作成する場合よりミスを見落としがち。 検証する仕組みや、実際にSQLを書かない人でもレビューを完遂できる手法については、いまだ試行錯誤の段階にあり、改善していきたいところです。 また、生成AIの進化によって、これらの課題が解決されることにも期待しています！ 意図通りのデータ抽出のチェック 要件定義の不備や、私の頭の中の情報がうまく言葉にできていないケースがあり、意図通りの処理かを自動判定するのは難しい。細かいニュアンスや意図の伝え方がまだまだ課題です。 ● 次へのチャレンジ 件数チェックの自動化 レビュー高度化のファーストステップとして、抽出した件数が意図通りかどうかのチェック機能を実装していきたい。 「秘伝のタレ」とも言えるデータ加工方法の蓄積 使い込むほどに見えてくる効果的なデータ加工テクニックを、プロンプトにどんどん追加していきたい。 分析の自動化への拡張 最終的には、SQLの作成から抽出データの分析まで、ある程度自動で行える体制を目指します！

Introduction I'm super_spa3 (Hirofumi Nakano), a product manager at KINTO Technologies Corporation. I had the chance to attend our very first overseas conference: the WWDC24! Although it's been a little while since the event, I'd like to share my experience and some related updates here on the Tech Blog. You might be wondering what a product manager has to do with iOS. When I first started my career, I was actually assigned to iOS development (though I had been tinkering with Android as a hobby before that). Later, I joined my second company, KINTO Technologies, as an iOS engineer, and now I’m working as a product manager. At work, I often use App Store Connect to release apps, and I’m also a member of the Apple Developer Program, continuing to develop apps on my own. One of the apps I'm currently involved in is called the KINTO Kantan Moushikomi App (KINTO Easy Application App), which allows users to seamlessly estimate costs and complete the screening process for the KINTO New Vehicle Subscription. https://kinto-jp.com/entry_app/ How I Got to Attend WWDC24 Participation in WWDC is determined by a lottery system. Around March each year, Apple sends out notification emails to members registered in the Apple Developer Program. Members can then click on the provided link to apply, and only those selected in the lottery get the chance to attend the event in person. However, I had tried applying multiple times during my previous job but was never successful. So, when I applied this time, I didn't have high expectations and just pressed the apply button. On the day of the lottery results, I opened my inbox and saw an email "Great News, Hirofumi!". As I read through the message, I realized I had actually been selected!  A clear sign of good news email It turned out that I was the only one from our Mobile App Development Group who got selected. After getting approval from President Kotera, I was set to attend the WWDC24! Eve of the Event at Apple Infinite Loop Campus On the day before the main event, I visited Apple's former headquarters, Apple Infinite Loop One Campus, to collect event merchandise and my name badge.  The entrance to Infinite Loop One is very minimalistic and stylish  My name badge during the event displayed TOYOTA FINANCIAL SERVICE CORPORATION, where our apps are published. The SWAG contents seem to change every year, and this year, it included a picnic mat. It's perfect now that travel is unrestricted post-COVID. How about some open-air coding?  WWDC24 swag bag and additional pins I got from Apple Park  The Apple Vision Pro badge is cool! Once entered the venue, I saw everyone taking photos in front of the WWDC24 logo. Since the event was held in the courtyard, I decided to strike a pose like on TBS's Another Sky.  The 3D WWDC24 board, I took a photo here afterwards!  This is my Another Sky rendition, Apple Infinite Loop Campus! (lol) With food and drinks in hand, I had the chance to interact with fellow developers working within the Apple ecosystem. Since I attended alone, I actively looked for someone to talk to. Whenever I saw an open seat I asked if I could sit next to them and started chatting about what we were developing, which led to great networking moments. Main Event at Apple Park The main event starts early in the morning. Check-in was scheduled for 7 a.m., but I heard that many people planned to arrive even earlier. So, I decided to get there well ahead of schedule and lined up before 7 a.m. at Apple Park Visitor Center, located next to the Apple Park, which houses an Apple Store, a café, a diorama, and a terrace. It seems that Japanese people are particularly good at early mornings because a sizable group of Japanese was already at the front of the line, eagerly waiting for the event to begin. As the clock approached 8 a.m., excitement built up, and the crowd spontaneously broke into chants of "dub-dub-dee-see!", hyping up the atmosphere (of course, I joined in with full enthusiasm!) Then, at 8 o'clock, the gates opened. I rushed to the Keynote venue and managed to secure a seat in the sixth row from the front.  It's so close! I had free time from this point until 10 a.m., so I grabbed breakfast, explored Apple Park, and hopped on a Zoom call with my team members in Tokyo, who were staying up late to watch the Keynote live. Here is a tweet from that time: https://twitter.com/ioco95/status/1800239213851574355 Since my seat was on the right side of the stage, I got a behind-the-scenes glimpse of Tim Cook and Craig Federighi as they prepared to go on stage. Craig's movements were incredibly acrobatic this year. The highlight of this year's Keynote was Apple Intelligence. But personally, what caught my attention the most was the Math Notes feature, which allows handwritten equations to be instantly solved, and the Mac Virtual Display in visionOS 2, which transforms into an ultra-wide monitor view on Apple Vision Pro (even though I don't own an Apple Vision Pro yet.) At lunch, Caffe Macs offered a variety of regional dishes on both sides of the dining area. Some people even lined up multiple times to try different meals. In the afternoon, I watched the Platform State of the Union inside Apple Park. Events at In-Person Labs and Exploring the Venue And finally, it's time for the In-Person Labs. This time is for meeting Apple engineers and staff, who can answer questions and help troubleshoot challenges in different areas. Since I had pre-booked a session for the Design Labs, I headed up to the third-floor area.  At the Design Labs, I was able to hear direct feedback from Apple employees on the KINTO Kantan Moushikomi App, as well as what the designers saw as potential challenges. It's a rare opportunity to get feedback from Apple employees, so this session was incredibly valuable. One lesson I learned from the Design Labs was the importance of ensuring that apps are accessible internationally by using TestFlight or other means. When I applied for the event, I included the App Store URL in the pre-registration form, assuming that Apple employees would be able to access it. However, since our app was Japan-only, they weren't able to check it in advance. As a result, I had to spend part of the limited time explaining the app and its challenges before I could ask my questions. (Something to keep in mind for anyone attending in the future!) After the Design Labs session, I walked around several areas to get answers to the questions I had from our in-house engineers. In between sessions, I also took some time to explore Apple Park (both the courtyard and inside the buildings).  There were a lot of people who looked like they wanted a photo, so I offered, "Would you like me to take a picture for you?" and naturally ended up making new friends. Attending the OMT Conference Actually, the on-site WWDC event only takes place for three days—Sunday, Monday, and Tuesday. While sessions run from Monday to Friday, the on-site event doesn't extend. So, I also attended the One More Thing (OMT) Conference held at a hotel near Cupertino, where Apple Park is located, from Tuesday to Friday.  A lineup of familiar sponsors  I got badges here, too. It says Pro Ticket, but it's actually a free ticket OMT Conference was divided into speaker sessions (Main Room) and consultation sessions (Big & Small Tents) where attendees could discuss various topics with experts. We could listen to talks from different speakers or discuss specific topics with experts. In addition, on Thursday afternoon, there was a workshop by Paul Hudson. Originally, it was planned to focus on SwiftData, but since there weren't many updates in WWDC24, he adapted the content and led a three-hour workshop on "What's New in SwiftUI" instead. I often check out Hacking with Swift, but this was my first time attending one of Paul's workshops in person. I was impressed by how amazing he is as a teacher and how quickly he answered questions. Attending Swift Social 2024 and Core Coffee In addition to attending WWDC and OMT conference, I also joined two community events. The first was Swift Social 2024. This event was organized by the Swift community and rented out a bar in downtown San Jose to celebrate Swift's 10th anniversary.  *Happy 10th Anniversary! *  Stylish SWIFT logo sign During the event, participants competed in a Swift-related quiz using Kahoot, racing to answer as quickly as possible. The top scorers won exclusive Swift merchandise as prizes.  I made it pretty far, but alas! The second was Core Coffee. This gathering took place every day during the WWDC week at different locations, and the session I joined was held on the rooftop terrace of Apple Park Visitor Center, right next to Apple Park. Perhaps due to its proximity, a few Apple employees also joined. Discussions naturally centered around the topics announced at WWDC24, sharing their thoughts and interests. One particularly lively debate was about how a color tint on the home screen might affect. I also had the chance to hear about the work styles of Apple employees and the products they are involved in, which made me feel even more engaging. Establishing Standards For Attending Overseas Conferences After I attended WWDC, several other conferences took place on the U.S. West Coast, and more members from our company began participating in overseas events as well. We are also working on establishing internal guidelines for attending international conferences. These global events offer valuable inspiration, and bringing it back home helps boost both individual and team motivation. Conclusion Attending WWDC for the first time was a long-awaited experience. It reignited my passion for development and game me the opportunity to build new connections. Everyone at WWDC is involved in developing something within the Apple ecosystem, and through conversations, I learned about the apps they were building, their roles, and the challenges they face in development. (Of course it was't just developers, there were also product managers and other contributors to the ecosystem.) It was incredibly inspiring that I also met several attendees whose primary job was not app development, but who were passionate about building apps as a hobby. I also had the chance to connect with developers from Japan, and some of those I spoke with during the event became familiar faces after returning home. If you're involved in product development for the Apple ecosystem, I highly recommend attending WWDC. Connecting with developers from around the world is truly exciting!

こんにちは、 Prism Japan でエンジニアを担当している宇田川です。 React Emailを活用したメールの自動配信を行うマーケティング施策について紹介したいと思います。 マーケティング施策において、直面していた課題 Prism Japanは、2022年8月にローンチされ、サービス開始当初から、様々なマーケティング施策を通じてユーザーを獲得してきました。しかし、一度獲得したユーザーが継続的にサービスを利用し続けてもらえるわけではありません。サービス開始から約2年半が経過しましたが、依然として休眠ユーザーが増加傾向にあります。 この問題に対処するため、プッシュ通知による再訪施策を実施していましたが、いくつか課題に直面しました。 通知設定をオフにしているユーザーには、プッシュ通知は届かない。 再訪を促すプッシュ通知を配信しても、アプリをアンインストールしているユーザーには届かないため、狙った効果を得られない。 実際、プッシュ通知の許諾率は約48%にとどまっており、この許諾率と、アンインストール済みユーザーを考慮すると、通知が届くユーザー数は、かなり絞られてきます。さらに、他のアプリからの通知もあるため、他の通知に埋もれがちです。このようにプッシュ通知で行える再訪施策の効果には限界がありました。 一方で、会員登録時にメールの登録をお願いしているのですが、こちらで登録されたメールの許諾率は、約90%と非常に高い水準を維持しています。アプリを削除していても、退会していないユーザーにはメールを届けられるため、再訪施策に適したマーケティングチャネルです。 しかし、運用面を考えるといくつか課題がありました。 まず、マーケティング担当のリソースが限られており、施策立案からSNS運用まで幅広い業務を、一人で担当している状況でした。メールのコンテンツ制作においては、ランキングの手動集計や適切な画像選定、レイアウト設計などに、工数が多く必要となります。このように、マーケティング担当のリソースを考えると、頻繁な配信が困難です。そのため、効果的なマーケティング手段と認識しつつも、頻度高くメール配信を行うことは、運用負荷を考えると現実的ではありませんでした。 メールの作成を自動化するために、React Emailを使うことに そこで「 メールの作成から配信までを自動化できないか 」という発想が生まれました。 コンテンツに表示する情報を自動的に収集し、メールコンテンツを自動で作成し、スケジュールされた日時に決められたレイアウトで自動的に配信できる仕組みを構築できれば、限られた人的リソースの中でも、ユーザーに合ったメールを送信できます。 しかし、HTMLメールを自動的に作成する処理を考える際に、エンジニアとして、実装方法に頭を悩ませました。 HTMLを直接操作するような処理を実装してしまうと、再利用性が低く、受信するメーラーによって表示が異なるといった問題も発生します。そして、将来的に、コンテンツの差し替えが発生することを見据えると、再利用性が高く、新しいコンテンツを柔軟に追加できるような実装が必要です。 このような悩みを抱える中で発見したのが「React Email」というライブラリです。 この「React Email」には、下記のような特徴があります。 JSXでHTMLメールを作成できる リアルタイムのプレビュー機能 コンポーネント化による高い再利用性 特に重要なのは、再利用可能なコンポーネント化によって、新しいコンテンツの作成が必要になった際に、容易に追加できる点です。また、Reactで記述されていることにより、掲載内容の動的な差し替えも容易になります。このような利点は、ユーザーの行動や興味に基づいて、コンテンツを差し替えることにより、 パーソナライズされたコンテンツの提供 を、低コストで行うことができます。全ユーザーに対して、同一のコンテンツを一斉配信するのではなく、各ユーザーの関心に合わせたコンテンツ配信を行うことで、 高い再訪率やエンゲージメント向上 が期待できます。 React Emailを利用することで、メール配信施策における課題を効果的に解決できる見通しが立ち、効率的なユーザー再訪施策の実現に向けて、施策を進めることができました。 React Emailを使ったHTML生成 ここからは、実装部分について触れていきたいと思います。 実装としては、メールのHTMLを生成する部分に、React Emailを利用しています。 React Emailのrender関数を使って、JSXからHTMLを生成する流れを採用しました。 まずは、以下のようなコンポーネントを作成します。 import React from "react"; const AppCheckSection = () => { return ( <div style={{ padding: "20px 0", borderBottom: "1px dashed #cccccc" }}> <div> <p> 詳しいスポットの情報やアクセス情報はアプリで確認してみましょう。 <br /> 他にも、アプリではあなたにだけのおすすめスポットを掲載中！ </p> <a style={{ padding: "10px 70px", background: "rgb(17,17,17)", borderRadius: "5px", textAlign: "center", textDecoration: "none", color: "#fff", display: "inline-block", marginBottom: "10px", }} > <span>アプリをチェック</span> </a> <br /> <a href="https://deeplink.sample.hogehoge/"> うまく開かない方はこちら </a> </div> </div> ); }; export default AppCheckSection; このように、メールを構成するための、コンポーネントを作成していきます。 その後、親コンポーネントで、作成したコンポーネントを組み合わせるだけで、メールのテンプレートが完成します import React from 'react'; import AppCheckSection from '../shared/AppCheckSection'; import FooterSection from '../shared/FooterSection'; import RankingHeaderSection from './RrankingHeader'; import RankingItems from './RankingItem'; export type RankingContents = { imageURL: string; name: string; catchPhrase: string; }; export type WeeklyRankingProps = { areaName: string; contents: RankingContents[]; }; const WeeklyRanking: React.FC<WeeklyRankingProps> = ({ areaName, contents }) => { return ( <div style={{ backgroundColor: '#f4f4f4', padding: '20px 0' }}> <div> <RankingHeaderSection /> <RankingItems areaName={areaName} contents={contents} /> <AppCheckSection /> <FooterSection /> </div> </div> ); }; export default WeeklyRanking; そして、メールHTMLの生成には、React Emailのrender関数を使用します。 fetchRegionalRankingDataを用いて、住んでいる地域ごとに、異なるコンテンツ情報を取得して、メールを作成することができます。 import { render } from '@react-email/render'; import { WeeklyRankingEmail } from '../emails/weekly-ranking'; import { fetchRegionalRankingData } from './ranking-service'; export async function generateWeeklyRankingEmail(areaName: string): Promise<string> { const contents = await fetchRegionalRankingData(region); const htmlContent = await render(await WeeklyRanking({ areaName, contents })); return emailHtml; } render 関数で生成された HTML は、SaaS サービスの API を介して送信されるメールの本文として使用されます。 バッチ処理では、EventBridge によってスケジュールされたタイミングで ECS が起動し、メールの作成および送信処理が実行されます。 実際に送信されるメールは、以下のような内容になります。 画像では、関東地域に絞られた内容が表示されていますが、ユーザーが設定した地域に応じて内容を柔軟に変更できる仕組みを実装しています。なので、居住地域が、大阪であれば関西地方のランキングがメールとして配信されます。 そして、React Emailでは、プレビュー機能があり、普通にReactで開発するときと同じように、メールの実装を進めることができます。プレビューなしで実装することは、かなり難易度が高いので、この機能は非常に助かります。この機能を活用し、マーケティング担当の方とレイアウトについて、適宜確認しながら実装作業を進められました。 コンポーネント化により、ランキング以外にもフッターやアプリ起動促進セクションなど、様々な要素を再利用可能なパーツとして構成しました。これにより、新規コンテンツの作成においても既存コンポーネントを組み合わせることで、効率的かつ一貫性のあるメール配信が可能になっています。 スケジューリングされたメール配信では、同じようなコンテンツの繰り返しになってしまい、ユーザーの興味低下や、最悪の場合は迷惑メールに登録されてしまい受信を拒否されてしまう可能性があります。自動化されたシステムであっても、 ユーザーの関心を継続的に引きつけるコンテンツ提供 が求められるはずです。 このような状況を踏まえると、素早く提供するコンテンツの変更が出来るように、コンポーネント化による再利用性の高い設計が、重要だと考えています。 メールを自動配信した効果 React Emailとバッチ処理による自動メール配信を開始した結果、配信した日(2月22日)の付近は、インストール数が増加していました。これは、メールを見た休眠ユーザーが、アプリに興味を持ち、 再インストールを促すことができた と考えています。また、メールを配信日付近のDAUが大きく上昇し、メール自動配信の施策開始以降のDAUは持続的に増加傾向を示していました。 このように、アプリをアンインストールしたユーザーも含む、休眠ユーザーの再訪を促すことに成功しました。 まとめ React Emailを活用した自動メール配信により、人の手を介さずに 休眠ユーザーの掘り起こし や、 DAUを増加 させることに成功しました。 アプリ開発で休眠ユーザーが多く、マーケティングリソースに限りがあるという課題は多くのマーケティング担当者が頭を抱えているのではないでしょうか。React Emailを利用したメール作成の自動化は、メールのコンテンツ内容を毎週考える負荷を軽減し、 効率的で、効果的なマーケティング活動 をすることができます。さらに、コンテンツの改善を継続的に行い、それを 素早くリリース するためにも、「React Email」は非常に有用だと感じました。 また、コミュニケーション手段が多様化した現代であっても、 ユーザーの興味に沿ったコンテンツ を届ければ、マーケティングチャネルとして、メール配信は有効に機能することがわかりました。 再訪ユーザーが伸び悩んでいたり、休眠ユーザーの掘り起こしでお悩みの方は、ぜひこのアプローチを検討してみてはいかがでしょうか。

This article is the entry for day 23 in the KINTO Technologies Advent Calendar 2024 🎅🎄 1. Introduction Hi there! I’m Nishida, a backend engineer at KINTO FACTORY. Today, I’d love to share how we managed to slash our AWS costs. 2. What Made Us Start Working on Cost Reduction? At KINTO Technologies, we use Amazon QuickSight to visualize our AWS usage fees, making it easy to track costs for each product. A little while after launching KINTO FACTORY, we casually checked the project’s costs, and to our surprise, it turned out to be the second most expensive product in the entire company. We certainly didn’t anticipate incurring such high costs so early on. This "Wait, what?!" moment served as the catalyst for our applications team to spring into action and begin working on cost reduction efforts. 3. What We Actually Did Now, let’s dive into the specific cost-cutting steps we took. What We Actually Did When we broke down the costs, ECS Fargate stood out as the clear front-runner in expenses. Not too surprising, since KINTO FACTORY’s applications run on ECS Fargate but still, we figured there had to be ways to optimize it. The first thing that caught our eye was: "Wait... are the number of instances in the development environment the same as production?" That definitely didn’t make sense. The development environment shouldn't require nearly as much computing power. So, we reduced the number of instances in the development environment to only those that were absolutely essential. Upon further investigation, we discovered that there is another Fargate launch type called Fargate Spot that is different from the usual one. Fargate Spot is a system that lets you tap into unused AWS resources, offering discounts of up to 70% compared to the regular Fargate. Honestly, why wouldn’t you use it? Fargate Spot allows Amazon ECS tasks that can handle interruptions to run at a much lower cost. It works by running tasks on AWS’s spare compute capacity. When that capacity is needed elsewhere, tasks are interrupted, with a two-minute warning that is provided to wrap things up. -- https://docs.aws.amazon.com/en_us/AmazonECS/latest/developerguide/fargate-capacity-providers.html That said, as noted in the documentation, Fargate Spot relies on spare compute capacity, so tasks may occasionally be interrupted if AWS reclaims those resources. Applying this to the production environment wasn’t really an option, so the settings were adjusted for the development environment, where occasional interruptions wouldn’t cause any issues. Automating Startup and Shutdown of the Development Environment The development environment had also been running 24/7, just like production environment. To avoid unnecessary costs, it was set up to automatically shut down during off-hours — late nights, weekends, and holidays — when it wasn’t needed. Step Functions were used to automate the start-up and shutdown processes for both the application and the database. In the first stage, EventBridge is used to send start and stop triggers to Step Functions based on schedules set with cron. One key point: database startup takes time. Starting ECS right after the DB starts can lead to connection errors. To prevent that, the system first checks the DB status before launching ECS. For reference, here’s a sample code (YAML file) showing the Step Functions workflow: "DB Startup": { "Type": "Task", "Parameters": { "DbClusterIdentifier": "${db_cluster_identifier}" }, "Resource": "arn:aws:states:::aws-sdk:rds:startDBCluster", "Next": "Wait" }, "Wait": { "Type": "Wait", "Seconds": 300, "Next": "Checking DB status after startup" }, "Checking DB status after startup": { "Type": "Task", "Parameters": { "DbClusterIdentifier": "${db_cluster_identifier}" }, "Resource": "arn:aws:states:::aws-sdk:rds:describeDBClusters", "Next": "Check if startup is complete" }, "Check if startup is complete": { "Type": "Choice", "Choices": [ { "Variable": "$.DbClusters[0].Status", "StringEquals": "available", "Next": "Start up each service" } ], "Default": "Wait" }, Moving to a Serverless Architecture Next, while reviewing the batch processing for a certain feature, it became clear that the process (which took less than a minute) was running as a resident service on ECS. This setup wasn’t exactly cost-friendly too, so we moved to a serverless architecture using Lambda. Lambda, with its pay-as-you-go model based on request count and execution time, is perfect for short, quick processes or tasks that don’t need to run constantly. 4. Cost-Saving Results The efforts paid off quickly, with costs dropping by 65% compared to the peak. Less than half; who would’ve thought? Seeing just how much room there was to cut costs was honestly surprising. 5. Conclusion This time, the focus was on sharing cost reduction efforts. The steps taken weren’t complicated or anything fancy, but they still delivered solid results. Along the way, some valuable insights emerged. Cost reduction isn’t just about saying, “Look, we saved money!” By reviewing and eliminating unnecessary resources, we were able to gain better visibility into the entire system. It also provided a great opportunity to revisit the architecture. Asking questions like, "Is this resource really necessary?" naturally became part of the process. That mindset shift brings long-term benefits, making the effort more than worth it. Hopefully, this can be helpful for anyone facing similar challenges.

こんにちは こんにちは、2024年12月入社のまきどんです！ 本記事では、2024年12月入社のみなさまに、入社直後の感想をお伺いし、まとめてみました。 KINTOテクノロジーズ（以下、KTC）に興味のある方、そして、今回参加下さったメンバーへの振り返りとして有益なコンテンツになればいいなと思います！ Fsk 自己紹介 業務システム開発部 業務システムGでフロントエンド開発を担当しています。 これまで主にNextjsを用いたフロントエンド開発を中心に携わっており、皆さんにとって使いやすい画面づくりを心がけています。 まだまだ学ぶことも多いですが、少しでも皆さんのお役に立てるよう頑張っていきたいと思っています。 所属チームの体制は？ 自分を含めて5人体制です。PM１名、フロントエンドエンジニア２名、バックエンドエンジニア２名です。 KTCへ入社したときの第一印象？ギャップはあった？ copilot、chatGPTなど生成AIを使用するのはすごく助かります。 入社前は少し緊張していましたが、皆さんが温かく迎えてくださったので、すぐに安心しました。 現場の雰囲気はどんな感じ？ 困ったときにすぐ相談できる雰囲気があるのはすごくありがたいです。 ブログを書くことになってどう思った？ 自分の考えや感じていることを皆さんに共有する機会ができて、とても良いと思っています。 フランクさん → Fskさんへの質問 もしロボットに1つだけ面倒な日々のタスクを任せられるとしたら、どの作業を任せたいですか？ 無論、掃除です！毎日掃除に時間を取られるのはもったいないですし、その時間を他のことに使いたいですね。 高橋 自己紹介 オウンドメディアＧ、兼マーケティングプロダクト開発でプロジェクトマネージャーを担当しています。 クライアント/事業部の良き隣人として、また、エンジニアと事業部間の橋渡し役としてメンバーが同じ目標に向かって歩めるように日々活動しています。 前職ではWEBデザイナーとしての経験を積み、開発部へ異動してからは、内製の会員、決済、ポイント、施設情報などプラットフォーム系を兼任するマネージャーやってました。 所属チームの体制は？ オウンドメディアＧはPM1名、エンジニア２名。マーケティングプロダクト開発は静的コンテンツの開発に特化して、チームリーダー1名、PM1名、テックリード1名、エンジニア２名です。 KTCへ入社したときの第一印象？ギャップはあった？ 第一印象はオフィスが静か（前職では営業が同じフロアですぐ近くにいてうるさかった）。 ギャップは開発スタイルがグループごとに異なるのでマインドシフトが柔軟にできるように身構えておく事くらい。 現場の雰囲気はどんな感じ？ 静か。飲料の缶をゴミ箱に捨てるときにちょっと気を遣う。 ブログを書くことになってどう思った？ 社内の自己紹介ではモンハンが好きだしか伝わってなかったと思うので書く機会があってよかったです。 Fskさん → 高橋さんへの質問 ワールドとワイルズ、どっちが好き？ww これからのアップデートを期待してワイルズ！ワールドに負けじと10年以上遊べてほしい！ 現在、生成AIはデザイン分野で活用されていますが、多くのエンジニアが「AIプロンプト」エンジニアと呼ばれています、この現象について、どう考えていますか？ 著作権や肖像権に障らぬように気を付けつつ利用する…であれば、生成AIに任せても良いモノもあると思います。ただ、創造力を競うようなコンクールやコンテストなどの場に出るとなった時はふさわしくないと考えます。 呂 自己紹介 現在、業務システム開発部の業務システムGに所属し、主にバックエンドのシステム開発を担当しています。日々の業務では、社内の業務効率化やデータ連携を目的とした各種システムの設計・実装・運用保守を行っており、安定性と拡張性を重視した開発に取り組んでいます。 以前はIBMに勤務しており、日本国内の大規模な病院向けに医療情報システムの開発プロジェクトに従事していました。要件定義から設計、開発、導入、アフターサポートまで、幅広い工程を経験しており、現場のニーズに寄り添ったシステムづくりを意識してきました。 これまでの経験を活かしつつ、より実用的で価値のあるシステムを提供できるよう、日々技術力と業務理解の向上に努めています。 所属チームの体制は？ 自分を含めて5人体制です。PM１名、フロントエンドエンジニア２名、バックエンドエンジニア２名です。みんなは各領域・業務でのプロでチーム内で非常に勉強できました。 KTCへ入社したときの第一印象？ギャップはあった？ 入社して最初に感じたのは、同僚同士の関係がとても温かく、上下関係にとらわれずに気軽にコミュニケーションが取れる雰囲気があることでした。また、社内イベントが充実していることや、さまざまなサークル活動が活発に行われている点も印象的でした。福利厚生制度も社員に優しく、とても働きやすい環境だと感じました。 入社前に想像していた印象と比べても、大きなギャップはなく、むしろ想像以上に良い職場環境だと実感しています。 現場の雰囲気はどんな感じ？ とても明るく楽しい雰囲気です。日々の業務に関する検討はもちろんのこと、ちょっとした面白い出来事やアイデアも気軽に共有し合える環境が整っています。メンバー同士の距離も近く、誰とでもすぐに打ち解けられるので、安心して働くことができます。 ブログを書くことになってどう思った？ 自分の体験をこうして共有できることを、とても嬉しく思います。日々の業務や感じたことを通じて、少しでも誰かの参考になれば幸いです。 高橋さん → 呂さんへの質問 もしKINTOで車を購入するとしたらどの車に乗りたい？ やはり「クラウン」に乗りたいですね。昔からずっとかっこいいと思っていましたし、実は日頃の業務でテストデータを作成する際にも、この車種をよく使っていて、ちょっとした愛着があります（笑）。 また、社員割引の福利制度を活用すれば、非常に手頃な価格でクラウンに乗れるのも魅力的です。さらに、充実した保険制度など、お客様にも優しいサービスが揃っていて、本当に素晴らしいと感じています。 まきどん 自己紹介 12月入社のまきどんです。モビリティプロダクト開発部のマーケティングプロダクト開発Gに所属しています。業務では分析・機械学習周りを担当しています。主な役割は、データから課題を抽出し施策立案や出口戦略のご提案、機械学習を用いたシステムの設計等でサポートさせて頂いています。 前職ではスタートアップで建築×ITを主軸とした事業のPMを担当していました。 所属チームの体制は？ 私が所属しているデータ分析・ML活用チームはPM/PdM1名、スクラムマスター1名、エンジニア6名の計8名で構成されています。 KTCへ入社したときの第一印象？ギャップはあった？ 第一印象は大企業の傘下のため、やはり堅実なイメージがありました。しかし入社してみるとslackに生成AIが活用されていたり、システムにもAI活用を推進していたりと、想定していたよりずっとテックカンパニーかつベンチャー気質のあるスピード感を持った会社だと感じました。 現場の雰囲気はどんな感じ？ 同じチーム内はもちろんのこと、他部署でも悩んだり困ったことがあればすぐに相談でき、安心して働ける環境です。 ブログを書くことになってどう思った？ 実はエントリーより先に テックブログ を執筆させていただきました。ブログ自体書いた経験が無かったのですが、グループ内で相談・協力してくれたので不安なく書けたのが非常に良い経験になりました。今後も新しい知見や経験を社内外に展開していけるよう頑張ります！ 呂さん → まきどんへの質問 これまでの業務の中で、最も誇りに思っていることは何ですか? 機械学習ツールを用いてアウトプットしていたものを内製化することで、コストダウンとクリック率UPを実現できました！ ニーゼン フランク(Frank Neezen) 自己紹介 ビジネスディベロップメントグループのフランク・ニーゼンです。 正式にはビジネス開発マネージャーという肩書きですが、主な役割はテクニカルアーキテクトとして、グローバルフルサービス製品の設計と実装を支援することです。 私のバックグラウンドはコンサルティングで、Salesforceを活用してお客様の技術的・運用的なニーズに応えるためのアドバイスに注力してきました。 所属チームの体制は？ 体制は自分含め、さまざまなスキルを持つ4人のメンバーで構成されています。開発チームと密に連携し、グローバル向けのフルサービスリース事業ソフトウェアソリューションを開発しています。 KTCへ入社したときの第一印象？ギャップはあった？ アムステルダムのSalesforceから東京のKTCへの異動は驚くほどスムーズでした！当初は文化の違いに慣れることができるか不安もありましたが、素晴らしいオリエンテーションと神保町オフィスの心強いメンバーにより全く問題ありませんでした。初日から温かく歓迎いただいたおかげで、スムーズに馴染むことができました。どちらかと言いますと、一番苦労したのは日本語がまだ話せないため、例えば銀行の手続きや住民登録など、生活の準備を整えることでした。でもKTCのメンバーにはたくさん助けてもらえましたのでなんとかうまくいきました。 現場の雰囲気はどんな感じ？ 我々チームは全員神保町オフィス在中でして、多くのエンジニアと隣り合わせに座っています。雰囲気はオープンでプロフェッショナルですが、リラックスできる環境です。良いチーム感があり、みんな自分の仕事を成功させたいと思っているメンバーばかりです。 ブログを書くことになってどう思った？ 過去にSalesforce関連の記事を書いたことがあります。KTCに入社するまでの個人的なストーリーを書いたりなど、共有できるところは喜んでシェアしていきたいと思います！ まきどん → フランクさんへの質問 日本へ来て驚いたことがあれば教えて下さい！ 日本の安全さには本当に驚かされます。世界最大の都市である東京のどこを歩いていても、全く不安を感じません！さらに本当に驚くべきことは、財布や携帯電話などをなくしても、ほとんどの場合手元に戻ってくることです。自分がなくしたことにすら気づいていなかったのに、誰かが私のところに来て、失くした物を届けてくれたことも何度かありました。本当に新鮮で素晴らしい体験です！ さいごに みなさま、入社後の感想を教えてくださり、ありがとうございました！ KINTO テクノロジーズでは日々、新たなメンバーが増えています！ 今後もいろんな部署のいろんな方々の入社エントリが増えていきますので、楽しみにしていただけましたら幸いです。 そして、KINTO テクノロジーズでは、まだまださまざまな部署・職種で一緒に働ける仲間を募集しています！ 詳しくは こちら からご確認ください！

Hello, this is HOKA from the Organizational Human Resources Team in the Human Resources Group. This blog is a continuation of Overcoming Challenges in the In-House Rollout of the 10X Innovation Culture Program [Part 1] August: Lessons learned from Speaking at Google Cloud Next Tokyo '24 Google Cloud Next Tokyo '24, held by Google Cloud Japan, LLC on August 1–2, 2024, is a flagship event bringing together business leaders, innovators, and engineers for a major cloud conference. The event featured a variety of programs, including keynote speeches, live sessions, and hands-on sessions, all centered around key business topics like generative AI and security.  For more details, click here -> https://cloudonair.withgoogle.com/events/next-tokyo-24 From our company, Kisshi, General Manager of the Development Support Division, and Awache, Manager of the Database Reliability Engineering Group, took the stage at the 10X Innovation Culture Program's experience workshop. The workshop was a great opportunity to highlight real-world examples of the 10X Innovation Culture Program in action. Leading up to the actual event, we had numerous meetings, rehearsed at the Google office, and had lunch together, giving us plenty of time to interact with people from Google. And then the real thing began. The extraordinary atmosphere of the spacious venue at Pacifico Yokohama naturally created a feeling of "let’s learn something new today" kind of mindset. First, a representative from Google introduced us to 10X. The speaker shared, in down-to-earth and relatable words, the background, reasons, and results behind Google’s promotion of the 10X Innovation Culture Program both inside and outside the company. Hearing this, the KINTO Technologies (KTC) members in the audience couldn’t help but think, "10X is exactly what we need too!" and the energy in the room instantly lifted. After that, Kisshi and Awache took the stage. Kisshi and Awache talked about the following three topics: The background to the cultural transformation initiative and why we decided to implement the 10X Innovation Culture Program Key takeaways and important points learned from actually implementing the program Future plans and next steps In particular, they shared the following key takeaways from their experience implementing the 10X Innovation Culture Program: It's important to discard the notion that "only Google can do it" and just give it a try without overthinking it. It is important to develop internal facilitators to achieve in-house independence from the bottom up. Culture is not something that is created in one leap, but rather something that is noticed and experienced many times before becoming ingrained. Many questions came up during the Q&A session. One of them was, “10X sounds amazing, but isn’t it difficult to actually implement?” While significant results from the 10X Innovation Culture Program have yet to emerge, we hope these efforts can provide at least a little inspiration. Having attended the Google Cloud Next Tokyo '24, I felt an overwhelming difference from the 10X Innovation Culture Program created and held internally in July. What stood out were: A dedicated venue, separate from the office, puts participants in the right mindset to truly listen. Hearing "10X" directly from Google employees carries real weight and resonates deeply. The event felt discussion-heavy, but perhaps a stronger focus on input and learning would be even more impactful. With that foundation, participants may naturally start thinking, "I want to bring 10X thinking into my daily work." Having these as hypotheses, plans are in place to put them to the test at the 10X Innovation Culture Program this fall. September: Participating in the 10X Facilitator Training First, we decided to start by training facilitators who can guide smooth understanding and discussion of 10X concepts. Our hypotheses were: "10X" when shared by Google employees, resonates strongly. What seemed like a discussion-focused setting turned out to need more input. The reason? Those running the 10x program didn’t have enough clarity or input themselves on what it really is, making it tough to convey to others. After reaching out to Google for advice, they kindly agreed to set up facilitation trainings. Selecting members for the facilitation training was a challenge. But since the 10X Innovation Culture Program is, at its core, a leadership program, managers were chosen to participate. Facilitator Training Day At the beginning of the training, Kota-san from Google, who is the main person in charge of this session, said, "It's going to be extremely intense." The goal is to hear something once, remember it, and be able to say it yourself. It’s designed to let you experience two phrases that Google lives by: Steal with pride Feedback is a gift! "You’re about to spend more time thinking about 10X than Google employees do", was what we were told. "What on earth is about to begin?" The training kicked off in that atmosphere. [First half of the training] Input Time Just as Kota-san mentioned, the first part of the session was all about learning. A Googler gave a presentation on the six key elements of 10X, and KTC employees had some solid "input time" to really understand each one. Take the first element, DEI, for example. At Google, 50% of employees have taken the training Unconscious Bias @ Work , which helps raise awareness of unconscious bias. On top of that, they run weekly Googlegeist to track how well they’re doing in areas like inclusiveness and equity. We got to hear not just about these concrete initiatives, but also the impact they’ve had. Similarly, for each of the six 10X elements, the presenter shared real examples from Google, along with their own personal thoughts and stories from working there. [Second half of the training] Output Time After hearing the presentation from Google, the KTC members split into groups of six. Each person took one of the six elements, memorized it, and gave a presentation that included their own personal story or experience. The listeners weren’t just passive after each presentation, everyone gave feedback on what worked well and what could be improved. We also received feedback from the Google team. The whole process might’ve felt a bit intense, just like Kota-san warned, but it was the perfect chance to "steal with pride" from Google’s approach and truly embrace the idea that "feedback is a gift."  And the feedback wasn’t just technical. Ideas started flowing, too: "We should be sharing the vision more across the whole company," "What’s the current status of our goal-setting process?", "I want to be able to give more concrete examples: let’s increase opportunities to talk directly with executives!" and so on. More and more ideas bubbled up. By learning about 10X, we got to witness that “aha!” moment when people started comparing it to where KTC stands today. In fact, after the training, a lot of KTC members shared feedback such as: "I always thought 10X was great, but it felt like something distant, like it was a distant thing. But by memorizing it, tying it to my own experiences and presenting it to others in such a short time, it suddenly became something I truly owned." And getting immediate feedback from the Google team on the spot was such a valuable opportunity. October: Preparing for the Third 10X Innovation Culture Program I wrote this blog in October, after completing the facilitator training and while gearing up for the third round of the 10X Innovation Culture Program. The third 10X iteration will be divided into two days. It's a brand-new approach for us. Additionally, 47% of participants this time are joining for the first time. Will they be even more eager and engaged than the group in July? Will all the effort we put into the September’s facilitator training pay off? I’ll share how it all turns out in a follow-up post on the TechBlog. To be continued in the 10X Innovation Culture Program: The Struggles and Challenges of Building It In-House [Part 2], coming January 2025.

Introduction Managing assets in a multi-package Flutter project can get tricky, especially when it comes to loading local JSON files. This requires a different approach than a normal single package project, leaving developers scratching their heads. In this article, we’ll break down how to load local JSON files effectively in a multi-package Flutter project. This article is the entry for day 23 in the KINTO Technologies Advent Calendar 2024 🎅🎄 Test Project Setup For this study, we prepared a simple project managed with multiple packages. Here’s how it’s structured: 🎯 Dart SDK: 3.5.4 🪽 Flutter SDK: 3.24.5 🧰 melos: 6.2.0 ├── .github ├── .vscode ├── app/ │ ├── android/ │ ├── ios/ │ ├── lib/ │ │ └── main.dart │ └── pubspec.yaml ├── packages/ │ ├── features/ │ │ ├── assets/ │ │ │ └── sample.json │ │ ├── lib/ │ │ │ └── package1.dart │ │ └── pubspec.yaml │ ├── .../ ├── analysis_options.yaml ├── melos.yaml ├── pubspec.yaml └── README.md Load a File in Asset You’ll often see explanations showing that, in a typical single-package project, you can load assets like this: flutter: assets: - assets/ # Specify the assets folder import 'package:flutter/services.dart' show rootBundle; Future<String> loadAsset() async { return await rootBundle.loadString('assets/sample.json'); } The official explanation is pretty much the same. https://docs.flutter.dev/ui/assets/assets-and-images I built a simple Widget that simply reads a JSON file from an Asset and displays it as a String in a Text widget. import 'dart:convert'; import 'package:flutter/material.dart'; import 'package:flutter/services.dart' show rootBundle; class LocalAssetPage extends StatefulWidget { const LocalAssetPage({super.key}); @override LocalAssetPageState createState() => LocalAssetPageState(); } class LocalAssetPageState extends State<LocalAssetPage> { String _jsonContent = ''; @override void initState() { super.initState(); _loadJson(); } Future<void> _loadJson() async { final response = await rootBundle.loadString('assets/sample.json'); final data = await json.decode(response); setState(() { _jsonContent = json.encode(data); }); } @override Widget build(BuildContext context) { return Scaffold( appBar: AppBar( title: const Text('Local Asset Page'), ), body: Center( child: _jsonContent.isEmpty ? const CircularProgressIndicator() : Text(_jsonContent), ), ); } } But this method doesn’t work for loading assets across multiple packages. 🤨 Loading Assets with flutter_gen In most cases, loading Assets across multiple packages can be handled easily with flutter_gen. flutter_gen is a tool that generates code from Asset paths and localization files, enabling type-safe asset loading. It also natively supports loading Assets across multiple packages. https://github.com/FlutterGen/flutter_gen To load Assets from multiple packages with flutter_gen, the following settings are required. flutter_gen: assets: outputs: package_parameter_enabled: true With this setting, running flutter_gen will generate code for loading assets from multiple packages. You can then use this generated code to load Assets in a type-safe way. Here’s how the previous example looks when rewritten using flutter_gen: import 'package:{YOUR_PACKAGE_NAME}/gen/assets.gen.dart'; Future<String> loadAsset() async { return await rootBundle.loadString(Assets.sample); } Assets from multiple packages can be loaded with code like this: import 'dart:convert'; + import 'package:feature_flutter_gen_sample/gen/assets.gen.dart'; import 'package:flutter/material.dart'; import 'package:flutter/services.dart' show rootBundle; class FlutterGenSamplePage extends StatefulWidget { const FlutterGenSamplePage({super.key}); @override FlutterGenSamplePageState createState() => FlutterGenSamplePageState(); } class FlutterGenSamplePageState extends State<FlutterGenSamplePage> { String _jsonContent = ''; @override void initState() { super.initState(); _loadJson(); } Future<void> _loadJson() async { + final response = await rootBundle.loadString(Assets.sample); - final response = await rootBundle.loadString('assets/sample.json'); final data = await json.decode(response); setState(() { _jsonContent = data.toString(); }); } @override Widget build(BuildContext context) { return Scaffold( appBar: AppBar( title: const Text('FlutterGen Sample'), ), body: Center( child: _jsonContent.isNotEmpty ? Text(_jsonContent) : const CircularProgressIndicator(), ), ); } } The file paths are structured, which makes things much neater! Asset management for team development is now a breeze. But sometimes, you might prefer not to rely on tools too much, right? Let’s look at how to handle that next. Loading Assets without flutter_gen Of course, you can also load Assets from multiple packages without flutter_gen. In that case, you can load Assets by specifying the path as follows. :::message packages/ {package name} / {folder path} /file name ::: Package name is the name specified in the pubspec.yaml of the package in which the asset is stored. Folder path is the path specified under assets in that Package's pubspec.yaml. name: local_asset ... flutter: assets: - assets/ import 'dart:convert'; import 'package:flutter/material.dart'; import 'package:flutter/services.dart' show rootBundle; class LocalAssetPage extends StatefulWidget { const LocalAssetPage({super.key}); @override LocalAssetPageState createState() => LocalAssetPageState(); } class LocalAssetPageState extends State<LocalAssetPage> { String _jsonContent = ''; @override void initState() { super.initState(); _loadJson(); } Future<void> _loadJson() async { + final response = await rootBundle.loadString('packages/local_asset/assets/sample.json'); - final response = await rootBundle.loadString('assets/sample.json'); final data = await json.decode(response); setState(() { _jsonContent = json.encode(data); }); } @override Widget build(BuildContext context) { return Scaffold( appBar: AppBar( title: const Text('Local Asset Page'), ), body: Center( child: _jsonContent.isEmpty ? const CircularProgressIndicator() : Text(_jsonContent), ), ); } } This way, you can load Assets from multiple packages without flutter_gen. For small projects or solo development, this approach should work just fine. Not gonna lie; without fully understanding these path rules, there was a lot of trial and error: trying relative paths, tweaking settings... And hitting build errors over and over. It was a struggle. About Paths Specified in pubspec.yaml When managing assets locally, pay close attention to the paths you specify in pubspec.yaml. Assets paths are set relative to pubspec.yaml, but watch out for differences between: /assets and /assets/{subfolder} . For example, if you move a JSON file into a subfolder like this: ├── packages/ │ ├── features/ │ │ ├── assets/ │ │ │ └── jsons/ │ │ │ └── sample.json <<< HERE │ │ ├── lib/ │ │ │ └── package1.dart │ │ └── pubspec.yaml │ ├── .../ I assumed that specifying /assets would also include files in /assets/{subfolder} , but even after changing the loadString path to packages/local_asset/assets/jsons/sample.json , it still wouldn't load. When you move files into a subfolder, you need to explicitly include that subfolder in pubspec.yaml, like this: flutter: assets: - /assets/jsons/ Now, you can load packages/local_asset/assets/jsons/sample.json . If you manage assets in subfolders, make sure to add those subfolders to pubspec.yaml. By the way, the examples so far have used assets folder, but you can also change the name of it. As long as the pubspec.yaml and actual path configuration match, assets can also be loaded without any problems from folders other than assets folder. Summary This time, we covered how to load local JSON files in a Flutter multi-package setup. Since my main focus is iOS development, I thought it would be easy, but it turned out to be more challenging than expected. There’s limited information available on working in multi-package environments, so I'd like to share more insights as I continue learning in the future.

KINTOテクノロジーズで my route(iOS) を開発しているRyommです。 CIクレジットの節約のため、 いくつか取り組んできたことを紹介します。 はじめに 弊プロジェクトにおいては、CIツールとしてBitriseを利用しています。 昨年は通常のユニットテストに加えて スナップショットテストを導入 したり、 SPMに移行 したりしました。 気付くとBitrise CIの1回あたりの実行時間が多いときは約25分ほどに膨れ上がり、多くの実装が行われた月は予算を超過してしまうこともしばしば発生するようになってしまいました。 Bitriseは契約分を超過すると高額になってしまうので、執筆時点の為替レートでは超過分のCI1回の実行に約400円ほど掛かっている計算です。たっっか！ そういうわけで、クレジットが超過しそうになるとCIを動かさないためにPRのマージも必要最低限に抑える動きが生まれてしまっていました。 この状況を打破すべく取り組んできた、弊プロジェクトにおけるクレジット節約術を紹介します。 CLIツールのセットアップを見直す BitriseのBuild結果を見ると、どのステップにどのくらい時間が掛かったのか見ることができます。 BitriseのBuild結果 これを見ると、「Script Runner」にて12分も掛かっていることがわかります。 これは、swiftLintやLicensePlistのセットアップを行っているステップです。 以前執筆した記事 にて紹介しましたが、Build Phaseで実行するためにworkspaceとは独立して作成したパッケージにて、ライブラリを落として使えるようにしています。 ま〜たしかにこれは時間かかるよね〜というところなので、短縮していきます。 幸いここで使いたいライブラリはBuild Tool Pluginに対応しているため、そちらに移し替えていくことでこのステップを省くことができます。 元々 license_plist.yml や .swiftlint.yml などの設定は済んでいるため、ただプロジェクトの Package Dependencies にパッケージを追加し、ターゲットの Build Phase の Run Build Tool Plug-ins にプラグインを追加してあげればOKです。 Build Phaseの設定 LicensePlistはプラグインだと outputPath の場所の指定が効かないため、 README にあるように Settings.bundle の下にライセンスのファイルを移動するようBuildPhaseに含める必要があります。 また、パッケージはFrameworksでリンクしているパッケージではなく、アプリ本体に含める必要があります。 これで「Script Runner」のステップが丸々不要になったので、12分の短縮...そして消費クレジットも半分になりました！🎉 BitriseのBuild結果 さらにプロジェクト構成が単純化し、セットアップやバージョン更新の際に別途シェルを実行するような必要もなくなりました。 今回のケースでは全てBuild Tool Pluginに対応していたため構成ごと変更しましたが、別のアプローチとして nest も試しました。こちらは既存のCLIツールを別パッケージに分けて管理する構成のまま、CI時間を短縮させることができます。 tools ディレクトリ配下にあったCLIツールをインストールするためのパッケージをnestに置き換えます。 Project/ ├── Hoge.xcworkspace ├── Hoge.xcodeproj ├── Test/ │ └── ... ├── ... └── tools └── nextfile.yaml // ここを置き換える nest bootstrap nestfile.yaml を実行すると tools/.nest/bin 内にバイナリがインストールされていることが確認できるので、これを Build Phase で実行されるように設定します。 Build Phaseでswiftlintを設定 Build Tool Plugin に対応していなかったりする場合には有用かもしれません。 テストを見直す 弊プロジェクトでは、1つのテストターゲット内に全てのテストが詰め込まれていたため、常に全てのテストが実行されていました。 またその中でもスナップショットテストは全て実行すると1時間ほど掛かる非常に重いテストのため、リファレンス画像と比較するメソッドはCI上ではコメントアウトして実行されないようにしていました。しかし、比較前の非同期の描画処理の待機などは実行されてしまうため、失敗時はタイムアウトが積もり積もって長い時間待つこととなり、これもクレジットを食い潰す要因の一つとなっていました。 そこで、CI上では動かしていないスナップショットテストを別のテストターゲットへ切り離し、TestPlanを使って実行するテストをコントロールするようにしてみました。 まずは、時間のかかるスナップショット用のテストターゲットを作成します。 テストターゲットを作成 既存のテストターゲットを参考にターゲットの設定を行ったら、 Build Phases の Compile Sources もしくは各テストファイルの File inspector より Target Membership から、スナップショットテストを新規作成したターゲットへ移していきます。 このとき、移動したテストファイルが元のテストターゲット内のテストファイルと依存関係があるとビルドできなくなるため、都度依存を切り離していきます。 ターゲットを変更していく 次に、TestPlanを作成します。 TestPlanとは、実行するテストとテストの設定をまとめたものです。その際、実行するテストを指定できる範囲はテストターゲット単位です。 このためにテストターゲットを分離させました。 TestPlanはスキーマに紐づけることができ、弊アプリではスキーマとTestPlanが1対1になるようにしています。 そしてCI上で使用するためのスキーマ用のTestPlanにおいては、スナップショットテストを実行しないようにします。 TestPlanの設定 実際に動かしてみると、CI上では失敗しない限り大きくは実行時間は変わりません。しかし、ローカルにおけるテスト体験は大きく改善しました。ロジックのみの変更でもスナップショットテストが実行されてしまっていたのが、チェックを外すだけで実行しないようにできるため、大幅な時間短縮が実現しました。 元々CI上では動かしていなかったテストをそもそも実行しないようにした形ですが、クレジット利用状況とのバランスを調整した上でスナップショットテストも実行できるようにしていきたいと考えています。 おわりに ここに紹介したもの以外にも、型推論に時間が掛かっているコードを修正したり、使用していないアセットを削除することなどもビルド時間の短縮につながります。 これらの取り組みによってCI1回あたりの時間は平均約22分程度から12分程度まで短縮され、45%ほどのクレジット節約が実現しました。 今回はすぐにできる範囲としてビルド前後の時間短縮が主でしたが、次はビルド自体の時間ももっと短縮したいです。

1. Introduction Hello, this is Torii ( @yu_torii ) from the Common Service Development Group. I'm a fullstack software engineer, primarily working on both backend and frontend development. As part of the KINTO Member Platform Development Team, I focus on frontend engineering while also contributing to internal initiatives involving generative AI. This article introduces Sherpa, our internal chatbot powered by LLM and integrated into Slack. We’ll explore its RAG capabilities and its translation function, which works through Slack reactions. Sherpa is designed to facilitate generative AI adoption within the company by enabling employees to naturally interact with AI in their daily Slack conversations. The goal is to eliminate the need for frontend development and deploy it quickly within the company, enhancing efficiency and collaboration. The name Sherpa is inspired by the Sherpa people, known for guiding climbers on Mount Everest. Just as Sherpas are reliable supporters for mountaineers, Sherpa aims to be a dependable assistant for improving work efficiency and streamlining information sharing within the company. By the way, the Sherpa on the banner was generated by team members of the Creative Group as a surprise reveal for the KINTO Technologies General Meeting (KTC CHO All-Hands Meeting). A big thank you to them! This initiative was carried out in collaboration with Wada-san ( @cognac_n ), who is leading the Generative AI Development Project. We have been driving the adoption of generative AI within the company through various improvements, such as introducing a RAG pipeline, setting up a local development environment, and implementing a translation and summarization feature using Slack emoji reactions. :::details Click here for Wada-san's article @ card @ card @ card ::: Note that this article does not go into detail about RAG or generative AI technology itself. Instead, it focuses on the implementation process and feature enhancements. Additionally, Sherpa's LLM runs on Azure OpenAI. What You Will Gain from This Article How to use chat functions powered by generative AI in a Slack Bot This article introduces an implementation example of a chatbot that combines Slack and LLM, allowing users to trigger translations and summaries using emoji reactions or natural message posts. Techniques for retrieving Confluence data, including HTML sanitization Learn how to fetch Confluence documents using Go and sanitize HTML to prepare text for summarization and embedding. Implementing a simple RAG pipeline using FAISS and S3 This section explains the steps and considerations for setting up a simple RAG pipeline using the FAISS indexing and S3. While the response speed may not be very fast, this approach provides a cost-effective way to integrate a basic RAG system. :::message This article is based on the implementation status at the time of writing. Please note that there are still many areas for improvement. Since this is an internal tool, it is not designed for production-level performance. Additionally, this article does not cover details on setting up the development environment, such as deployment with AWS SAM, building a pipeline with Step Functions, or configuring CI/CD with GitHub Actions. ::: :::details Table of Contents (Expanded) 1. Introduction What You Will Gain from This Article 2. Overview of the Overall Architecture 3. AI-Powered Chat Function Using Slack Bot Chat Function and Reaction Function Chat Function: Usage Scenario Reaction Feature: Usage Scenario Benefits of This Configuration 3.5 Actual Use Cases Summary of Chapter 3 4. Implementation Policy and Internal Design Overall Process Flow Function Determination by Conditional Branching The Role of Sanitization Limiting RAG Usage to Confluence References Considerations for Scalability and Maintainability 5. Introduction to Code examples and Configuration Files 5.1 [Go] Receiving and Parsing Slack Events 5.2 [Go] HTML Text Sanitization 5.3 [Python] Example of an LLM Query 5.4 [Python] Example of a RAG Search Call 5.5 [Python] Embedding and FAISS Indexing ::: 2. Overview of the Overall Architecture Here, we will explain the process by which Sherpa returns answers using generative AI from two perspectives: - The generative AI chat function with users - The process of indexing Confluence documents. Processing Generative AI Chat Functions with Users Calling Sherpa on Slack There are two ways to call Sherpa: via chat or with a reaction emoji. Users can call Sherpa by mentioning it in a channel or sending a direct message to request a generative AI response. A reaction call allows you to request a translation or summary of a message by adding a translation/summary emoji reaction. Go Slack Bot Lambda The bot that handles Slack events is built with Go and runs on AWS Lambda. It receives questions and reactions, then determines processing based on the request type. When using Azure OpenAI for LLM queries or retrieving embedded data, this component generates requests and forwards them to Python Lambda. Request to LLM and RAG reference in Python Lambda Python Lambda is divided into functions responsible for LLM queries, RAG references, and related processes. It receives requests from Go Lambda, queries the LLM, and generates an answer using RAG. Returning an answer to Slack The generated answer is sent back to Slack via Go Slack Bot Lambda. The translation and summary functions that can be invoked via emoji reactions are also integrated into this flow. :::details Architecture diagram (processing user requests) flowchart LR subgraph "Processing user requests" A["User(Slack)"] -->|"Question・Emoji"| B["Go Slack Bot(Lambda)"] B -->|"Request"| C["Python Lambda RAG/LLM"] C -->|"Answer"| B B -->|"Answer"| A end ::: The Process of Indexing Confluence Documents To use the RAG pipeline, you need to prepare your Confluence documents in a way that makes them easy to summarize and embed. These preprocessing steps are structured into workflows using StepFunctions and are executed automatically on a scheduled basis. Document retrieval and HTML sanitization (Go implementation) A Lambda function implemented in Go retrieves documents from the Confluence API, cleans up HTML tags, and makes the text more manageable. The sanitized text is output as JSON. Summary processing (Go + Python Lambda invocation) Summarization aims to refine the text, making it easier to process with Embedding and RAG. The Go implementation of Lambda invokes a Python Lambda that processes requests to the Azure OpenAI Chat API, shortens the text, and converts it back to JSON. FAISS indexing and S3 storage (Indexer Lambda) The Indexer Lambda embeds the summarized text and generates a FAISS index, then stores the index and meta information in S3. This enables instant retrieval of indexed data upon a query, ensuring the RAG pipeline runs smoothly. :::details Architecture diagram (Confluence document indexing flow) flowchart LR subgraph "Indexing Preparation StepFunctions" D["Go Lambda (Document Retrieval/HTML Sanitization)"] D --> E["Go Lambda (summary/Python call)"] E --> F["Indexer Lambda(Embedding/FAISS/S3)"] end ::: By combining this pre-processing with request-time processing, Sherpa enables generative AI responses that incorporate company-specific knowledge with simple operations in Slack. In the following chapters, we'll dive deeper into these components. 3. AI-Powered Chat Function Using Slack Bot The previous chapter provided an overview of the architecture. In this chapter, we’ll focus on how users can make use of Sherpa’s features with simple, natural interactions in Slack, and how these features can benefit them. Here, we will illustrate what Sherpa can do and which scenarios it can be useful in, while the next chapters will systematically explain the implementation details. Chat Function and Reaction Function Sherpa offers a variety of generative AI capabilities, powered by natural interactions within Slack. Chat function : By posting a question, attaching files or images, including external links, or sending a message in a specific format, users can trigger LLM queries or, in certain cases, RAG searches to obtain relevant answers. By integrating AI into Slack, an everyday tool, users can seamlessly adopt generative AI without the need to learn new environments or commands. Reaction function : By simply adding specific emoji reactions to a message, users can trigger translations, delete messages, and perform other actions—enabling additional operations without requiring commands. Chat Function: Usage Scenario Basic Questions and Answers Simply posting a question allows users to receive AI-generated responses through LLM. Example scenario: Asking "What are the steps for this project?" provides an instant answer, taking into account thread history and speaker context for a more accurate response. File, image, and external link processing Attaching a file and asking "Summarize this" will generate a summary of the document. Uploading an image allows Sherpa to extract text and provide relevant answers. Sharing an external link enables the bot to analyze and summarize webpage content, incorporating it into the LLM-generated response. Example scenario: -Get a short summary of meeting notes from a text file. -Extract text information from an image. -Generate a concise summary of an external article. Confluence page lookup (RAG integration) :confluence: By using index:Index name , users can perform a RAG-based search on Confluence pages containing company documentation, such as internal rules and application procedures. Example scenario: If company policies and procedures are documented in Confluence, users can instantly access relevant information tailored to internal workflows. Example scenario: Easily retrieve project-specific settings and instructions. that would otherwise be difficult to search for. Reaction Feature: Usage Scenario Adding specific emoji reactions makes calling up functions even more intuitive. Translation : Adding a translation emoji automatically translates the message into the specified language, helping break down language barriers and improve communication. Message deletion Unnecessary Sherpa responses can be deleted instantly by adding a single emoji, keeping Slack channels organized. Benefits of This Configuration Seamless and intuitive AI usage : Users can use AI without needing new commands—they simply interact with Slack as they normally do (e.g., posting messages, adding emojis reactions), reducing learning costs. Embedding AI into everyday tool (Slack) : By integrating generative AI directly into Slack, AI can be naturally incorporated into daily workflows without friction. Additionally, reaction-based interactions enable users to perform actions like translation and deletion without typing commands, making AI even more accessible. Scalable and easily extendable : If new models or additional features need to be introduced, the existing flow (chat-based queries and emoji interactions) can be easily expanded by adding conditions. 3.5 Actual Use Cases Below are some real-world examples of how Sherpa is used within Slack. Example 1: Image Recognition When you attach an image to a message, Sherpa recognizes the text within the image and responds with its content.  Image Recognition Example 2: Answers Based on Confluence Documentation By using :confluence: emoji, users can retrieve answers based on the Confluence documentation.  Answer based on Confluence documentation As shown above, Sherpa can be triggered from workflows, allowing it to function similarly to a prompt store. Example 3: Requesting English Translations via the Translation Reaction Add translation reactions to messages that users want to translate into English To share a Japanese message with an English speaker, simply add the :sherpa_translate_to_english: reaction emoji. This will automatically translate the message into English.  English translation reaction To prevent misunderstandings and ensure users do not overly rely on automatic translations, a notification in multiple languages clarifies that the translation was AI-generated. Additionally, we provide instructions on how to use the feature to encourage adoption. Besides English, Sherpa supports translation into multiple languages. Summary of Chapter 3 In this chapter, we focused on the user perspective—"what can be done with what operations." Building on the usage scenarios discussed here, the following chapters will systematically explain the implementation details. We will cover: - How to integrate Go-based bot with Python Lambdas. - How Slack events are processed and how RAG search works. - The details of the file extraction and sanitization processes. 4. Implementation Policy and Internal Design In this chapter, we will explain the implementation policy and internal design behind how Sherpa provides a variety of generative AI functions through Slack messages and reactions. This section focuses on the overall concept, role distribution, and scalability considerations. Specific code snippets and configuration files will be provided in the next chapter. Overall Process Flow Slack event reception (Go Lambda) Events occurring in Slack—such as message posts, file attachments, image uploads, external link insertions, and emoji reactions—are sent to an AWS Lambda function written in Go via the Slack API. The Go function then analyzes these events and determines the appropriate action based on user intent (e.g., normal chat, translation, Confluence reference). Text processing and sanitization on the Go side Text is extracted from external links or files and added to the prompt as context. When referencing external links, meaningful tags such as table , ol , ul are preserved while unnecessary tags are removed to optimize token usage. LLM queries and RAG searches on the Python side If necessary, the Go function invokes a Python Lambda for LLM queries or a separate Python Lambda for RAG searches (e.g., for Confluence references). For example, if :confluence: is included in the request, the Go function calls the RAG search Lambda. If no index is specified, it defaults to the primary index. Otherwise, the text is passed to the LLM Lambda for standard query processing. Reply and display in Slack The Python Lambda returns the generated response to the Go function, which then posts it back to Slack. This enables users to access advanced features through familiar Slack interactions—such as emojis, keywords, and file attachments—without needing to memorize commands. Function Determination by Conditional Branching Processing is routed based on specific emojis (e.g., :confluence: ), keywords, the presence of files or images, and whether an external link is included. To add new features, simply introduce new conditions on the Go side and, if necessary, extend the logic for invoking the corresponding Python Lambdas (e.g., for LLM or RAG tasks). The Role of Sanitization Sanitizing on the Go side removes unnecessary HTML tags to improve token efficiency and ensure clean input for the model. Key structural elements such as table, ol, and ul are retained to preserve the information structure and maintain useful context for the model. Limiting RAG Usage to Confluence References RAG search is only performed when explicitly specified with :confluence: . By default, summarization, translation, and Q&A tasks are handled via direct LLM queries, ensuring RAG logic is triggered only for Confluence references. Embedding generation for Confluence documents and FAISS index updates are handled periodically by StepFunctions, ensuring that the latest index is always available for queries. Considerations for Scalability and Maintainability Conditional branching based on emojis, keywords, or the presence of files/images minimizes the number of code modifications required when introducing new features, enhancing maintainability. The separation of concerns—where text formatting and sanitization are handled on the Go side, while LLM queries and RAG searches are managed on the Python side—improves code clarity and facilitates future model replacements or additional processing logic. In the next chapter, we will introduce specific code snippets and configuration examples based on these design principles. 5. Introduction to Code examples and Configuration Files This chapter introduces a brief implementation example based on the implementation policy and design concepts explained in Chapter 4. This chapter contains the following sections: 5.1 [Go] Receiving and parsing Slack events Explains how to use Slack's Events API to receive and process events such as messages and emoji reactions. 5.2 [Go] HTML text sanitization Provides an example of sanitizing HTML when referencing external links. 5.3 [Python] Example of an LLM query Shows how to query an LLM using a Python-based Lambda function. 5.4 [Python] Example of a RAG search call Demonstrates how to perform a RAG search call, such as for Confluence lookups. 5.5 [Python] Embedding and FAISS indexing. Provides an example of Lambda code that periodically embeds Confluence documents and updates the FAISS index. 5.1 [Go] Receiving and Parsing Slack Events This section explains the basic steps for using the Slack Events API to receive and analyze events with Go code on AWS Lambda. We will also cover the settings on the Slack side (OAuth & Permissions, event subscription) and how to check the scopes required when using the chat.postMessage method (such as chat:write ), to clarify the necessary preparations before implementation. Configuration Steps on Slack Side Create an app and check the App ID : Create a new app at https://api.slack.com/apps . Once created, find your App ID (a string starting with A ) on the Basic Information page ( https://api.slack.com/apps/APP_ID/general , where APP_ID is a unique ID for your app). This App ID identifies your Slack App and can be used to access the URLs for OAuth & Permissions and Event Subscriptions pages described below. Granting scopes via OAuth & Permissions : Visit the OAuth & Permissions page ( https://api.slack.com/apps/APP_ID/oauth ) and add the necessary scopes to Bot Token Scopes. For example, if the chat.postMessage method is needed to post messages to a channel, checking this page ( https://api.slack.com/methods/chat.postMessage ) under "Required scopes" will indicate that chat:write is required. After granting the scope, click "reinstall your app" to apply the changes in your workspace. Then, the changes will be reflected.  Checking required scopes  Setting scope Enabling the Events API and subscribing to events : Enable the Events API on the "Event Subscriptions" page ( https://api.slack.com/apps/APP_ID/event-subscriptions ) and set the AWS Lambda endpoint described below in "Request URL". Add the events you want to subscribe to, such as message.channels or reaction_added . This allows Slack to send a notification to the specified URL whenever a subscribed event occurs.  Event Reception and Analysis on AWS Lambda Once the configuration is complete on the Slack side, Slack will send a POST request to AWS Lambda via API Gateway whenever a subscribed event occurs. Step 1: Parsing Slack Events Use the slack-go/slackevents package to parse the received JSON into an EventsAPIEvent structure. This makes it easier to identify event types, such as URL validation and CallbackEvent. func parseSlackEvent(body string) (*slackevents.EventsAPIEvent, error) { event, err := slackevents.ParseEvent(json.RawMessage(body), slackevents.OptionNoVerifyToken()) if err != nil { return nil, fmt.Errorf("Failed to parse Slack event: %w", err) } return &event, nil } @ card Step 2: Handling URL Verification Requests When setting up the integration, Slack will initially send an event with type=url_verification . To verify the URL, simply return the challenge value as received. Once verified, Slack will continue sending event notifications. func handleURLVerification(body string) (events.APIGatewayProxyResponse, error) { var r struct { Challenge string `json:"challenge"` } if err := json.Unmarshal([]byte(body), &r); err != nil { return createErrorResponse(400, err) } return events.APIGatewayProxyResponse{ StatusCode: 200, Body: r.Challenge, }, nil } @ card Step 3: Verifying Signatures and Ignoring Retry Requests Slack includes a request signature that allows verification of authenticity (implementation omitted). Additionally, in case of a failure or outage, Slack may resend the request as a retry. The X-Slack-Retry-Num header can be used to identify retry attempts and prevent processing the same event multiple times. func verifySlackRequest(body string, headers http.Header) error { // Signature verification process (omitted) return nil } func isSlackRetry(headers http.Header) bool { return headers.Get("X-Slack-Retry-Num") != "" } func createIgnoredRetryResponse() (events.APIGatewayProxyResponse, error) { responseBody, _ := json.Marshal(map[string]string{"message": "Ignoring Slack retry request"}) return events.APIGatewayProxyResponse{ StatusCode: 200, Headers: map[string]string{"Content-Type": "application/json"}, Body: string(responseBody), }, nil } Step 4: Handling CallbackEvent The CallbackEvent includes actions such as message postings and adding reactions. At this stage, the system checks whether :confluence: is included in the message, if a file is attached, or if a translation-related emoji is present. Based on this assessment, it proceeds to text processing and Python Lambda invocation, as described in section 5.2 and beyond. // handleCallbackEvent processes callback events (covered in Section 5.1). func handleCallbackEvent(ctx context.Context, isOrchestrator bool, event *slackevents.EventsAPIEvent) (events.APIGatewayProxyResponse, error) { innerEvent := event.InnerEvent switch innerEvent.Data.(type) { case *slackevents.AppMentionEvent: // Processing for AppMentionEvent (details explained in 5.2) case *slackevents.MessageEvent: // Processing for MessageEvent (details explained in 5.2) case *slackevents.ReactionAddedEvent: // Processing for ReactionAddedEvent (details explained in 5.2) } return events.APIGatewayProxyResponse{Body: "OK", StatusCode: http.StatusOK}, nil } Complete Handler Code Example These steps combine to define an AWS Lambda handler. :::details Complete code example of the handler func handler(ctx context.Context, request events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) { event, err := parseSlackEvent(request.Body) if err != nil { return createErrorResponse(400, err) } if event.Type == slackevents.URLVerification { return handleURLVerification(request.Body) } headers := convertToHTTPHeader(request.Headers) err = verifySlackRequest(request.Body, headers) if err != nil { return createErrorResponse(http.StatusUnauthorized, fmt.Errorf("Failed to validate request: %w", err)) } if isSlackRetry(headers) { return createIgnoredRetryResponse() } if event.Type == slackevents.CallbackEvent { return handleCallbackEvent(ctx, event) } return events.APIGatewayProxyResponse{Body: "OK", StatusCode: 200}, nil } func convertToHTTPHeader(headers map[string]string) http.Header { httpHeaders := http.Header{} for key, value := range headers { httpHeaders.Set(key, value) } return httpHeaders } func createErrorResponse(statusCode int, err error) (events.APIGatewayProxyResponse, error) { responseBody, _ := json.Marshal(map[string]string{"error": err.Error()}) return events.APIGatewayProxyResponse{ StatusCode: statusCode, Headers: map[string]string{"Content-Type": "application/json"}, Body: string(responseBody), }, err } ::: Summary of 5.1 In this section, we explained how to obtain the Slack App's App ID, grant scopes using OAuth & Permissions, and configure event subscriptions in Event Subscriptions. We also covered the process of receiving and parsing Slack events, including handling URL verification, signature validation, ignoring retry requests, and processing CallbackEvents. From section 5.2 onwards, we will introduce specific examples of CallbackEvent processing, text handling in Go, and sending queries to Python Lambda. 5.2 [Go] HTML Text Sanitization Sanitizing External Link References HTML text retrieved from external links may contain unnecessary tags such as script and style , which are not needed for generating responses. Passing this directly to the LLM increases the token count, leading to higher model costs and potentially reducing response accuracy. The following code uses the bluemonday package for basic sanitization. It removes unnecessary tags while preserving important ones like table , ol , and ul , ensuring the text remains well-structured and readable. Additionally, the addNewlinesForTags function inserts line breaks after specific tags, improving text formatting. This helps optimize queries to the model by ensuring that only the necessary information is passed in a structured and efficient format. @ card func sanitizeContent(htmlContent string) string { // Basic sanitization ugcPolicy := bluemonday.UGCPolicy() sanitized := ugcPolicy.Sanitize(htmlContent) // Allow specific tags in a custom policy customPolicy := bluemonday.NewPolicy() customPolicy.AllowLists() customPolicy.AllowTables() customPolicy.AllowAttrs("href").OnElements("a") // Add line breaks after specific tags to improve readability formattedContent := addNewlinesForTags(sanitized, "p") // Apply final sanitization after enforcing the custom policy finalContent := customPolicy.Sanitize(formattedContent) return finalContent } func addNewlinesForTags(htmlStr string, tags ...string) string { for _, tag := range tags { closeTag := fmt.Sprintf("</%s>", tag) htmlStr = strings.ReplaceAll(htmlStr, closeTag, closeTag+"\n") } return htmlStr } This process ensures that the model receives only text with unnecessary tags removed, improving response accuracy and cost efficiency. By preserving essential structures such as tables and bullet points while inserting line breaks after specific tags, the model can better interpret the provided context. 5.3 [Python] Example of an LLM Query Below is an example of how to query an LLM (e.g. Azure OpenAI) in Python. With OpenAIClientFactory , you can dynamically switch models and endpoints, enabling the reuse of a common client creation process across multiple Lambda handlers. Client Creation Process OpenAIClientFactory dynamically generates a client for either Azure OpenAI or OpenAI, depending on api_type and model . Since API keys and endpoints are retrieved from environment variables and secret management services, code modifications are minimized even when updating models or configurations. import openai from shared.secrets import get_secret class OpenAIClientFactory: @staticmethod def create_client(region="eastus2", model="gpt-4o") -> openai.OpenAI: secret = get_secret() api_type = secret.get("openai_api_type", "azure") if api_type == "azure": return openai.AzureOpenAI( api_key=secret.get(f"azure_openai_api_key_{region}"), azure_endpoint=secret.get(f"azure_openai_endpoint_{region}"), api_version=secret.get( f"azure_openai_api_version_{region}", "2024-07-01-preview" ), ) elif api_type == "openai": return openai.OpenAI(api_key=secret.get("openai_api_key")) raise ValueError(f"Invalid api_type: {api_type}") LLM Query Processing The chatCompletionHandler function extracts messages , model , temperature , and other parameters from the JSON received in the HTTP request. It then queries the LLM using the client generated by OpenAIClientFactory . Responses are returned in JSON format. If an error occurs, a properly formatted error response is generated using a common error handling function. import json from typing import Any, Dict, List import openai from openai.types.chat import ChatCompletionMessageParam from shared.openai_client import OpenAIClientFactory def chatCompletionHandler(event: Dict[str, Any], context: Any) -> Dict[str, Any]: request_body = json.loads(event["body"]) messages: List[ChatCompletionMessageParam] = request_body.get("messages", []) model = request_body.get("model", "gpt-4o") client = OpenAIClientFactory.create_client(model=model) temperature = request_body.get("temperature", 0.7) max_tokens = request_body.get("max_tokens", 4000) response_format = request_body.get("response_format", None) completion = client.chat.completions.create( model=model, stream=False, messages=messages, max_tokens=max_tokens, frequency_penalty=0, presence_penalty=0, temperature=temperature, response_format=response_format, ) return { "statusCode": 200, "body": json.dumps(completion.to_dict()), "headers": { "Content-Type": "application/json", "Access-Control-Allow-Origin": "*", "Access-Control-Allow-Methods": "OPTIONS,POST", "Access-Control-Allow-Headers": "Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token", }, } This mechanism allows different Lambda handlers to make LLM queries using the same procedure, ensuring flexibility in adapting to models and endpoint changes. 5.4 [Python] Example of a RAG Search Call This section provides instructions on performing a Retrieval Augmented Generation (RAG) search in Python. By vectorizing internal knowledge, such as Confluence documents, and performing similarity searches using the FAISS index, it is possible to integrate highly relevant information into LLM responses. A key consideration is the handling of the faiss library. faiss is a large package and may exceed the capacity limits of the Lambda Layers. To work around this, it is common to use EFS or containerize the Lambda function. To simplify deployment, the setup_faiss function dynamically downloads and extracts faiss from S3, then adds it to sys.path , making faiss available at runtime. What is FAISS? FAISS (Facebook AI Similarity Search) is an approximate nearest neighbor search library developed by Meta (Facebook). It provides tools for creating indexes to efficiently search for similar images and text. @ card FAISS Setup Using the setup_faiss Function To use FAISS in the Lambda environment, the setup_faiss function performs the following steps: Build and archive the faiss package in a local/CI environment Developers install the faiss-cpu package in a CI environment such as GitHub Actions and package the necessary binaries into a tar.gz archive. Upload to S3 The archived faiss_package.tar.gz is uploaded to an S3 bucket. By storing the package in an appropriate bucket and path (e.g., for staging or production), the Lambda function can dynamically retrieve it during execution. Dynamic loading with setup_faiss when running Lambda In the Lambda execution environment, the setup_faiss function downloads and extracts faiss_package.tar.gz from S3 at startup and adds it to sys.path . This enables the Lambda function to run import faiss , allowing for efficient vector searches using embeddings. Example: Uploading the FAISS Package to S3 Using GitHub Actions The following GitHub Actions workflow demonstrates how to install faiss-cpu , package it for Lambda use, and upload it to S3. This setup uses GitHub Actions Secrets and Environment Variables to manage AWS credentials and S3 bucket names securely, avoiding hardcoded values. @ card name: Build and Upload FAISS on: workflow_dispatch: inputs: environment: description: Deployment Environment type: environment default: dev jobs: build-and-upload-faiss: environment: ${{ inputs.environment }} runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-python@v5 with: python-version: "3.11" # Install required packages (faiss-cpu) - name: Install faiss-cpu run: | set -e echo "Installing faiss-cpu..." pip install faiss-cpu --no-deps # Archive the faiss binary - name: Archive faiss binaries run: | mkdir -p faiss_package pip install --target=faiss_package faiss-cpu tar -czvf faiss_package.tar.gz faiss_package # Set AWS credentials (configure Secrets or Roles based on your environment) - name: Configure AWS credentials uses: aws-actions/configure-aws-credentials@v3 with: aws-access-key-id: ${{ secrets.CICD_AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ secrets.CICD_AWS_SECRET_ACCESS_KEY }} aws-region: ap-northeast-1 # Upload to S3 - name: Upload faiss binaries to S3 run: | echo "Uploading faiss_package.tar.gz to S3..." aws s3 cp faiss_package.tar.gz s3://${{ secrets.AWS_S3_BUCKET }}/lambda/faiss_package.tar.gz echo "Upload complete." In the above example, faiss_package.tar.gz is uploaded to S3 with the key lambda/faiss_package.tar.gz . Dynamic Loading Process on Lambda side ( setup_faiss function) The setup_faiss function handles the dynamic loading of FAISS at runtime. It downloads faiss_package.tar.gz from S3, extracts it to the /tmp directory, and appends the package path to sys.path . This enables import faiss to be executed within Lambda, allowing FAISS index lookups to be performed. # setup_faiss example: Download the FAISS package from S3 and add it to sys.path import os import sys import tarfile from shared.logger import getLogger from shared.s3_client import S3Client logger = getLogger(__name__) def setup_faiss(s3_client: S3Client, s3_bucket: str) -> None: try: import faiss logger.info("faiss has already been imported.") except ImportError: logger.info("faiss not found. Downloading from S3.") faiss_package_key = "lambda/faiss_package.tar.gz" faiss_package_path = "/tmp/faiss_package.tar.gz" faiss_extract_path = "/tmp/faiss_package" # Download the package from S3 and extract it s3_client.download_file(bucket_name=s3_bucket, key=faiss_package_key, file_path=faiss_package_path) with tarfile.open(faiss_package_path, "r:gz") as tar: for member in tar.getmembers(): member.name = os.path.relpath(member.name, start=member.name.split("/")[0]) tar.extract(member, faiss_extract_path) sys.path.insert(0, faiss_extract_path) import faiss logger.info("faiss was imported successfully.") RAG Search Using Embeddings and FAISS Indexes The search_data function loads the FAISS index retrieved from S3 locally and searches for documents that best match the query. Documents are vectorized using the Embeddings client (Azure OpenAI or OpenAI) generated by the get_embeddings function, enabling fast searches using faiss . from typing import Any, Dict, List, Optional from langchain_community.vectorstores import FAISS from langchain_core.documents.base import Document from langchain_core.vectorstores.base import VectorStoreRetriever from shared.secrets import get_secret from shared.logger import getLogger from langchain_openai import AzureOpenAIEmbeddings, OpenAIEmbeddings logger = getLogger(__name__) def get_embeddings(secrets: Dict[str, str]): api_type: str = secrets.get("openai_api_type", "azure") if api_type == "azure": return AzureOpenAIEmbeddings( openai_api_key=secrets.get("azure_openai_api_key_eastus2"), azure_endpoint=secrets.get("azure_openai_endpoint_eastus2"), model="text-embedding-3-large", api_version=secrets.get("azure_openai_api_version_eastus2", "2023-07-01-preview"), ) elif api_type == "openai": return OpenAIEmbeddings( openai_api_key=secrets.get("openai_api_key"), model="text-embedding-3-large", ) else: logger.error("An invalid API type specified.") raise ValueError("Invalid api_type") def search_data( query: str, index_folder_path: str, search_type: str = "similarity", score_threshold: Optional[float] = None, k: Optional[int] = None, fetch_k: Optional[int] = None, lambda_mult: Optional[float] = None, ) -> List[Dict]: secrets: Dict[str, str] = get_secret() embeddings = get_embeddings(secrets) db: FAISS = FAISS.load_local( folder_path=index_folder_path, embeddings=embeddings, allow_dangerous_deserialization=True, ) search_kwargs = {"k": k} if search_type == "similarity_score_threshold" and score_threshold is not None: search_kwargs["score_threshold"] = score_threshold elif search_type == "mmr": search_kwargs["fetch_k"] = fetch_k or k * 4 if lambda_mult is not None: search_kwargs["lambda_mult"] = lambda_mult retriever: VectorStoreRetriever = db.as_retriever( search_type=search_type, search_kwargs=search_kwargs, ) results: List[Document] = retriever.invoke(input=query) return [{"content": doc.page_content, "metadata": doc.metadata} for doc in results] Asynchronous Downloads and Lambda Handlers Within async_handler , setup_faiss is executed, and the FAISS index file is retrieved from S3 using download_files . Afterward, search_data performs a RAG search, and the results are returned in JSON format. import asyncio import json import os from shared.s3_client import S3Client from shared.logger import getLogger from shared.token_verifier import with_token_verification logger = getLogger(__name__) RESULT_NUM = 5 @with_token_verification async def async_handler(event: Dict[str, Any], context: Any) -> Dict[str, Any]: env = os.getenv("ENV") s3_client = S3Client() s3_bucket = "bucket-name" setup_faiss(s3_client, s3_bucket) request_body_str = event.get("body", "{}") request_body = json.loads(request_body_str) query = request_body.get("query") index_path = request_body.get("index_path") local_index_dir = "/tmp/index_faiss" await download_files(s3_client, s3_bucket, index_path, local_index_dir) results = search_data( query, local_index_dir, search_type=request_body.get("search_type", "similarity"), score_threshold=request_body.get("score_threshold"), k=request_body.get("k", RESULT_NUM), fetch_k=request_body.get("fetch_k"), lambda_mult=request_body.get("lambda_mult"), ) return create_response(200, results) def retrieverHandler(event: Dict[str, Any], context: Any) -> Dict[str, Any]: return asyncio.run(async_handler(event, context)) def create_response(status_code: int, body: Any) -> Dict[str, Any]: return { "statusCode": status_code, "body": json.dumps(body, ensure_ascii=False), "headers": { "Content-Type": "application/json", }, } async def download_files(s3_client: S3Client, bucket: str, key: str, file_path: str) -> None: loop = asyncio.get_running_loop() await loop.run_in_executor(None, download_files_from_s3, s3_client, bucket, key, file_path) def download_files_from_s3(s3_client: S3Client, s3_bucket: str, prefix: str, local_dir: str) -> None: keys = s3_client.list_objects(bucket_name=s3_bucket, prefix=prefix) if not keys: logger.info(f"No file found in '{prefix}'") return for key in keys: relative_path = os.path.relpath(key, prefix) local_file_path = os.path.join(local_dir, relative_path) os.makedirs(os.path.dirname(local_file_path), exist_ok=True) s3_client.download_file(bucket_name=s3_bucket, key=key, file_path=local_file_path) Summary of 5.4 Avoid Lambda layer capacity issues with setup_faiss faiss dynamic loading. Asynchronous I/O and S3 usage allow FAISS index to be loaded without containerization or EFS connectivity. search_data searches the embedded index, enabling RAG to quickly provide similar documents. This enables high-speed knowledge searches using RAG, providing LLM answers enriched with company-specific information. 5.5 [Python] Embedding and FAISS Indexing This section provides an example of periodic batch processing that embeds internal company documents (such as Confluence pages) and creates or updates the FAISS index. The index used in the RAG pipeline is essential for generative AI to incorporate company-specific knowledge into its responses. To maintain accuracy, we regularly update embeddings and rebuild the FAISS index, ensuring that the latest information is always accessible. Process Overview Retrieve JSON-formatted documents from S3. Generate embeddings for the retrieved documents (using the Embeddings API from OpenAI or Azure OpenAI). Index the embedded text using FAISS. Upload the FAISS index to S3. By executing these steps periodically via Lambda batch processing or a Step Functions workflow, RAG searches will always use the latest index when queried. Step 1: Loading a JSON document Download and parse a JSON file from S3 (e.g., summarized Confluence pages) and convert it into a list of Document objects. import json from typing import Any, Dict, List from langchain_core.documents.base import Document from shared.logger import getLogger logger = getLogger(__name__) def load_json(file_path: str) -> List[Document]: """ Reads a JSON file and returns a list of Document objects. The JSON format is expected to be: [{"title": "...", "content": "...", "id": "...", "url": "..."}] """ with open(file_path, "r", encoding="utf-8") as f: data = json.load(f) if not isinstance(data, list): raise ValueError("The top-level JSON structure is not a list.") documents = [] for record in data: if not isinstance(record, dict): logger.warning(f"Skipped record (not a dictionary): {record}") continue title = record.get("title", "") content = record.get("content", "") metadata = { "id": record.get("id"), "title": title, "url": record.get("url"), } # Create a Document object combining the title and content doc = Document(page_content=f"Title: {title}\nContent: {content}", metadata=metadata) documents.append(doc) logger.info(f"Loaded {len(documents)} documents.") return documents Step 2: Embedding and FAISS indexing The vectorize_and_save function embeds the documents using the Embeddings client obtained from get_embeddings and creates a FAISS index. It then saves the index locally. import os from langchain_community.vectorstores import FAISS from langchain_core.text_splitter import RecursiveCharacterTextSplitter from shared.logger import getLogger logger = getLogger(__name__) def vectorize_and_save(documents: List[Document], output_dir: str, embeddings) -> None: """ Embed the documents, create a FAISS index, and save it locally. """ # Split the document into smaller chunks using a text splitter text_splitter = RecursiveCharacterTextSplitter(chunk_size=1024, chunk_overlap=128) split_docs = text_splitter.split_documents(documents) logger.info(f"{len(split_docs)} split documents") # Vectorize using embeddings and build FAISS index db: FAISS = FAISS.from_documents(split_docs, embeddings) logger.info("Vector DB construction completed.") os.makedirs(output_dir, exist_ok=True) db.save_local(output_dir) logger.info(f"Vector DB saved to {output_dir}") Step 3: Uploading the Index to S3 By uploading the locally created FAISS index to S3, it can be easily accessed by the RAG search Lambda. from shared.s3_client import S3Client from shared.logger import getLogger logger = getLogger(__name__) def upload_faiss_to_s3(s3_client: S3Client, s3_bucket: str, local_index_dir: str, index_s3_path: str) -> None: """ Upload the FAISS index to S3. """ index_files = ["index.faiss", "index.pkl"] for file_name in index_files: local_file_path = os.path.join(local_index_dir, file_name) s3_index_key = os.path.join(index_s3_path, file_name) s3_client.upload_file(local_file_path, s3_bucket, s3_index_key) logger.info(f"FAISS index file uploaded to s3://{s3_bucket}/{s3_index_key}") Step 4: Running the entire flow in Lambda The index_to_s3 function encapsulates the entire process. It downloads JSON from S3, generates embeddings, creates a FAISS index, and uploads the index to S3. This process can be executed periodically using a workflow such as Step Functions, ensuring that the index remains up to date. import os from shared.faiss import setup_faiss from shared.logger import getLogger from shared.s3_client import S3Client from shared.secrets import get_secret logger = getLogger(__name__) def index_to_s3(json_s3_key: str, index_s3_path: str) -> Dict[str, Any]: """ Download JSON from S3, generate embeddings, create a FAISS index, and upload the index to S3. """ env = os.getenv("ENV") if env is None: error_msg = "ENV environment variable not set." logger.error(error_msg) return {"status": "error", "message": error_msg} try: s3_client = S3Client() s3_bucket = "bucket-name" local_json_path = "/tmp/json_file.json" local_index_dir = "/tmp/index" # Set up faiss if necessary (download from S3) setup_faiss(s3_client, s3_bucket) # Download the JSON file from S3 s3_client.download_file(s3_bucket, json_s3_key, local_json_path) documents = load_json(local_json_path) # Get Embeddings client secrets = get_secret() embeddings = get_embeddings(secrets) # Vectorization and FAISS indexing vectorize_and_save(documents, local_index_dir, embeddings) # Upload the index file to S3 upload_faiss_to_s3(s3_client, s3_bucket, local_index_dir, index_s3_path) return { "status": "success", "message": "FAISS index created and uploaded to S3.”, "output": { "bucket": s3_bucket, "index_key": index_s3_path, }, } except Exception as e: logger.error(f"An error occurred during the indexing process: {e}") return {"status": "error", "message": str(e)} Summary of 5.5 load_json loads a JSON file, and vectorize_and_save generates embeddings and creates a FAISS index. upload_faiss_to_s3 uploads the local index to S3. index_to_s3 consolidates the entire process, ensuring that the latest index is created and updated through regular batch processing. This enables automated batch processing to embed internal documents and maintain FAISS indexes for RAG searches. 6. Summary In this article, we covered the development background and technical implementation of Sherpa, out internal chatbot powered by LLM and integrated into Slack. We also outlined the steps for implementing the RAG pipeline, sanitizing Confluence documents, building a search infrastructure using Embeddings and FAISS indexes, and extending functionality with features like translation and summarization. This system enables employees to seamlessly integrate generative AI into their Slack workflow, allowing them to access advanced information capabilities without needing to learn new tools or commands. 7. Future Outlook We will actively work on the following improvements and expansions to further enhance Sherpa. Strengthening Azure-based deployment We will fully integrate with Azure services such as Azure Functions and Azure CosmosDB, significantly improving the performance and scalability of the RAG pipeline. Introducing Azure Cosmos DB Vector Search We will implement vector search functionality on Azure Cosmos DB for NoSQL, enabling more advanced search capabilities. @ card Utilizing AI Document Intelligence By actively incorporating AI Document Intelligence, we aim to expand the knowledge scope of RAG and enhance information utilization across a broader range of use cases. @ card Diversification and sophistication of models We will continue integrating cutting-edge models by expanding support beyond GPT-4o to include GPT-o1, Google Gemini, and other state-of-the-art AI models. Implementing Web UI To overcome the expression and interaction limitations imposed by Slack, we will develop a Web UI, allowing for more diverse interactions and the flexible deployment of new features. Enhancing prompt management We will template existing prompts, making them easily reusable across different use cases. Additionally, we will enhance the prompt-sharing functionality to further promote the adoption of generative AI across the company. Realizing multi-agent capabilities By deploying specialized agents dedicated to tasks such as summarization, translation, and RAG search, and allowing flexible combinations through an Agent Builder, we will enable more advanced and adaptable information processing. Evaluating and improving RAG accuracy We will build test sets and conduct automated answer evaluations to quantitatively measure accuracy and continuously improve quality. Enhancements based on user feedback By incorporating real-world usage data and feedback, we will optimize dialogue flows, fine-tune prompts, and strengthen external service integrations, ensuring that Sherpa remains highly convenient and useful. Through these efforts, we will continue evolving Sherpa, growing it into a powerful internal support tool that meets a wide range of business needs.

1. Introduction Hello, this is Torii ( @yu_torii ) from the Common Service Development Group. I'm a fullstack software engineer, primarily working on both backend and frontend development. As part of the KINTO Member Platform Development Team, I focus on frontend engineering while also contributing to internal initiatives involving generative AI. This article introduces our internal generative AI tool, our internal chatbot powered by LLM and integrated into Slack. We’ll explore its RAG capabilities and its translation function, which works through Slack reactions. Internal generative AI tool is designed to facilitate generative AI adoption within the company by enabling employees to naturally interact with AI in their daily Slack conversations. The goal is to eliminate the need for frontend development and deploy it quickly within the company, enhancing efficiency and collaboration. Internal generative AI tool aims to be a dependable assistant for improving work efficiency and streamlining information sharing within the company. By the way, the character was generated by team members of the Creative Group as a surprise reveal for the KINTO Technologies General Meeting (KTC CHO All-Hands Meeting). A big thank you to them! This initiative was carried out in collaboration with Wada-san ( @cognac_n ), who is leading the Generative AI Development Project. We have been driving the adoption of generative AI within the company through various improvements, such as introducing a RAG pipeline, setting up a local development environment, and implementing a translation and summarization feature using Slack emoji reactions. :::details Click here for Wada-san's article @ card @ card @ card ::: Note that this article does not go into detail about RAG or generative AI technology itself. Instead, it focuses on the implementation process and feature enhancements. Additionally, Internal generative AI tool's LLM runs on Azure OpenAI. What You Will Gain from This Article How to use chat functions powered by generative AI in a Slack Bot This article introduces an implementation example of a chatbot that combines Slack and LLM, allowing users to trigger translations and summaries using emoji reactions or natural message posts. Techniques for retrieving Confluence data, including HTML sanitization Learn how to fetch Confluence documents using Go and sanitize HTML to prepare text for summarization and embedding. Implementing a simple RAG pipeline using FAISS and S3 This section explains the steps and considerations for setting up a simple RAG pipeline using the FAISS indexing and S3. While the response speed may not be very fast, this approach provides a cost-effective way to integrate a basic RAG system. :::message This article is based on the implementation status at the time of writing. Please note that there are still many areas for improvement. Since this is an internal tool, it is not designed for production-level performance. Additionally, this article does not cover details on setting up the development environment, such as deployment with AWS SAM, building a pipeline with Step Functions, or configuring CI/CD with GitHub Actions. ::: :::details Table of Contents (Expanded) 1. Introduction What You Will Gain from This Article 2. Overview of the Overall Architecture 3. AI-Powered Chat Function Using Slack Bot Chat Function and Reaction Function Chat Function: Usage Scenario Reaction Feature: Usage Scenario Benefits of This Configuration 3.5 Actual Use Cases Summary of Chapter 3 4. Implementation Policy and Internal Design Overall Process Flow Function Determination by Conditional Branching The Role of Sanitization Limiting RAG Usage to Confluence References Considerations for Scalability and Maintainability 5. Introduction to Code examples and Configuration Files 5.1 [Go] Receiving and Parsing Slack Events 5.2 [Go] HTML Text Sanitization 5.3 [Python] Example of an LLM Query 5.4 [Python] Example of a RAG Search Call 5.5 [Python] Embedding and FAISS Indexing ::: 2. Overview of the Overall Architecture Here, we will explain the process by which internal generative AI tool returns answers using generative AI from two perspectives: - The generative AI chat function with users - The process of indexing Confluence documents. Processing Generative AI Chat Functions with Users Calling internal generative AI tool on Slack There are two ways to call internal generative AI tool: via chat or with a reaction emoji. Users can call internal generative AI tool by mentioning it in a channel or sending a direct message to request a generative AI response. A reaction call allows you to request a translation or summary of a message by adding a translation/summary emoji reaction. Go Slack Bot Lambda The bot that handles Slack events is built with Go and runs on AWS Lambda. It receives questions and reactions, then determines processing based on the request type. When using Azure OpenAI for LLM queries or retrieving embedded data, this component generates requests and forwards them to Python Lambda. Request to LLM and RAG reference in Python Lambda Python Lambda is divided into functions responsible for LLM queries, RAG references, and related processes. It receives requests from Go Lambda, queries the LLM, and generates an answer using RAG. Returning an answer to Slack The generated answer is sent back to Slack via Go Slack Bot Lambda. The translation and summary functions that can be invoked via emoji reactions are also integrated into this flow. :::details Architecture diagram (processing user requests) flowchart LR subgraph "Processing user requests" A["User(Slack)"] -->|"Question・Emoji"| B["Go Slack Bot(Lambda)"] B -->|"Request"| C["Python Lambda RAG/LLM"] C -->|"Answer"| B B -->|"Answer"| A end ::: The Process of Indexing Confluence Documents To use the RAG pipeline, you need to prepare your Confluence documents in a way that makes them easy to summarize and embed. These preprocessing steps are structured into workflows using StepFunctions and are executed automatically on a scheduled basis. Document retrieval and HTML sanitization (Go implementation) A Lambda function implemented in Go retrieves documents from the Confluence API, cleans up HTML tags, and makes the text more manageable. The sanitized text is output as JSON. Summary processing (Go + Python Lambda invocation) Summarization aims to refine the text, making it easier to process with Embedding and RAG. The Go implementation of Lambda invokes a Python Lambda that processes requests to the Azure OpenAI Chat API, shortens the text, and converts it back to JSON. FAISS indexing and S3 storage (Indexer Lambda) The Indexer Lambda embeds the summarized text and generates a FAISS index, then stores the index and meta information in S3. This enables instant retrieval of indexed data upon a query, ensuring the RAG pipeline runs smoothly. :::details Architecture diagram (Confluence document indexing flow) flowchart LR subgraph "Indexing Preparation StepFunctions" D["Go Lambda (Document Retrieval/HTML Sanitization)"] D --> E["Go Lambda (summary/Python call)"] E --> F["Indexer Lambda(Embedding/FAISS/S3)"] end ::: By combining this pre-processing with request-time processing, internal generative AI tool enables generative AI responses that incorporate company-specific knowledge with simple operations in Slack. In the following chapters, we'll dive deeper into these components. 3. AI-Powered Chat Function Using Slack Bot The previous chapter provided an overview of the architecture. In this chapter, we’ll focus on how users can make use of internal generative AI tool’s features with simple, natural interactions in Slack, and how these features can benefit them. Here, we will illustrate what internal generative AI tool can do and which scenarios it can be useful in, while the next chapters will systematically explain the implementation details. Chat Function and Reaction Function internal generative AI tool offers a variety of generative AI capabilities, powered by natural interactions within Slack. Chat function : By posting a question, attaching files or images, including external links, or sending a message in a specific format, users can trigger LLM queries or, in certain cases, RAG searches to obtain relevant answers. By integrating AI into Slack, an everyday tool, users can seamlessly adopt generative AI without the need to learn new environments or commands. Reaction function : By simply adding specific emoji reactions to a message, users can trigger translations, delete messages, and perform other actions—enabling additional operations without requiring commands. Chat Function: Usage Scenario Basic Questions and Answers Simply posting a question allows users to receive AI-generated responses through LLM. Example scenario: Asking "What are the steps for this project?" provides an instant answer, taking into account thread history and speaker context for a more accurate response. File, image, and external link processing Attaching a file and asking "Summarize this" will generate a summary of the document. Uploading an image allows internal generative AI tool to extract text and provide relevant answers. Sharing an external link enables the bot to analyze and summarize webpage content, incorporating it into the LLM-generated response. Example scenario: -Get a short summary of meeting notes from a text file. -Extract text information from an image. -Generate a concise summary of an external article. Confluence page lookup (RAG integration) :confluence: By using index:Index name , users can perform a RAG-based search on Confluence pages containing company documentation, such as internal rules and application procedures. Example scenario: If company policies and procedures are documented in Confluence, users can instantly access relevant information tailored to internal workflows. Example scenario: Easily retrieve project-specific settings and instructions. that would otherwise be difficult to search for. Reaction Feature: Usage Scenario Adding specific emoji reactions makes calling up functions even more intuitive. Translation : Adding a translation emoji automatically translates the message into the specified language, helping break down language barriers and improve communication. Message deletion Unnecessary internal generative AI tool responses can be deleted instantly by adding a single emoji, keeping Slack channels organized. Benefits of This Configuration Seamless and intuitive AI usage : Users can use AI without needing new commands—they simply interact with Slack as they normally do (e.g., posting messages, adding emojis reactions), reducing learning costs. Embedding AI into everyday tool (Slack) : By integrating generative AI directly into Slack, AI can be naturally incorporated into daily workflows without friction. Additionally, reaction-based interactions enable users to perform actions like translation and deletion without typing commands, making AI even more accessible. Scalable and easily extendable : If new models or additional features need to be introduced, the existing flow (chat-based queries and emoji interactions) can be easily expanded by adding conditions. 3.5 Actual Use Cases Below are some real-world examples of how internal generative AI tool is used within Slack. Example 1: Image Recognition When you attach an image to a message, internal generative AI tool recognizes the text within the image and responds with its content.  Image Recognition Example 2: Answers Based on Confluence Documentation By using :confluence: emoji, users can retrieve answers based on the Confluence documentation.  Answer based on Confluence documentation As shown above, internal generative AI tool can be triggered from workflows, allowing it to function similarly to a prompt store. Example 3: Requesting English Translations via the Translation Reaction Add translation reactions to messages that users want to translate into English To share a Japanese message with an English speaker, simply add the :ai_tool_translate_to_english: reaction emoji. This will automatically translate the message into English.  English translation reaction To prevent misunderstandings and ensure users do not overly rely on automatic translations, a notification in multiple languages clarifies that the translation was AI-generated. Additionally, we provide instructions on how to use the feature to encourage adoption. Besides English, internal generative AI tool supports translation into multiple languages. Summary of Chapter 3 In this chapter, we focused on the user perspective—"what can be done with what operations." Building on the usage scenarios discussed here, the following chapters will systematically explain the implementation details. We will cover: - How to integrate Go-based bot with Python Lambdas. - How Slack events are processed and how RAG search works. - The details of the file extraction and sanitization processes. 4. Implementation Policy and Internal Design In this chapter, we will explain the implementation policy and internal design behind how internal generative AI tool provides a variety of generative AI functions through Slack messages and reactions. This section focuses on the overall concept, role distribution, and scalability considerations. Specific code snippets and configuration files will be provided in the next chapter. Overall Process Flow Slack event reception (Go Lambda) Events occurring in Slack—such as message posts, file attachments, image uploads, external link insertions, and emoji reactions—are sent to an AWS Lambda function written in Go via the Slack API. The Go function then analyzes these events and determines the appropriate action based on user intent (e.g., normal chat, translation, Confluence reference). Text processing and sanitization on the Go side Text is extracted from external links or files and added to the prompt as context. When referencing external links, meaningful tags such as table , ol , ul are preserved while unnecessary tags are removed to optimize token usage. LLM queries and RAG searches on the Python side If necessary, the Go function invokes a Python Lambda for LLM queries or a separate Python Lambda for RAG searches (e.g., for Confluence references). For example, if :confluence: is included in the request, the Go function calls the RAG search Lambda. If no index is specified, it defaults to the primary index. Otherwise, the text is passed to the LLM Lambda for standard query processing. Reply and display in Slack The Python Lambda returns the generated response to the Go function, which then posts it back to Slack. This enables users to access advanced features through familiar Slack interactions—such as emojis, keywords, and file attachments—without needing to memorize commands. Function Determination by Conditional Branching Processing is routed based on specific emojis (e.g., :confluence: ), keywords, the presence of files or images, and whether an external link is included. To add new features, simply introduce new conditions on the Go side and, if necessary, extend the logic for invoking the corresponding Python Lambdas (e.g., for LLM or RAG tasks). The Role of Sanitization Sanitizing on the Go side removes unnecessary HTML tags to improve token efficiency and ensure clean input for the model. Key structural elements such as table, ol, and ul are retained to preserve the information structure and maintain useful context for the model. Limiting RAG Usage to Confluence References RAG search is only performed when explicitly specified with :confluence: . By default, summarization, translation, and Q&A tasks are handled via direct LLM queries, ensuring RAG logic is triggered only for Confluence references. Embedding generation for Confluence documents and FAISS index updates are handled periodically by StepFunctions, ensuring that the latest index is always available for queries. Considerations for Scalability and Maintainability Conditional branching based on emojis, keywords, or the presence of files/images minimizes the number of code modifications required when introducing new features, enhancing maintainability. The separation of concerns—where text formatting and sanitization are handled on the Go side, while LLM queries and RAG searches are managed on the Python side—improves code clarity and facilitates future model replacements or additional processing logic. In the next chapter, we will introduce specific code snippets and configuration examples based on these design principles. 5. Introduction to Code examples and Configuration Files This chapter introduces a brief implementation example based on the implementation policy and design concepts explained in Chapter 4. This chapter contains the following sections: 5.1 [Go] Receiving and parsing Slack events Explains how to use Slack's Events API to receive and process events such as messages and emoji reactions. 5.2 [Go] HTML text sanitization Provides an example of sanitizing HTML when referencing external links. 5.3 [Python] Example of an LLM query Shows how to query an LLM using a Python-based Lambda function. 5.4 [Python] Example of a RAG search call Demonstrates how to perform a RAG search call, such as for Confluence lookups. 5.5 [Python] Embedding and FAISS indexing. Provides an example of Lambda code that periodically embeds Confluence documents and updates the FAISS index. 5.1 [Go] Receiving and Parsing Slack Events This section explains the basic steps for using the Slack Events API to receive and analyze events with Go code on AWS Lambda. We will also cover the settings on the Slack side (OAuth & Permissions, event subscription) and how to check the scopes required when using the chat.postMessage method (such as chat:write ), to clarify the necessary preparations before implementation. Configuration Steps on Slack Side Create an app and check the App ID : Create a new app at https://api.slack.com/apps . Once created, find your App ID (a string starting with A ) on the Basic Information page ( https://api.slack.com/apps/APP_ID/general , where APP_ID is a unique ID for your app). This App ID identifies your Slack App and can be used to access the URLs for OAuth & Permissions and Event Subscriptions pages described below. Granting scopes via OAuth & Permissions : Visit the OAuth & Permissions page ( https://api.slack.com/apps/APP_ID/oauth ) and add the necessary scopes to Bot Token Scopes. For example, if the chat.postMessage method is needed to post messages to a channel, checking this page ( https://api.slack.com/methods/chat.postMessage ) under "Required scopes" will indicate that chat:write is required. After granting the scope, click "reinstall your app" to apply the changes in your workspace. Then, the changes will be reflected.  Checking required scopes  Setting scope Enabling the Events API and subscribing to events : Enable the Events API on the "Event Subscriptions" page ( https://api.slack.com/apps/APP_ID/event-subscriptions ) and set the AWS Lambda endpoint described below in "Request URL". Add the events you want to subscribe to, such as message.channels or reaction_added . This allows Slack to send a notification to the specified URL whenever a subscribed event occurs.  Event Reception and Analysis on AWS Lambda Once the configuration is complete on the Slack side, Slack will send a POST request to AWS Lambda via API Gateway whenever a subscribed event occurs. Step 1: Parsing Slack Events Use the slack-go/slackevents package to parse the received JSON into an EventsAPIEvent structure. This makes it easier to identify event types, such as URL validation and CallbackEvent. func parseSlackEvent(body string) (*slackevents.EventsAPIEvent, error) { event, err := slackevents.ParseEvent(json.RawMessage(body), slackevents.OptionNoVerifyToken()) if err != nil { return nil, fmt.Errorf("Failed to parse Slack event: %w", err) } return &event, nil } @ card Step 2: Handling URL Verification Requests When setting up the integration, Slack will initially send an event with type=url_verification . To verify the URL, simply return the challenge value as received. Once verified, Slack will continue sending event notifications. func handleURLVerification(body string) (events.APIGatewayProxyResponse, error) { var r struct { Challenge string `json:"challenge"` } if err := json.Unmarshal([]byte(body), &r); err != nil { return createErrorResponse(400, err) } return events.APIGatewayProxyResponse{ StatusCode: 200, Body: r.Challenge, }, nil } @ card Step 3: Verifying Signatures and Ignoring Retry Requests Slack includes a request signature that allows verification of authenticity (implementation omitted). Additionally, in case of a failure or outage, Slack may resend the request as a retry. The X-Slack-Retry-Num header can be used to identify retry attempts and prevent processing the same event multiple times. func verifySlackRequest(body string, headers http.Header) error { // Signature verification process (omitted) return nil } func isSlackRetry(headers http.Header) bool { return headers.Get("X-Slack-Retry-Num") != "" } func createIgnoredRetryResponse() (events.APIGatewayProxyResponse, error) { responseBody, _ := json.Marshal(map[string]string{"message": "Ignoring Slack retry request"}) return events.APIGatewayProxyResponse{ StatusCode: 200, Headers: map[string]string{"Content-Type": "application/json"}, Body: string(responseBody), }, nil } Step 4: Handling CallbackEvent The CallbackEvent includes actions such as message postings and adding reactions. At this stage, the system checks whether :confluence: is included in the message, if a file is attached, or if a translation-related emoji is present. Based on this assessment, it proceeds to text processing and Python Lambda invocation, as described in section 5.2 and beyond. // handleCallbackEvent processes callback events (covered in Section 5.1). func handleCallbackEvent(ctx context.Context, isOrchestrator bool, event *slackevents.EventsAPIEvent) (events.APIGatewayProxyResponse, error) { innerEvent := event.InnerEvent switch innerEvent.Data.(type) { case *slackevents.AppMentionEvent: // Processing for AppMentionEvent (details explained in 5.2) case *slackevents.MessageEvent: // Processing for MessageEvent (details explained in 5.2) case *slackevents.ReactionAddedEvent: // Processing for ReactionAddedEvent (details explained in 5.2) } return events.APIGatewayProxyResponse{Body: "OK", StatusCode: http.StatusOK}, nil } Complete Handler Code Example These steps combine to define an AWS Lambda handler. :::details Complete code example of the handler func handler(ctx context.Context, request events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) { event, err := parseSlackEvent(request.Body) if err != nil { return createErrorResponse(400, err) } if event.Type == slackevents.URLVerification { return handleURLVerification(request.Body) } headers := convertToHTTPHeader(request.Headers) err = verifySlackRequest(request.Body, headers) if err != nil { return createErrorResponse(http.StatusUnauthorized, fmt.Errorf("Failed to validate request: %w", err)) } if isSlackRetry(headers) { return createIgnoredRetryResponse() } if event.Type == slackevents.CallbackEvent { return handleCallbackEvent(ctx, event) } return events.APIGatewayProxyResponse{Body: "OK", StatusCode: 200}, nil } func convertToHTTPHeader(headers map[string]string) http.Header { httpHeaders := http.Header{} for key, value := range headers { httpHeaders.Set(key, value) } return httpHeaders } func createErrorResponse(statusCode int, err error) (events.APIGatewayProxyResponse, error) { responseBody, _ := json.Marshal(map[string]string{"error": err.Error()}) return events.APIGatewayProxyResponse{ StatusCode: statusCode, Headers: map[string]string{"Content-Type": "application/json"}, Body: string(responseBody), }, err } ::: Summary of 5.1 In this section, we explained how to obtain the Slack App's App ID, grant scopes using OAuth & Permissions, and configure event subscriptions in Event Subscriptions. We also covered the process of receiving and parsing Slack events, including handling URL verification, signature validation, ignoring retry requests, and processing CallbackEvents. From section 5.2 onwards, we will introduce specific examples of CallbackEvent processing, text handling in Go, and sending queries to Python Lambda. 5.2 [Go] HTML Text Sanitization Sanitizing External Link References HTML text retrieved from external links may contain unnecessary tags such as script and style , which are not needed for generating responses. Passing this directly to the LLM increases the token count, leading to higher model costs and potentially reducing response accuracy. The following code uses the bluemonday package for basic sanitization. It removes unnecessary tags while preserving important ones like table , ol , and ul , ensuring the text remains well-structured and readable. Additionally, the addNewlinesForTags function inserts line breaks after specific tags, improving text formatting. This helps optimize queries to the model by ensuring that only the necessary information is passed in a structured and efficient format. @ card func sanitizeContent(htmlContent string) string { // Basic sanitization ugcPolicy := bluemonday.UGCPolicy() sanitized := ugcPolicy.Sanitize(htmlContent) // Allow specific tags in a custom policy customPolicy := bluemonday.NewPolicy() customPolicy.AllowLists() customPolicy.AllowTables() customPolicy.AllowAttrs("href").OnElements("a") // Add line breaks after specific tags to improve readability formattedContent := addNewlinesForTags(sanitized, "p") // Apply final sanitization after enforcing the custom policy finalContent := customPolicy.Sanitize(formattedContent) return finalContent } func addNewlinesForTags(htmlStr string, tags ...string) string { for _, tag := range tags { closeTag := fmt.Sprintf("</%s>", tag) htmlStr = strings.ReplaceAll(htmlStr, closeTag, closeTag+"\n") } return htmlStr } This process ensures that the model receives only text with unnecessary tags removed, improving response accuracy and cost efficiency. By preserving essential structures such as tables and bullet points while inserting line breaks after specific tags, the model can better interpret the provided context. 5.3 [Python] Example of an LLM Query Below is an example of how to query an LLM (e.g. Azure OpenAI) in Python. With OpenAIClientFactory , you can dynamically switch models and endpoints, enabling the reuse of a common client creation process across multiple Lambda handlers. Client Creation Process OpenAIClientFactory dynamically generates a client for either Azure OpenAI or OpenAI, depending on api_type and model . Since API keys and endpoints are retrieved from environment variables and secret management services, code modifications are minimized even when updating models or configurations. import openai from shared.secrets import get_secret class OpenAIClientFactory: @staticmethod def create_client(region="eastus2", model="gpt-4o") -> openai.OpenAI: secret = get_secret() api_type = secret.get("openai_api_type", "azure") if api_type == "azure": return openai.AzureOpenAI( api_key=secret.get(f"azure_openai_api_key_{region}"), azure_endpoint=secret.get(f"azure_openai_endpoint_{region}"), api_version=secret.get( f"azure_openai_api_version_{region}", "2024-07-01-preview" ), ) elif api_type == "openai": return openai.OpenAI(api_key=secret.get("openai_api_key")) raise ValueError(f"Invalid api_type: {api_type}") LLM Query Processing The chatCompletionHandler function extracts messages , model , temperature , and other parameters from the JSON received in the HTTP request. It then queries the LLM using the client generated by OpenAIClientFactory . Responses are returned in JSON format. If an error occurs, a properly formatted error response is generated using a common error handling function. import json from typing import Any, Dict, List import openai from openai.types.chat import ChatCompletionMessageParam from shared.openai_client import OpenAIClientFactory def chatCompletionHandler(event: Dict[str, Any], context: Any) -> Dict[str, Any]: request_body = json.loads(event["body"]) messages: List[ChatCompletionMessageParam] = request_body.get("messages", []) model = request_body.get("model", "gpt-4o") client = OpenAIClientFactory.create_client(model=model) temperature = request_body.get("temperature", 0.7) max_tokens = request_body.get("max_tokens", 4000) response_format = request_body.get("response_format", None) completion = client.chat.completions.create( model=model, stream=False, messages=messages, max_tokens=max_tokens, frequency_penalty=0, presence_penalty=0, temperature=temperature, response_format=response_format, ) return { "statusCode": 200, "body": json.dumps(completion.to_dict()), "headers": { "Content-Type": "application/json", "Access-Control-Allow-Origin": "*", "Access-Control-Allow-Methods": "OPTIONS,POST", "Access-Control-Allow-Headers": "Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token", }, } This mechanism allows different Lambda handlers to make LLM queries using the same procedure, ensuring flexibility in adapting to models and endpoint changes. 5.4 [Python] Example of a RAG Search Call This section provides instructions on performing a Retrieval Augmented Generation (RAG) search in Python. By vectorizing internal knowledge, such as Confluence documents, and performing similarity searches using the FAISS index, it is possible to integrate highly relevant information into LLM responses. A key consideration is the handling of the faiss library. faiss is a large package and may exceed the capacity limits of the Lambda Layers. To work around this, it is common to use EFS or containerize the Lambda function. To simplify deployment, the setup_faiss function dynamically downloads and extracts faiss from S3, then adds it to sys.path , making faiss available at runtime. What is FAISS? FAISS (Facebook AI Similarity Search) is an approximate nearest neighbor search library developed by Meta (Facebook). It provides tools for creating indexes to efficiently search for similar images and text. @ card FAISS Setup Using the setup_faiss Function To use FAISS in the Lambda environment, the setup_faiss function performs the following steps: Build and archive the faiss package in a local/CI environment Developers install the faiss-cpu package in a CI environment such as GitHub Actions and package the necessary binaries into a tar.gz archive. Upload to S3 The archived faiss_package.tar.gz is uploaded to an S3 bucket. By storing the package in an appropriate bucket and path (e.g., for staging or production), the Lambda function can dynamically retrieve it during execution. Dynamic loading with setup_faiss when running Lambda In the Lambda execution environment, the setup_faiss function downloads and extracts faiss_package.tar.gz from S3 at startup and adds it to sys.path . This enables the Lambda function to run import faiss , allowing for efficient vector searches using embeddings. Example: Uploading the FAISS Package to S3 Using GitHub Actions The following GitHub Actions workflow demonstrates how to install faiss-cpu , package it for Lambda use, and upload it to S3. This setup uses GitHub Actions Secrets and Environment Variables to manage AWS credentials and S3 bucket names securely, avoiding hardcoded values. @ card name: Build and Upload FAISS on: workflow_dispatch: inputs: environment: description: Deployment Environment type: environment default: dev jobs: build-and-upload-faiss: environment: ${{ inputs.environment }} runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-python@v5 with: python-version: "3.11" # Install required packages (faiss-cpu) - name: Install faiss-cpu run: | set -e echo "Installing faiss-cpu..." pip install faiss-cpu --no-deps # Archive the faiss binary - name: Archive faiss binaries run: | mkdir -p faiss_package pip install --target=faiss_package faiss-cpu tar -czvf faiss_package.tar.gz faiss_package # Set AWS credentials (configure Secrets or Roles based on your environment) - name: Configure AWS credentials uses: aws-actions/configure-aws-credentials@v3 with: aws-access-key-id: ${{ secrets.CICD_AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ secrets.CICD_AWS_SECRET_ACCESS_KEY }} aws-region: ap-northeast-1 # Upload to S3 - name: Upload faiss binaries to S3 run: | echo "Uploading faiss_package.tar.gz to S3..." aws s3 cp faiss_package.tar.gz s3://${{ secrets.AWS_S3_BUCKET }}/lambda/faiss_package.tar.gz echo "Upload complete." In the above example, faiss_package.tar.gz is uploaded to S3 with the key lambda/faiss_package.tar.gz . Dynamic Loading Process on Lambda side ( setup_faiss function) The setup_faiss function handles the dynamic loading of FAISS at runtime. It downloads faiss_package.tar.gz from S3, extracts it to the /tmp directory, and appends the package path to sys.path . This enables import faiss to be executed within Lambda, allowing FAISS index lookups to be performed. # setup_faiss example: Download the FAISS package from S3 and add it to sys.path import os import sys import tarfile from shared.logger import getLogger from shared.s3_client import S3Client logger = getLogger(__name__) def setup_faiss(s3_client: S3Client, s3_bucket: str) -> None: try: import faiss logger.info("faiss has already been imported.") except ImportError: logger.info("faiss not found. Downloading from S3.") faiss_package_key = "lambda/faiss_package.tar.gz" faiss_package_path = "/tmp/faiss_package.tar.gz" faiss_extract_path = "/tmp/faiss_package" # Download the package from S3 and extract it s3_client.download_file(bucket_name=s3_bucket, key=faiss_package_key, file_path=faiss_package_path) with tarfile.open(faiss_package_path, "r:gz") as tar: for member in tar.getmembers(): member.name = os.path.relpath(member.name, start=member.name.split("/")[0]) tar.extract(member, faiss_extract_path) sys.path.insert(0, faiss_extract_path) import faiss logger.info("faiss was imported successfully.") RAG Search Using Embeddings and FAISS Indexes The search_data function loads the FAISS index retrieved from S3 locally and searches for documents that best match the query. Documents are vectorized using the Embeddings client (Azure OpenAI or OpenAI) generated by the get_embeddings function, enabling fast searches using faiss . from typing import Any, Dict, List, Optional from langchain_community.vectorstores import FAISS from langchain_core.documents.base import Document from langchain_core.vectorstores.base import VectorStoreRetriever from shared.secrets import get_secret from shared.logger import getLogger from langchain_openai import AzureOpenAIEmbeddings, OpenAIEmbeddings logger = getLogger(__name__) def get_embeddings(secrets: Dict[str, str]): api_type: str = secrets.get("openai_api_type", "azure") if api_type == "azure": return AzureOpenAIEmbeddings( openai_api_key=secrets.get("azure_openai_api_key_eastus2"), azure_endpoint=secrets.get("azure_openai_endpoint_eastus2"), model="text-embedding-3-large", api_version=secrets.get("azure_openai_api_version_eastus2", "2023-07-01-preview"), ) elif api_type == "openai": return OpenAIEmbeddings( openai_api_key=secrets.get("openai_api_key"), model="text-embedding-3-large", ) else: logger.error("An invalid API type specified.") raise ValueError("Invalid api_type") def search_data( query: str, index_folder_path: str, search_type: str = "similarity", score_threshold: Optional[float] = None, k: Optional[int] = None, fetch_k: Optional[int] = None, lambda_mult: Optional[float] = None, ) -> List[Dict]: secrets: Dict[str, str] = get_secret() embeddings = get_embeddings(secrets) db: FAISS = FAISS.load_local( folder_path=index_folder_path, embeddings=embeddings, allow_dangerous_deserialization=True, ) search_kwargs = {"k": k} if search_type == "similarity_score_threshold" and score_threshold is not None: search_kwargs["score_threshold"] = score_threshold elif search_type == "mmr": search_kwargs["fetch_k"] = fetch_k or k * 4 if lambda_mult is not None: search_kwargs["lambda_mult"] = lambda_mult retriever: VectorStoreRetriever = db.as_retriever( search_type=search_type, search_kwargs=search_kwargs, ) results: List[Document] = retriever.invoke(input=query) return [{"content": doc.page_content, "metadata": doc.metadata} for doc in results] Asynchronous Downloads and Lambda Handlers Within async_handler , setup_faiss is executed, and the FAISS index file is retrieved from S3 using download_files . Afterward, search_data performs a RAG search, and the results are returned in JSON format. import asyncio import json import os from shared.s3_client import S3Client from shared.logger import getLogger from shared.token_verifier import with_token_verification logger = getLogger(__name__) RESULT_NUM = 5 @with_token_verification async def async_handler(event: Dict[str, Any], context: Any) -> Dict[str, Any]: env = os.getenv("ENV") s3_client = S3Client() s3_bucket = "bucket-name" setup_faiss(s3_client, s3_bucket) request_body_str = event.get("body", "{}") request_body = json.loads(request_body_str) query = request_body.get("query") index_path = request_body.get("index_path") local_index_dir = "/tmp/index_faiss" await download_files(s3_client, s3_bucket, index_path, local_index_dir) results = search_data( query, local_index_dir, search_type=request_body.get("search_type", "similarity"), score_threshold=request_body.get("score_threshold"), k=request_body.get("k", RESULT_NUM), fetch_k=request_body.get("fetch_k"), lambda_mult=request_body.get("lambda_mult"), ) return create_response(200, results) def retrieverHandler(event: Dict[str, Any], context: Any) -> Dict[str, Any]: return asyncio.run(async_handler(event, context)) def create_response(status_code: int, body: Any) -> Dict[str, Any]: return { "statusCode": status_code, "body": json.dumps(body, ensure_ascii=False), "headers": { "Content-Type": "application/json", }, } async def download_files(s3_client: S3Client, bucket: str, key: str, file_path: str) -> None: loop = asyncio.get_running_loop() await loop.run_in_executor(None, download_files_from_s3, s3_client, bucket, key, file_path) def download_files_from_s3(s3_client: S3Client, s3_bucket: str, prefix: str, local_dir: str) -> None: keys = s3_client.list_objects(bucket_name=s3_bucket, prefix=prefix) if not keys: logger.info(f"No file found in '{prefix}'") return for key in keys: relative_path = os.path.relpath(key, prefix) local_file_path = os.path.join(local_dir, relative_path) os.makedirs(os.path.dirname(local_file_path), exist_ok=True) s3_client.download_file(bucket_name=s3_bucket, key=key, file_path=local_file_path) Summary of 5.4 Avoid Lambda layer capacity issues with setup_faiss faiss dynamic loading. Asynchronous I/O and S3 usage allow FAISS index to be loaded without containerization or EFS connectivity. search_data searches the embedded index, enabling RAG to quickly provide similar documents. This enables high-speed knowledge searches using RAG, providing LLM answers enriched with company-specific information. 5.5 [Python] Embedding and FAISS Indexing This section provides an example of periodic batch processing that embeds internal company documents (such as Confluence pages) and creates or updates the FAISS index. The index used in the RAG pipeline is essential for generative AI to incorporate company-specific knowledge into its responses. To maintain accuracy, we regularly update embeddings and rebuild the FAISS index, ensuring that the latest information is always accessible. Process Overview Retrieve JSON-formatted documents from S3. Generate embeddings for the retrieved documents (using the Embeddings API from OpenAI or Azure OpenAI). Index the embedded text using FAISS. Upload the FAISS index to S3. By executing these steps periodically via Lambda batch processing or a Step Functions workflow, RAG searches will always use the latest index when queried. Step 1: Loading a JSON document Download and parse a JSON file from S3 (e.g., summarized Confluence pages) and convert it into a list of Document objects. import json from typing import Any, Dict, List from langchain_core.documents.base import Document from shared.logger import getLogger logger = getLogger(__name__) def load_json(file_path: str) -> List[Document]: """ Reads a JSON file and returns a list of Document objects. The JSON format is expected to be: [{"title": "...", "content": "...", "id": "...", "url": "..."}] """ with open(file_path, "r", encoding="utf-8") as f: data = json.load(f) if not isinstance(data, list): raise ValueError("The top-level JSON structure is not a list.") documents = [] for record in data: if not isinstance(record, dict): logger.warning(f"Skipped record (not a dictionary): {record}") continue title = record.get("title", "") content = record.get("content", "") metadata = { "id": record.get("id"), "title": title, "url": record.get("url"), } # Create a Document object combining the title and content doc = Document(page_content=f"Title: {title}\nContent: {content}", metadata=metadata) documents.append(doc) logger.info(f"Loaded {len(documents)} documents.") return documents Step 2: Embedding and FAISS indexing The vectorize_and_save function embeds the documents using the Embeddings client obtained from get_embeddings and creates a FAISS index. It then saves the index locally. import os from langchain_community.vectorstores import FAISS from langchain_core.text_splitter import RecursiveCharacterTextSplitter from shared.logger import getLogger logger = getLogger(__name__) def vectorize_and_save(documents: List[Document], output_dir: str, embeddings) -> None: """ Embed the documents, create a FAISS index, and save it locally. """ # Split the document into smaller chunks using a text splitter text_splitter = RecursiveCharacterTextSplitter(chunk_size=1024, chunk_overlap=128) split_docs = text_splitter.split_documents(documents) logger.info(f"{len(split_docs)} split documents") # Vectorize using embeddings and build FAISS index db: FAISS = FAISS.from_documents(split_docs, embeddings) logger.info("Vector DB construction completed.") os.makedirs(output_dir, exist_ok=True) db.save_local(output_dir) logger.info(f"Vector DB saved to {output_dir}") Step 3: Uploading the Index to S3 By uploading the locally created FAISS index to S3, it can be easily accessed by the RAG search Lambda. from shared.s3_client import S3Client from shared.logger import getLogger logger = getLogger(__name__) def upload_faiss_to_s3(s3_client: S3Client, s3_bucket: str, local_index_dir: str, index_s3_path: str) -> None: """ Upload the FAISS index to S3. """ index_files = ["index.faiss", "index.pkl"] for file_name in index_files: local_file_path = os.path.join(local_index_dir, file_name) s3_index_key = os.path.join(index_s3_path, file_name) s3_client.upload_file(local_file_path, s3_bucket, s3_index_key) logger.info(f"FAISS index file uploaded to s3://{s3_bucket}/{s3_index_key}") Step 4: Running the entire flow in Lambda The index_to_s3 function encapsulates the entire process. It downloads JSON from S3, generates embeddings, creates a FAISS index, and uploads the index to S3. This process can be executed periodically using a workflow such as Step Functions, ensuring that the index remains up to date. import os from shared.faiss import setup_faiss from shared.logger import getLogger from shared.s3_client import S3Client from shared.secrets import get_secret logger = getLogger(__name__) def index_to_s3(json_s3_key: str, index_s3_path: str) -> Dict[str, Any]: """ Download JSON from S3, generate embeddings, create a FAISS index, and upload the index to S3. """ env = os.getenv("ENV") if env is None: error_msg = "ENV environment variable not set." logger.error(error_msg) return {"status": "error", "message": error_msg} try: s3_client = S3Client() s3_bucket = "bucket-name" local_json_path = "/tmp/json_file.json" local_index_dir = "/tmp/index" # Set up faiss if necessary (download from S3) setup_faiss(s3_client, s3_bucket) # Download the JSON file from S3 s3_client.download_file(s3_bucket, json_s3_key, local_json_path) documents = load_json(local_json_path) # Get Embeddings client secrets = get_secret() embeddings = get_embeddings(secrets) # Vectorization and FAISS indexing vectorize_and_save(documents, local_index_dir, embeddings) # Upload the index file to S3 upload_faiss_to_s3(s3_client, s3_bucket, local_index_dir, index_s3_path) return { "status": "success", "message": "FAISS index created and uploaded to S3.”, "output": { "bucket": s3_bucket, "index_key": index_s3_path, }, } except Exception as e: logger.error(f"An error occurred during the indexing process: {e}") return {"status": "error", "message": str(e)} Summary of 5.5 load_json loads a JSON file, and vectorize_and_save generates embeddings and creates a FAISS index. upload_faiss_to_s3 uploads the local index to S3. index_to_s3 consolidates the entire process, ensuring that the latest index is created and updated through regular batch processing. This enables automated batch processing to embed internal documents and maintain FAISS indexes for RAG searches. 6. Summary In this article, we covered the development background and technical implementation of internal generative AI tool, out internal chatbot powered by LLM and integrated into Slack. We also outlined the steps for implementing the RAG pipeline, sanitizing Confluence documents, building a search infrastructure using Embeddings and FAISS indexes, and extending functionality with features like translation and summarization. This system enables employees to seamlessly integrate generative AI into their Slack workflow, allowing them to access advanced information capabilities without needing to learn new tools or commands. 7. Future Outlook We will actively work on the following improvements and expansions to further enhance internal generative AI tool. Strengthening Azure-based deployment We will fully integrate with Azure services such as Azure Functions and Azure CosmosDB, significantly improving the performance and scalability of the RAG pipeline. Introducing Azure Cosmos DB Vector Search We will implement vector search functionality on Azure Cosmos DB for NoSQL, enabling more advanced search capabilities. @ card Utilizing AI Document Intelligence By actively incorporating AI Document Intelligence, we aim to expand the knowledge scope of RAG and enhance information utilization across a broader range of use cases. @ card Diversification and sophistication of models We will continue integrating cutting-edge models by expanding support beyond GPT-4o to include GPT-o1, Google Gemini, and other state-of-the-art AI models. Implementing Web UI To overcome the expression and interaction limitations imposed by Slack, we will develop a Web UI, allowing for more diverse interactions and the flexible deployment of new features. Enhancing prompt management We will template existing prompts, making them easily reusable across different use cases. Additionally, we will enhance the prompt-sharing functionality to further promote the adoption of generative AI across the company. Realizing multi-agent capabilities By deploying specialized agents dedicated to tasks such as summarization, translation, and RAG search, and allowing flexible combinations through an Agent Builder, we will enable more advanced and adaptable information processing. Evaluating and improving RAG accuracy We will build test sets and conduct automated answer evaluations to quantitatively measure accuracy and continuously improve quality. Enhancements based on user feedback By incorporating real-world usage data and feedback, we will optimize dialogue flows, fine-tune prompts, and strengthen external service integrations, ensuring that internal generative AI tool remains highly convenient and useful. Through these efforts, we will continue evolving internal generative AI tool, growing it into a powerful internal support tool that meets a wide range of business needs.

Hello, this is HOKA from the HR Group, Organizational Human Resources Team. Today, I’d like to share what happened following the 10X Innovation Culture Program in March 2024 . Introduction KINTO Technologies (KTC from now on) is challenging itself to create an organizational environment that fosters innovation. We are still in the process of trial and error, but I would be happy if this article were read by those considering the introduction of the 10X Innovation Culture Program or those who want to create an organization that fosters innovation. July: We Tried Implementing The 10X Innovation Culture Program Ourselves I made a resolution even before joining the 10X Innovation Culture Program at the Google offices in March. I decided to implement the 10X Innovation Culture Program within our company. It was in March 2024 when we made the following decision: We will rollout the program every three months and review the assessment results. From the second round onward, we will commit to run the 10X Innovation Culture Program independently. Awacchi and myself began preparing to run the program in the summer. The first step was to select facilitators. Since there would be no Google staff to support us this time, we needed to take on the role of facilitators in-house. When I reached out on Slack, four members stepped forward to volunteer. The 10X Innovation Culture Program broadly consisted of the three main components listed below. We asked ourselves: "What would I do if I were a participant?" and "Which approach would be best for KTC?" [1] A "pre-meeting" to watch a video and understand what the 10X Innovation Culture Program (from now on, the "10X") is [2] Understanding our current status from assessment results [3] Discussing the elements of 10X as the theme  These were the only things we had to do. And yet, it was so difficult!! Key Challenges: [Time Allocation] We were unsure whether the program should take 2 hours or 3. [Content] There were two groups of participants: first-time attendees and those joining for a second time. How should we move forward? [Quality] The section defining what 10X is important, but when KTC members presented, it came across as if they were simply reading from a script. 　 So, it was a trial-and-error process, starting from the very basics. Together with Awacchi and the four facilitators, despite our uncertainties, we managed to create the next program, discussing how things should unfold based on the March program. Reviewing Each Content [1] A pre-meeting to watch a video and understand what 10X is. As with the previous program, we held a pre-meeting via Zoom. A facilitator introduced the concept of 10X, followed by a recorded playback of the 10X content. Lastly, we conducted an assessment questionnaire. It seemed that this time, the enthusiasm wasn't as high as it had been in March. Perhaps we were lacking motivation, particularly in terms of why KTC was learning this. That was the impression of the organizing team. [2] Understanding our current status from the assessment results We held our in-house 10X Innovation Culture Program at the Muromachi office on July 2, 2024. A total of 39 people participated, including those who came from the Osaka Tech Lab (our Osaka office) and Nagoya. There were both first-time and second-time participants. We looked at the results of the assessment questionnaire that everyone had completed and reported whether each numerical value had gone up or down since the previous session, but the response seemed lukewarm. Maybe the participants did not feel attached to the reported numerical values because they haven't done anything special between the last program (March) and this one (July). Simply receiving the assessment results may not be sufficient for participants to determine whether the outcomes were good or bad. That was the hypothesis of the organizing team. [3] Discussing the elements of 10X as the theme Next, we discussed how to review the past three months. Some participants didn’t remember anything at all. Some first-time participants said, “Since I only watched the video, I don’t know what I’m supposed to do.” When the program was held at the Google offices the previous time, the discussions went smoothly, so this situation was unexpected for the organizing team (though looking back it seems obvious.) Afterward, we discussed Autonomy, one of the key elements of 10X. As in the previous time, participants were divided into groups to write down issues and discuss potential solutions, but this time we had the below discoveries: Some of them haven't been actively practicing 10X over the past three months. For some reason, the atmosphere this time seemed less conducive to active discussions compared to the previous session. Post-Feedback Survey Compared to last time , the number of items with ratings 3 and 2 increased, and there were also some items rated as 1. The motivation to promote this activities also decreased. In the questionnaire comments, we received the following advice from participants: Overall impression: Thank you to the organizing team for all your hard work. It was a fun event. No particular comments for the moment. I would love to see this implemented in other teams as well. It’s a great activity, even just for learning about other people's perspectives. We also saw feedback about the previous session held: I would have appreciated a brief recap of the content from the previous program. There were moments when those who missed the previous program felt left behind. It took me some time to refresh my memory on the issues from the previous program, so I wish there had been time for a review beforehand. I felt that the discussions were more lively last time. This time, no one was proactively steering the conversation (a lack of initiative), and there was no advance preparation, so it took some time to get into the groove. On time and schedule It was short in terms of time. It might have been better if we had more time for group work and discussions. The time schedule was too tight (since things don't always go as planned, there should have been some leeway in the program. It’s too risky to plan by the minute). Maybe the problem was time allocation (I think that was the goal, but overall, it didn’t seem like we were clear on the next action.) On group discussions When forming teams for group discussions, I thought it might have been better to select members from teams that actually work together, rather than mixing people from different teams. It was stimulating and refreshing to discuss with members from different teams, like in the previous program and this one. However, I also feel that it is difficult to achieve results that are directly applicable to our work. So, I thought it might be a good idea to have a conversation about 10X again among members from the same team. On facilitation and management of the workshop There was too little time for ice-breaker activities, including self-introductions. When starting a workshop, it is very important to ease the atmosphere among unfamiliar members. We didn’t even have the time to finish self-introductions. The questions in the workshop were too abstract, making it difficult for participants to come up with concrete answers. Maybe the organizing team overlooked setting goals for the workshop. It would have been better to set a goal at the beginning, such as telling the participants, "please take this back with you." On content understanding and follow-up It was my first time participating, so I felt that the contents seemed more suited for those who had prior knowledge, and overall, it gave an impression of being somewhat vague. It might have been better to either narrow down the content for a more compact format, or extend the duration to provide more thorough explanations. Others I don’t understand the purpose of the paper (for writing responses) that was distributed. It would have been better to assign a leader to each working team from the start. Proactivity and discussions within teams Looking back after the previous session, there were few actions taken, so I would like to allocate time for regular discussions within the team. I am interested in Intrinsic Motivation (another of the 10x key concepts). I believe that combining it with Autonomy can create an even greater sense of speed. After the 10X Innovation Culture Program the facilitator members gathered to conduct a retrospective session. A lot of ideas were gathered. Now, where should we start? Meanwhile, I was given the opportunity by Google to speak at Google Cloud Next Tokyo'24 . <<To be continued in Part 2>>

This article is part of the day 21 in the KINTO Technologies Advent Calendar 2024 🎅🎄 Introduction Hello! I’m Uehara ( @penpen_77777 ) from the KINTO FACTORY Development Group. I joined the company in July 2024 and have been working on backend development for KINTO FACTORY. This time, KINTO Technologies employees participated in ISUCON14, which was held on December 8th, and would like to share the content and results. What is ISUCON14? ISUCON is a tuning competition organized by LINE Yahoo Japan Corporation. Participants are given a predefined web service and compete to optimize its performance as much as possible while adhering to specific regulations. The winning team receives a grand prize of 1 million yen! This year, the competition took place on Sunday, December 8th, from 10:00 AM to 6:00 PM. I (Uehara) have been participating in ISUCON since last year's ISUCON13, not only to gain knowledge about performance tuning but also to challenge myself as an engineer and improve my skills. :::message ISUCON is a trademark or registered trademark of LINE Yahoo Japan Corporation. https://isucon.net/ ::: Team "ktc-isucon-bu" I recruited members through the company's Slack and formed the team "ktc-isucon-bu." The team members are as follows. Uehara( @penpen_77777 ) Participated in ISUCON13 last year, making this their second time. Furuya Participating in ISUCON for the first time. Nishida Participating in ISUCON for the first time. ISUCON allows participants to choose from several programming languages for the initial implementation. This time, we chose Go. Since many of our team members were first-time participants, rather than aiming for a top ranking, we set a more achievable goal: scoring over 10,000 points. Preparation Before the Competition At ISUCON, it is crucial to automate or streamline repetitive tasks so that we can focus on optimizing during the actual competition. This time, we made the following preparations: Setting up environment provisioning and deployment commands Developing documentation generation tools Preparing measurement tools Conducting individual and team practice Setting up environment provisioning and deployment commands go-task is a task runner specialized for executing tasks in the terminal. https://taskfile.dev/ Traditionally, make is often used as a task runner, but I personally find go-task more convenient, so I chose to use it this time. (Unlike make, it requires installation, which is a minor drawback, but aside from that, I think it can be effectively utilized in real-world projects.) For example, create the following taskfile.yaml and define the setup task and deploy task. version: '3' tasks: setup: cmds: # Write commands for environment setup - echo "setup" deploy: cmds: # Write the deployment command - echo "deploy" After creating the file, you can execute a task by running task name as shown below: # Run the environment setup task $ task setup setup # Run the deploy task $ task deploy deploy Additionally, you can pass command-line arguments when running a task, embedding them into the task execution. version: '3' tasks: setup: cmds: - echo "setup {{ .CLI_ARGS }}" deploy: cmds: - echo "deploy {{ .CLI_ARGS }}" # Execute the setup task for server isu1 $ task setup -- isu1 setup isu1 # Execute the deploy task for server isu2 $ task deploy -- isu2 deploy isu2 The example above is a simple demonstration, but in real use, leveraging command-line arguments allows you to easily switch target servers, making it more efficient for teams to divide and manage tasks. Other benefits of using go-task include: Even if you are working in a subdirectory, you can find the taskfile.yaml in the parent directory and execute the task. You can execute tasks without worrying about the directory location. Tasks can be called from other tasks, increasing the reusability of the tasks you define. You can combine tasks, such as running a setup task before a deploy task. By describing all the tools used in ISUCON in taskfile.yaml, you can execute them just by typing the task command, even if you don't know how to use them (such as the required options). A template taskfile.yaml was prepared in advance so that variables could be modified easily during the competition, allowing for flexible responses to any issues that might arise. Developing documentation generation tools The following two tools were used: https://github.com/k1LoW/tbls Reads the contents of the database and generates documentation (Markdown) that includes ER diagram and schema descriptions. Allows checking the schema definition without connecting to the database, making it useful for understanding the database structure. Automatically generating documentation in the CI/CD pipeline makes it easier to track changes in the database structure. For more details, see the following article: https://devblog.thebase.in/entry/auto_generated_er_graph_by_tbls_and_github_actions https://github.com/mazrean/isucrud Analyzes application code, visualizes relationships with database tables, and generates documentation (Markdown). The drawback was that the diagrams became difficult to understand as the number of functions increased, but now it's much easier to use because you can interactively narrow down the parts you want to see on a web browser. Preparing measurement tools The following tools were used: https://github.com/kaz/pprotein Collects and visualizes access logs, slow query logs, and profile data. Automatically starts collecting data via webhook when a benchmark is run, making it convenient. Also records commit hashes at the time of data collection, making it easy to track which commits contributed to score improvements. ^[Encountered an issue where the Git commit hash could not be obtained properly with pprotein, so the repository was forked, the code was fixed, and a PR was submitted. https://github.com/kaz/pprotein/pull/37] Conducting individual and team practice For first-time participants, jumping straight into past ISUCON problems can be overwhelming and discouraging due to their high difficulty level. To help them get a feel for performance tuning, we first had them read the book Web Performance Tuning: Experts' Guide. https://gihyo.jp/book/2022/978-4-297-12846-3 As we progressed through the book, we gathered as a team on weekends to work through the following practice problems. https://github.com/catatsuy/private-isu (The book also provides guidance on how to optimize using private-isu) Once we were able to consistently score around 100,000 to 200,000 points, we shifted our focus to solving problems from ISUCON13. At this point, we had gained enough confidence to transition into competition-focused practice, allowing us to simulate real ISUCON conditions smoothly. This year’s Task https://www.youtube.com/watch?v=UFlcAUvWvrY The theme for ISUCON 14 was improving the ride chair service "ISURIDE" Chair owners provide chairs for transportation. Users request chairs via the app and travel to their destination. Scores are based on factors like distance traveled and ride completion rate. It is necessary to make improvements to increase user satisfaction. https://github.com/isucon/isucon14 Results The results are as follows. We surpassed our goal of 10,000 points! Final Ranking: 135th place out of 831 teams Final Scores: 12514 points What We Achieved Preparation of deployment scripts (Uehara 10:00-10:30) Manual for the day (Furuya 10:00) Checking running processes and understanding the general configuration (Furuya 10:00) Reading the application manual (Nishida 10:00) Logging into MySQL and checking table sizes Adding indexes (Nishida 11:21) -- chair_locations CREATE INDEX idx_chair_id ON chair_locations(chair_id); CREATE INDEX idx_chair_id_created_at ON chair_locations(chair_id, created_at); -- chairs CREATE INDEX idx_owner_id ON chairs(owner_id); -- ride_statuses CREATE INDEX idx_ride_statuses_ride_id_chair_sent_at_created_at ON ride_statuses (ride_id, chair_sent_at, created_at); Preparing pprotein (Uehara 11:48) I had a hard time fixing the nginx settings and it took me over an hour Adding more indexes (Nishida 11:54) CREATE INDEX idx_ride_statuses_created_at_ride_id_chair_sent_at ON ride_statuses (created_at, ride_id, chair_sent_at); -- rides CREATE INDEX idx_chair_id_updated_at ON rides (chair_id, updated_at DESC); Enabling dynamic parameters and speeding up json processing (Uehara 12:38) // Changed import statements "encoding/json" → "github.com/goccy/go-json" // Enabled InterpolateParams when connecting to the database dbConfig.InterpolateParams = true Adding More Indexes (Uehara 13:05) -- chairs CREATE INDEX idx_chairs_access_token ON chairs(access_token); -- ride_statuses CREATE INDEX idx_ride_statuses_ride_id_app_sent_at_created_at ON ride_statuses (ride_id, app_sent_at, created_at); -- rides CREATE INDEX idx_rides ON rides (user_id, created_at); -- coupons CREATE INDEX idx_coupons_used_by ON coupons(used_by); Implementing user status cache (Furuya 14:30) In-memory cache for users and ride_statuses tables Modifying transaction scope (Nishida 14:48, 15:09) Adjusting notification polling intervals (Furuya 16:27) Changed the RetryAfterMs returned in appGetNotification and chairGetNotification from 30 ms to 300 ms. Shortened chair-user matching interval (Nishida 17:18, 17:28) Shortened ISUCON_MATCHING_INTERVAL from 0.5 s to 0.1 s Score nearly doubled (most effective change) Stopped bin log (Furuya 17:24) Disabled bin logs in MySQL settings (/etc/mysql/mysql.conf.d/mysqld.cnf) disable-log-bin=1 innodb_flush_log_at_trx_commit=0 Disabled log output (Uehara 17:43) Stopped log output for nginx, MySQL, and application logs Improved ownerGetChairs (Nishida 17:43) Implemented memoization for distance Adjusting the number of connections (Uehara 17:49) db.SetMaxIdleConns(50) db.SetMaxOpenConns(50) Measures That Could Not Be Implemented Due to Time Constraints or Score Issues Fixed matching process (Uehara 13:00-16:00) Attempted improvements but failed to resolve chair dispatch errors and had to abandon it. Caching chair access tokens (Furuya 15:36) Significantly reduced queries from 30,000 to 100. However, the score did not increase. Likely due to a bottleneck elsewhere. nginx parameter tuning (Furuya 17:39) Unable to apply optimizations due to benchmark errors... Server split (Uehara 16:00-17:00) Attempted to split nginx+Go and MySQL, into two machines but encountered data inconsistency errors. Later realized that both servers were running the chair-matching service, which led to data inconsistencies... (If only we had stopped one...) Pros Achieved our goal of scoring over 10,000 points. Ranked 135th out of 831 teams with 12,514 points, a solid result for a team with one returning participant and two first-timers. Everyone contributed to improvement tasks, ensuring no idle time. Environment setup and deployment scripts worked smoothly. No major deployment issues, except minor struggles with nginx setup for pprotein. It took me about an hour to do so, so I would like to add a note to the instructions. Using automation tools significantly improved our workflow, and we plan to expand this approach. Points to Reflect on Lacked a deep understanding of the application before making optimizations. In this problem, understanding the application specifications was important. Mechanically optimizing based on slow query logs or application logs didn’t lead to a significant score increase, and there were few areas where we could make effective improvements. Need to develop a structured approach to understand application behavior before tuning. Understanding specifications is important in daily work as well, so I want to keep this in mind moving forward. Struggled with the matching process bug, which delayed implementation. Tried to optimize using only queries without modifying the table structure, which made implementation difficult. I realized that adding appropriate columns to the tables and simplifying the application code would have made implementation much smoother. I had no prior knowledge of SSE (Server-Sent Events), so I plan to study it. It was presented as a way to update the user’s chair status in real time. Lessons to Apply in Future Work Understanding the application is more critical than just focusing on technical optimizations. Use task runners (go-task) and auto-documentation tools (tbls, etc.) to improve workflow efficiency. Explore using SSE (Server-Sent Events) in product development. Summary We participated in ISUCON14 and achieved a score of 12514 as the team "ktc-isucon-bu", placing 135th out of 831 teams. Although there are things we need to improve on, we were pleased that we were able to achieve our goal of scoring over 10,000 points, even though some of our members were participating in ISUCON for the first time. We will use these lessons to improve further at ISUCON15. Special thanks to Furuya-san and Nishida-san for joining despite it being their first ISUCON. Also, a big thank you to all KINTO Technologies employees who showed interest, even if they couldn’t participate this time. Lastly, a huge shoutout to the ISUCON organizers for hosting this incredible event.

This article is the entry for day 20 in the KINTO Technologies Advent Calendar 2024 🎅🎄 How We Applied the Impact/Effort Matrix to Improve Cross-Team Interaction Hello! We are Maya and Kinoshita, team members of KINTO Technologies' Developer Relations (DevRel) Group. Introduction We previously held an internal event in Jimbocho to improve office communication and foster stronger ties between teams. Here’s the article from when we held the event ↓ https://blog.kinto-technologies.com/posts/2023-09-19-JimbochoISM/ For this event, we used an Impact/Effort Matrix during the planning phase to decide on the activities. This helped us organize ideas to make the event more engaging and clarify task priorities. As a result, we were able to smoothly run multiple events. In this article, we’ll share our approach to planning, running, and reflecting on events using the Impact/Effort Matrix. We hope this article will be useful for managing your project, for event planning and hope you enjoy the read till the end. About the Impact/Effort Matrix The Impact/Effort Matrix is a simple yet effective tool for efficiently determining the priority of projects and tasks. By categorizing tasks or ideas along two axes (Impact and Effort) this matrix can provide a clear, visual way to decide where to focus your team resources. Basic Structure The Impact/Effort Matrix consists of four quadrants: Quick Wins: Tasks that deliver high impact with minimal effort. They should be prioritized and executed quickly. Major Projects: Tasks that offer high impact but require significant effort. Resource planning is crucial and should be addressed strategically. Fill-ins: Tasks that have low impact and require minimal effort. These should be tackled when there’s extra capacity, but they’re lower priority. Parking Lot: Tasks that offer low impact and require a lot of effort. These should generally be avoided. For more details, you can check the Miro template description page . Why We Decided to Use the Impact/Effort Matrix After we held the first Jimbocho Information Sharing session, we received quite positive feedback from our collagues. However, not all participants felt a sense of unity, so we asked ourselves: how can we create more of a lively and energetic atmosphere? So we wanted to brainstorm this together. While discussing among the organizing team, we clarified our goals and selected the tasks to focus on and those to exclude. During this process, we explored appropriate methods to share task priorities with the team, and decided to use the Impact/Effort Matrix, which some team members had previous experience with. Percieved Benefits Clear prioritization: By placing all tasks relative to each other in the four quadrants, as discussions progress and sticky notes are arranged, the tasks that should be prioritized become visually apparent. Easily shareable, with high team alignment: Building on the previous point, the discussion about what should be prioritized serves as an opportunity to align everyone’s understanding. By progressing with the team’s consensus, commitment is high, and it becomes easier for everyone to take ownership and move forward together. Team resource optimization: By ensuring everyone agrees on the next actions, it helps prevent rework and ensures smoother progress. Challenges and Solutions As is often the case, the most challenging part for the team was decision-making. During the first round of implementing the Impact/Effort Matrix, we encountered many issues with differing opinions and a lack of shared understanding. It took a lot of discussions to organize the information and reach a consensus on how to use it for improving the next event. The granularity of the details was inconsistent, and the scope of what we wanted to accomplish was too broad, so we weren’t able to achieve the desired results. In the second round, we refined the granularity of the issues, clarified our objectives, and narrowed the scope, which allowed us to achieve more concrete outcomes. As a result, the team’s understanding deepened, and the process moved more smoothly. We were able to better define the direction for the event, and the second round was a success. We then created an action plan for the third Jimbocho Sharing Session and visualized the tasks on a Jira board. At this point, several new members joined the organizing team, many of whom were experiencing the Impact/Effort Matrix for the first time. Despite the presence of many newcomers, we were able to smoothly progress based on our prior experience, and successfully held the third Jimbocho Sharing Session. Thoughts on Using the Impact/Effort Matrix Even after understanding the Impact/Effort Matrix methodology, when we tried to apply it to our own situation, there were moments of doubt in the team about whether we were on the right track. However, by discussing the points that felt off or needed improvement and respecting each other's opinions, we were able to organize the various ideas that came up and clearly identify which tasks should be prioritized. During the process of making the Impact/Effort Matrix method work, there were disagreements, but by objectively evaluating each idea, we were able to move forward in a way that everyone could agree on. As preparations for the event progressed, the sense of unity within the team grew, and in the end, we received high praise from participants at the event. Conclusion In this article, we shared our approach to planning, executing, and reflecting on events with the Impact/Effort Matrix, using the Jimbocho internal event as an example. Initially, there were uncertainties and doubts, and we lacked confidence in whether we were doing things correctly. However, through repeated practice, we gradually became more comfortable with the process. Even when new team members joined midway, we were able to run the event smoothly. We hope this methodology will be helpful for the success of your own projects and events as it was for us. Furthermore, KINTO Technologies is looking for talents to join us! If this article has sparked even a bit of your interest, please feel free to reach out to us. We look forward to hearing from you! https://hrmos.co/pages/kinto-technologies/jobs

This article is the entry for day 22 in the KINTO Technologies Advent Calendar 2024 🎅🎄 Nice to meet you! I am kikumido, and I am developing the Unlimited (Android) app at KINTO Technologies. For our app, we use Apollo Kotlin for the GraphQL client. Apollo Kotlin v4 introduces numerous enhancements, including better performance and new features. To take advantage of these benefits, we decided to upgrade from v3 to v4. In this article, I will explain in detail the process of migrating to Apollo Kotlin v4 (which was released this July), and the issues we encountered along the way. We had initially been hoping for a nice, smooth migration, but ran into some unexpected exceptions, and sometimes struggled to find solutions. I would be delighted if this article proves to be a helpful resource for those considering the upgrade themselves. Migrating From v3 to v4 We will proceed in accordance with the official website . You can also do the upgrade semi-automatically by installing a plugin into Android Studio. We wanted to check what the necessary steps were as we went along, so we did it manually without using the plugin. On the other hand, I was curious about how much the plugin can automate, so in the second half of this article, I will compare the manual approach with the automated process. 1. Things We Had to Do When migrating from v3 to v4, there were five things we had to do in Android Studio to eliminate errors in our app. I will go through them one by one below. :::message alert What you need to do will very likely vary depending on the implementation status of your own apps, so please be aware that this will only be what we did for ours. ::: 1.1. Upgrading the Library Anyway, first, we will update the library . apollographql = "3.8.5" apollo-runtime = { module = "com.apollographql.apollo3:apollo-runtime", version.ref = "apollographql" } apollographql-apollo = { id = "com.apollographql.apollo3", version.ref = "apollographql" } ↓ // At the time of writing this article, v4.1.0 has been released. However, I have based my explanation on 4.0.1, which was the latest version when we did the migration ourselves. apollographql = "4.0.1" apollo-runtime = { module = "com.apollographql.apollo:apollo-runtime", version.ref = "apollographql" } apollographql-apollo = { id = "com.apollographql.apollo", version.ref = "apollographql" } Hmm. Instead of turning into com.apollographql.apollo4 , com.apollographql.apollo3 lost its “3” instead. 1.2. Modifying the Gradle File This did not need to be done for v3, but wrapping in a service is required for v4. apollo { packageName.set("com.example.xxx.xxx.xxx") ・・・ } ↓ apollo { service("service") { packageName.set("com.example.xxx.xxx.xxx") ・・・ } } 1.3. Changing the Import We could see this coming when the “3” disappeared, but you need to change the import package name. Just delete the “3.” import com.apollographql.apollo3.* ↓ import com.apollographql.apollo.* 1.4. Modify the Exception Handling We changed things so that execute() will not throw a fetch error, so we will deal with that. I expect that the approach will vary by project, but for now, we have chosen to take measures to minimize the impact on existing processes. This method consists of modifying things so that DefaultApolloException will be thrown under the same conditions as when a fetch error arose and ApolloException was thrown with v3. By adjusting the common handling, we ensured that the behavior remained the same for the calling side without modifying its implementation. apolloClient.query(query).execute() apolloClient.mutation(mutation).execute() ↓ execute(apolloClient.query(query)) execute(apolloClient.mutation(mutation)) private suspend inline fun <D : Operation.Data> execute(apolloCall: ApolloCall<D>): ApolloResponse<D> { val response = apolloCall.execute() if (response.data == null) { response.exception?.let { // Fetch error if response.data is null and response.exception is not null. throw DefaultApolloException(it.message, it.cause) } } return response } If it is difficult to migrate all in one go due to the existing implementation or the v4 support policy, executeV3() has been provided as a helper for migrating without changing the v3 behavior, so you can temporarily replace it with that. apolloClient.query(query).executeV3() apolloClient.mutation(mutation).executeV3() This approach is a good choice if you want to migrate to v4 gradually (for example, feature by feature). :::message alert Note that executeV3() is a deprecated method, so be aware that you’ll need to address it eventually. ::: 1.5. Modifying ApolloException to DefaultApolloException ApolloException has become a sealed class , so replace any places where instances were being generated with DefaultApolloException . The above are the steps required to eliminate errors in Android Studio. 2. Build Now that we’ve cleared all the errors...let’s try running the long-awaited build! Drum roll, aaand...ta-daaa! Yes! We got something! Build errors...! Although we didn’t expect everything to go perfectly smoothly, it was still quite a shock.... Okay, we will pull ourselves back together and check the error log. 2.1. Checking the Error Log A flood of unfamiliar error log entries suddenly appeared. In short, it seems that KSP[^1] is missing a class required for AssistedInjectProcessingStep.[^2] [^1]: For our app, we are using KSP . [^2]: For our app, we are using Hilt as a dependency injection library. If 'error.NonExistentClass' is a generated type, check for compilation errors above that may have prevented its generation. Otherwise, ensure that 'error.NonExistentClass' is included in your classpath. e: [ksp] AssistedInjectProcessingStep was unable to process 'XXXXXViewModel(java.lang.String,long,com.xx.xx.XXXXXRepository)' because 'error.NonExistentClass' could not be resolved. Of course, it is a class that exists in the source, and the build had been succeeding with no problems before we started doing this upgrade. Had we missed out something that was necessary for the v4 migration? We checked that and various other possibilities, but still couldn't pinpoint the issue... After much investigation, we managed to get the build to succeed by using the following three methods. A. Change Hilt back from KSP to kapt B. Add some code to build.gradle.kts: Pattern 1 androidComponents { onVariants(selector().all()) { variant -> afterEvaluate { val variantName = variant.name.replaceFirstChar { if (it.isLowerCase()) it.titlecase(Locale.getDefault()) else it.toString() } val generateTask = project.tasks.findByName("generateServiceApolloSources") val kspTask = project.tasks.findByName("ksp${variantName}Kotlin") as? org.jetbrains.kotlin.gradle.tasks.AbstractKotlinCompileTool<*> kspTask?.run { generateTask?.let { setSource(it.outputs) } } } } } C. Add some code to build.gradle.kts: Pattern 2 apollo { service("service") { packageName.set("com.example.xxx.xxx.xxx") // Start of additional code outputDirConnection { connectToAndroidSourceSet("main") } // End of additional code ・・・ } } By the way, we are using Hilt ’s AssistedInject (which is apparently sometimes not a problem depending on how it is defined); we are using KSP in Hilt ; and we wrap it in a service in “1.2. Modifying the Gradle file” above. If all the above conditions are met, the issue also occurs in v3, meaning it isn't strictly a 'v4-specific' support update. However, since we addressed it at the same time, I have included it here. For our app, we initially went with “B.” However, due to a bug in ApolloExtension’s service ,[^3] we subsequently found that we could do it using “C,” so we adopted that. [^3]: Issues → This was also reproduced with v4.1.0, the latest version at the time of writing. With the above changes, the build succeeded, allowing us to migrate to v4 while ensuring the app launched and functioned just as before! 3. Dealing with Deprecated Parts Next, we will address the following two things that caused deprecation warnings. 3.1. Modifying ApolloResponse.Builder @Deprecated("Use 2 params constructor instead", ReplaceWith("Builder(operation = operation, requestUuid = requestUuid).data(data = data)")) We received the above warning and made the necessary modifications accordingly ApolloResponse.Builder(operation(), UUID_CONST, data).build() ↓ ApolloResponse.Builder<D>(operation(), UUID_CONST).data(data).build() data has been moved outside, correct It seems to have been modified to follow the Builder pattern. 3.2. Modifying the Error Instance Creation Processing to Builder Update any instances where a constructor was used to create an Error instance to instead use the Builder pattern. Error("occurred Error", null, null, mapOf("errorCode" to responseCode), null) ↓ Error.Builder(message = "occurred Error") .putExtension("errorCode", responseCode) .build() Doing the above also successfully eliminated a warning. 4. Migrating Using the Plugin Using the Apollo plugin in Android Studio will automate the migration to some extent. It is very simple to do: just install the Apollo plugin in Android Studio, then tap Tools > Apollo > Migrate to Apollo Kotlin 4... . Easy, right?  Okay, let’s check the results of running it. The aforementioned 1.1. Upgrading the Library 1.2. Modifying the Gradle File 1.3. Changing the Import got done automatically. 1.4. Modifying the Exception Handling only consists of switching to executeV3() , so the appropriate action for v4 needs to be taken manually. 1.5. Modifying ApolloException to DefaultApolloException Was not completed, so it must be done manually. Essentially, it automates any migration tasks that can be handled mechanically. Therefore, another viable approach is to use the plugin for migration and then manually address only the necessary parts. By the way, this plugin not only assists with migration by graying out unused fields but is also useful for other tasks. I highly recommend installing it. 5. Summary That concludes the discussion of migrating Apollo Kotlin from v3 to v4. It's been a while since we actually completed this, so I imagine many people have already done it themselves. Still, I hope this article provides something useful. Thank you for reading all the way to the end. 6. Related links https://www.apollographql.com/docs/kotlin/migration/4.0 https://www.apollographql.com/docs/kotlin/testing/android-studio-plugin https://developer.android.com/build/migrate-to-ksp?hl=ja