Week 2 - Friday 2nd May
Last week, we had identified a key challenge in the upstream stages of the platform i.e. the need to solve for publication and news coverage. Hence the two most critical changes we made to FinVerve this week include
Implementing Google CSE
This turned out to be more challenging than I had assumed. The Google programmable search engine is quite powerful but it requires a lot of configuration and I wnet through a lot of tiny experiments to get my requirements met. The set up that I now have is :
- Define a cluster of keywords in the db.
- e.g “QR Payments OR Tokenized Payments or Card Tokenization or Visa or MasterCard” etc.
- Each such cluster is mapped to an industry or function or a combination. e.g I have two clusters around payments. You may ask why 2 clusters - coz there is a limit of the number of keywords you can pass in the API query to this custom search engine
- Map each such cluster to the CSE ID (figured may eventually need different CSE instances for different keyword clusters)
- Choose the publications(sites) to search from instead of letting Google do an Internet wide search (too much noise )
- This is also tricky, coz one is looking for specific sections of content in specific websites. SO you have to define patterns wherever possible (to reduce noise)
- Also choose the sites to ignore. These were basically anti-patterns from the same set of sites I had above e.g. from a typical site I do NOT want pages like homepage.com/tag/* or homepage.com/author/*. These are list pages not an individual post page. E.g. Found a few Indian business news sites Hindi sub-domains popping up.
- Interestingly the API returns 10 results at a time and one needs to do another one to get 11-20 results - think how we navigate to the next “page” of Google Search results. The API behavior is similar
- Once the URLs and their title is fetched - the downstream flow was easy. I just plugged it into the existing FinVerve flow.
- Will test it a few times more before releasing into production over the weekend.
User's Dashboard to load with market-level filters pre-populated.
User feedback is gold. In one such brief conversation with an early user I heard that he's just reading our daily emails but not even logging in because most of the news were global (interesting but not actionable). He's super focused on India market fintech news. My immediate reaction was Sh*t! How'd I miss this simple bit. User's can choose markets and we should load the dashboard and the daily newsletter basis their preferred markets. So quickly built this out on the dashboard. Daily newsletter tweaks are still WIP.
Plan for next week - Allowing user's to generate the Linkedin Post in their own voice.