Notes from a CTO #4: Choosing the right problem to solve, Docsumo's Architecture, Generative AI.
Let's discuss how to choose the right problem to solve and Docsumo's architecture.
Hey, it’s Bkrm from Docsumo.
You’re reading the fourth issue of Notes from a CTO: my raw canvas of thoughts and collection of interesting resources I found online. My goal here is to spread knowledge at scale. If this is up your alley, have a read and let me know what you think!
1. Running a startup
This section is a dump of the various facets of startups I have experienced while running one. This week, let's talk about choosing the right market and figuring out the right problem to solve.
You must’ve read various articles on market research, finding the GTM of your idea, doing a bottom-up or top-down approach, figuring out if your product is a pain killer or vitamin, and if your segment is blue ocean or red ocean. These are important concepts that every founder should understand, but I found them completely useless when I tried to put them into action. Though, one concept that worked out pretty well was: What you have to hunt to be a $100 million ARR business.
Distilling the article: You need to focus and define your ideal customer profile(ICP). As a startup, you have one limitation: you can't try everything, as you have resource constraints, but you also have one advantage: you can be customer-focused and churn out features quickly. When these two things are combined, your only option is to make sure you understand the problem inside out. You should have many customer interviews & interactions (keeping the mom’s test principles in mind to gather feedback).
Using these, you can make a robust ICP and make a list of features that you need to serve your customer. This MVP can be given to the customer for feedback. Receiving this feedback and adding features that align with the company's long-term vision will determine your path to product market fit.
Though, it’s easier said than done. Honestly, when a customer is ready to pay it’s very hard to say no to that extra feature that your customer wants. But my advice, say no and keep on the lookout for your ideal ICP. We try to follow this philosophy, but every now and then we get distracted as the startup world is full of uncertainty and you have to make decisions with very little data. You cannot connect the dot forward, you can only do it backward.
Important note: don't think that any startup's example will apply to you. Show me an example of a startup strategy, and I will show you 10 startups that failed with the same strategy. Or better I will show you a more successful startup with a counter strategy. You can’t cherry-pick examples from one startup and apply them to the next. You can certainly take inspiration from an idea, modify it based on your current situation, and see if you can apply it.
2. Technology
Let's talk about Docsumo's journey of moving from a single Ec2 instance to K8S.
The architecture was straightforward when we first deployed my code: pull the code to an EC2 instance and run Gunicorn for Flask and nginx for React. This architecture was short-lived as systems got very complex fast. Then we separated the code base to handle auth, user settings, and document settings, so we took the worst of microarchitecture and monolithic apps and created a monster of our own. On the flask side, we used boilerplate code like flask-rest plus-boilerplate (P.S. use circleci at your own risk). For a repo, we chose GitLab, as they allowed free private repo, and moved to docker.
Docker was a lifesaver for us. Docker contained the database, code, backend, and frontend, and we used Docker compose to combine them. We deployed Docker in different EC2 instances. Once we had a few deployments every week, we decide to go with GitLab CI( Why you ask? Check out my presentation: CI/CD on Autopilot ). We initially used ssh to ec2 instance from the ci pipeline, pull docker and redeploy the code. The major issue with implementation was when traffic increased or if docker went down, we had to manually ssh to the server and fix it.
After this we were looking for container orchestration, We tried docker swarm but we had multiple issues managing the server, and overall, I was not happy with its CLI. At that time, I had heard about K8s. K8s was just a few years old and seemed like an interesting technology, but every article I read warned me not to use it due to its complexity and learning curve. Anyway, I decided to give it a try, and it was love at first sight. Over the weekend, I was creating clusters and destroying them repeatedly. It solved 99% of my problem, so I could not go back.
During that time, only GCP had managed clusters but it was expensive. We were using Digital Ocean, and they launched a managed cluster with three nodes for $30. That was the perfect opportunity. I was looking for our setup, stateless Docker, and a database in K8S itself. It was not considered best practice at the time, to have a stateful app in K8s; it is still not considered best practice today.
But the cost nudged us to give it a try, we did not follow the norm and tested it anyway. We ran 3 cluster testing, staging, and production. Testing and staging were $30 per cluster and production was $60. This cluster and vm handled the load for the initial 6 months. We ran the k8s cluster in digital ocean for 1.5 years, we just had 1 major outrage which caused the server to go down for 8 hrs. After that, we got some more credit from GCP as part of TechStars Barclays accelerator. We only moved it to GCP as we needed centralised logging, monitoring, a load balancer, and a more reliable system. The whole migration process was done with zero downtime using mongo mirror and other best practices.
In the beginning, I wanted to have total control and wanted to deploy everything on my own. This is when my mentality had taken a 360-degree turn; I no longer wanted to do that after going through all this pain. I looked at hosting EKL, promothesis, grafana, sentry, and more on my own, but I would not recommend it to anyone. Do the bare minimum, and let people who are experts handle the rest.
Today, the best-spent money is the $25 we pay to sentry every month. Your job as an engineer is to solve problems efficiently and fast with the resources you have. Do not go after shiny new things and make them more complex than it has to be.
On a side note: I have been following K3S, as Kubernetes solves a lot of issues with scaling, and networking, and I want to have my own cluster to deploy personal servers for some personal projects. Yes, I can always spin up K8S for my personal usage, but that's overkill, and where is all the fun in managing my own cluster?
Without a doubt, I would choose a managed cluster, but the engineer in me always wants to be in control. The satisfaction of having a whole cluster that I can manage and modify is hard to put into words.
3. Podcasts
It was great to engage in a conversation with Swagat Gyawali about Docsumo’s journey, our key learnings, future plans, brain drain in Nepal, and the future of Tech.
I hope you enjoy it.
4. Interesting links
Repos:
Balance: A sample from the population is imperative for DS/ML engineers. Here is a good library to make sure your sample has less bias.
30 Days Of Python: I get a lot of questions on how to get started in ML. The first step is always to get your python basics right. Here is a good repo for that.
Tuning playbook: Apart from sampling, tuning the model is equally essential in DS/ML’s life. Here is a very detailed guide from google research on the same.
Articles:
Who owns Generative AI: It’s common knowledge that generative AI is a big opportunity but also has big challenges. This a16z article captures the point perfectly. There is a saying in startups: No one buys your product for 2X better performance, rather for someone to even consider your offering, you should be 10X better and also solve their pain point.
Google Research, 2022 & Beyond: Language, Vision, and Generative Models: With all the hype from chatgpt and other generative models, let’s see if google has something up its sleeve.
Real-world engineering challenges: As an engineer, you will need to make so many choices. You are free to choose but be ready to pay the consequences of your choices. I appreciate the in-depth analysis of the choices others have made because it teaches me a lot when I have to make them. For example, if I had made better choices when I started designing Docsumo’s architecture, the team would have had a lot less trouble today.
Million-dollar products. The concept is simple: find a problem, figure out if that's something you want to solve, build great products, treat people well, and make money.
5. Quotes/ Books
Book recommendation: Crossing the Chasm and The Lean Startup
Two of the greatest books on product market fit and how to scale.
Workaholics miss the point, too. They try to fix problems by throwing sheer hours at them. They try to make up for intellectual laziness with brute force. This results in inelegant solutions. They even create crises. They don’t look for ways to be more efficient because they actually like working overtime. They enjoy feeling like heroes. They create problems (often unwittingly) just so they can get off on working more.
As a co-founder, I work 12 to 16 hours per day. But, one thing I make sure of is that our team has a work-life balance. At the end of the day, everyone at Docsumo must push an update on our slack channel. If people constantly update late, I make sure I talk to them and try to figure out why. We have hired you for 8 hours per week, 5 days a week. For these 8 hours, we need your undivided attention.
There will be times when you must work until 1 a.m., but this should be once in a while and not the norm. If someone cannot complete tasks in eight hours, you either hired the wrong person or you need to hire more people. In the latter case, people should be open enough to talk to you and work out a solution.
Working late is a recipe for disaster, and a startup is a marathon, not a sprint.
Our Slack channels have a great collection of memes.
That’s it for this edition, I hope you find it useful.
See you in two weeks!
Best,
Bikram Dahal
P.S. If you learned something new today, please share “Notes from a CTO” with your friends and spread the love. ✌🏻
Good to know this stuff. Thanks for sharing