Last month we launched our website coronainbangladesh. In the span of a month, we’ve received over 50,000 visitors, news coverage and an acknowledgement in the official Bangladesh Government website. However, having such a high amount of traffic comes with it’s own set of challenges. I’ve had previous experience developing production systems, but those were mainly Bots, Business integrations and CMS sites which don’t receive a fraction of the traffic this had.
This article is a series of lessons I learnt throughout the development and maintenance of coronainbangladesh. While reading, do keep in mind this guide is meant for novices.
So let’s begin
Lesson #1 - If you don’t have previous experience, use a Platform as a Service
- Simplified deployments - Deploy with just one command and with zero downtime
- Automatic scaling - Platform scales your app on the fly depending on the amount of traffic
- Multiple deployment slots - Test your code in a “production” environment before exposing it to the the public
Now PaaS aren’t a silver bullet, they have their own set of problems. They usually cost more and aren’t as easily customizable and flexible as a conventional Virtual Machine. However, they can serve as a really useful set of training wheels when you’re still learning the ropes.
Lesson #2 - Code like you’re being attacked (because you probably are)
The first time we got DDoS’d was on day 5. I have no idea why anyone would want to attack something as benign as a our website, but there it was. The chances of your website being attacked or exploited increases with the amount of traffic it receives. So security should be a first priority from the very beginning of development, not something done as an afterthought.
First start with the basics - Securing your API endpoints, use SSL, hash all user passwords, turning on CORS etc. You can find great framework specific security best practices all over the internet. Go through them before going to production. We followed this checklist on the express official website.
Lesson #3 - Constantly monitor your server resources
Keep an eye out for how much Memory, CPU, Bandwidth your system is consuming. These metrics are an important indicator of the overall health of your system. Anomalies in any of these can be due to bugs, memory leaks or other performance issues. This can also give you an idea about resource utilization and help you decide to scale up or down a specific resouce. If you’re using any modern cloud provider, then a lot of these monitoring tools should come built in.
Additionally, make sure to clean up any resource after you’re done using it. Otherwise you might find a nasty surprise waiting for you in your monthly bill. At one point I forgot to disconnect from the database when I was closing my server instances. In the end we had about 300 concurrent connections to our database server just from my development computer….yikes
Lesson #4 - Keep the scope of your project in mind
Decision you make should always consider the scope and scale of your project. Just because something make’s sense for Amazon does not mean it will also make sense for you. There is no one size fits all solution in software development. Over engineering can cause significant losses in time, money and resources.
In our case, when we needed to add a cache, everyone unanimously decided we should use redis. There wasn’t any real thought behind it, it’s just what we had all read on the internet. However, for our use case depolying a full fledged redis server was complete and utter overkill. It was costing us more than our web and database server combined and we weren’t even using it to 1% of it’s full capacity/bandwidth. A few days later, we decided to ditch it completely and just use a super simple in-memory caching library from npm. This worked perfectly, was easier to use and it was completely free.
Lesson #5 - Test, Test and Test some more
I’l start with a confession, I don’t like writing tests. It’s painstaking, repetitive and utterly boring. So naturally, I didn’t write any tests when we launched our website. And I got screwed. Weird bugs were popping up, dependencies were breaking and these problems manifested themselves in the most unsuspecting of ways. The source of the bug is never in the first place you look. I had to spend hours chasing down trivial issues that would have been immediately apparent if we had a comprehensive test suite.
I understand you’re working with limited time and resources. But you will soon realize that, writing tests actually saves you time in the long run. It’s an investment that ends up paying massive dividends. But if you really don’t have time to write a comprehensive test suite, prioritize and start with API (Component) testing. This is the easiest way to write and provides more coverage than unit testing (you may even craft API tests without code using tools like Postman).
Conclusion
When you’re trying to deploy your first website, it can feel quite overwhelming. There’s dozens of things you have to do perfectly. Worst of all, everyone kind of assumes that you already know all of this. I just want you to know that feeling is totally normal. You will make mistakes, but you’l also have the opportunity to learn from those mistakes.
People learn best from experience. So want to become a better developer? Go out there and make beautiful random things. See them break, and then fix them. It’s a brave new world!
If you have any questions, feel free to contact me at tasinishmam@gmail.com