Learnings.Today

In this series, we are going to look at throughput optimization. Just to recap, throuhput is measured as the number of requests an app can handle in a second without compromising on the latency sla. Before we go deep into how to improve the performance let’s look at the flow in a typical web service. Most of the webservices, pull data from a data store, then processes it and serves it as part of the response. Serialization is involved in reading the request and serving the resopnse back. To understand better, let’s take a look at how a single request flows through.

Software is often delivered in different ways to different set of users. Some get it via the web, others access it through the cloud, and some get it on specific device like a smartphone or personal computer. All users have different expectations of how they use the software and it has impact on how they perceive performance of the software. From a product angle, a software meets the performance expectations of the user if it delivers the function/feature promised within the agreed time. From an engineering perspective, a software is seen as performant if it makes the most use of the resources (cpu/memory/nodes/network).

More often than not, we would be in a situation where we have to investigate cpu starvation. While it is easy to find the process consuming the most cpu cycles, it gets difficult to identify the specific thread that is consuming the memory. Identifying the thread helps in taking correcttive action on logic causing it. In this article, I am outlining the procedure to find the specific jvm thread that is using the cpu.

Recently, I was working on a computation problem that was based on dynamic programming. And of course, to store the sub-problem results, a cache was built to serve the purpose. Here is how the flow looked like after a cache was baked in: