Recently we encountered some severe performance problems in one of our projects. Some SQL queries were very slow.
A certain query took over 6 seconds to finish in Azure’s Postgres service and about 350ms locally on my laptop. Both are unacceptable.
I don’t know much more than the postgres basics and the project was in all parts relevant here more or less new to me. But let me show you how I approach the unknown, what I learned and how I solved it.
By the way: The issue is a prime example of not spending more time on optimisation than what feels necessary at the time. So even though the initial code looks a bit dumb in hindsight I still consider this ok as the application was fast enough for over 2 years in production and as we later learned the issue was caused by some up to that point unprecedented data.
Now, let’s dive in!
The posts PostgreSQL: Partitioning big tables Part 1 and Part 2 describe the implementation of this performance measure from the database view. In an enterprise Java environment often a database is integrated by a JPA framework and closely coupled with the application. This 3rd part about partitioned tables in PostgreSQL shows additions necessary when using Hibernate.
Alongside various strategies for handling large amounts of data with indexes, PostgreSQL provides another feature: splitting the table with inheritance. This separation of a table’s data into different child tables is called “partitioning” in PostgreSQL.
In part one, we described the pro and cons of this method and its conditions of use. In this second part we describe the steps required to actually configure a partition for production. A third blog post shows the specifics to consider in a JPA Hibernate project. The proposed instructions are intended for use with a PostgreSQL version 9.2 database but may work on all versions from 8.4.
Alongside various strategies for handling large amounts of data with indexes, PostgreSQL provides another feature: splitting the table with inheritance. This separation of a table’s data into different child tables is called “partitioning” in PostgreSQL. Because it does require a bit of effort to implement, and because there are also limitations in how it can be used, you need to take some points into consideration when creating a partition.
This first part of the article presents the pro and cons and what to consider when using partitioning to improve performance. In the second part we will describe the steps to create a production-ready partition. A third blog post shows the specifics to consider in a JPA Hibernate project.