TimescaleDB released Hyperstore: a hybrid row-columnar storage engine for real-time analytics. We didn't understand much of that either at first! There's been a lot to keep up with in database-land. Let's start small and talk about row-based and columnar storage.
Relational databases (ye olde MySQL, SQLite, PostgreSQL…) have traditionally struggled with real-time analytics because they're not optimized to process large volumes of data across records as they're designed to prioritize data consistency. Analytics are often stored as time-series data: a sequence of data points collected and ordered chronologically. Each data point has a timestamp and a measurement to track visitor statistics, stock prices over time, weather measurements… which a relational database can struggle with in large sets of data.
Analytical databases mostly use columnar solutions (like ClickHouse, Snowflake…). Column data is co-located in storage in a columnar database, while row-based relational databases co-locate rows. If we'd store a set of first and last names in a row-based database, "Luke, Skywalker" and "Ben, Kenobi" would be co-located. A columnar database would store this as "Luke, Ben" and "Skywalker, Kenobi".
A row-based database can handle faster inserts because a row can be appended. Adding a third record to a columnar database is slower because it needs to modify the existing datasets. On the other hand, columnar databases are more efficient in reading specific columns across many records, like aggregating measurements over a timespan in real-time as they're already close together.
Which brings us to hybrid row-columnar storage engine: an attempt to get the best of both worlds. Hyperstore writes data into a rowstore to eventually migrate the data to a columnstore, allowing fast writes and powerful analytics.
The more you know! Now go make yourself popular and explain hybrid row-columnar storage to your unsuspecting dinner party guests.
