Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

For a primer on materialized views, and one of the key rationales for Materialize's existence, there's no better presentation than Martin Kleppman's "Turning the Database Inside-Out" (2015). (At my company it's required viewing for engineers across our stack, because every data structure is a materialized view no matter where on frontend or backend that data structure lives.)

https://www.confluent.io/blog/turning-the-database-inside-ou...

Confluent is building an incredible business helping companies to build these types of systems on top of Kafka, Samza, and architectural principles originally developed at LinkedIn, but more along the lines of "if you'd like this query to be answered, or this recommender system to be deployed for every user, we can reliably code a data pipeline to do so at LinkedIn scale" than "you can run this query right away against our OLAP warehouse without knowing about distributed systems." (If it's more nuanced than this please correct me!)

On the other hand, Materialize could allow businesses to realize this architecture, with its vast benefits to millisecond-scale data freshness and analytical flexibility, simply by writing SQL queries as if it was a traditional system. As its capabilities expand beyond parity with SQL (though I agree that's absolutely the best place for them to start and optimize), there are tremendous wins here that could power the next generation of real-time systems.

EDIT: some clarifications and additional examples



I also wrote a primer for why the world needs Materialize [1]. It had a big discussion on HN [2], and Materialize's cofounder said it was part of his motivation [3].

[1] https://medium.com/@lironshapira/data-denormalization-is-bro...

[2] https://news.ycombinator.com/item?id=12613586

[3] https://twitter.com/narayanarjun/status/1241450203095465986


Ha! Your blog post was one of the reasons that I trusted in the future of Materialize enough to decide to work here!

I agree, that is exactly the problem that I, in particular, think we are solving.


That's awesome, thanks for fixing denormalization :)




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: