Category Archives: Systems

Looking for New Ways to Bridge the Old Gap: New Ideas After the Conference

Before I went to this conference, I was resentful regarding the fact that the gap between applications and databases will never be closed. Even at the conference focused on both data engineering and software development, there was barely a place for me, and our talk barely got accepted.

I have to admit, I didn’t explore the program much before coming because I had never-ending work crisis, and we had to rework our presentation several times.

But when I took a closer look, I realized that I am way more interested in the SE sessions than in the database sessions. Day four, I could not miss a single moment, and I had several interesting conversations with the speakers.


It turned out that most of them were not even present on day one when I was giving my talk. And they said they would love to come if it won’t be on the first day.

Now I am wondering whether I did it right, never trying to present my work at the SE conferences. On the one hand, I am always saying that my success won’t be possible if I won’t have such an incredible backend team. On the other hand, I routinely say that inefficient programming is all application developers’ fault. That is not true.

One of the talks was about the refactoring techniques, and after the presentation, I asked the speaker whether he ever tried to consider taking into account the factor of accessing a database as a factoring criterion.
He replied that one of his colleagues tried to explore this option, but found it challenging: queries appear to be so entangled, so difficult to extract, that it led to nothing. I told him about my work and suggested that we would love to collaborate if he will find it interesting. He said that he would take a look, and then also mentioned that usually, the database people are not collaborating. He mentioned the lack of constraints and unwillingness to use views. I said that views are horrible because most of the time they decrease performance. But they provide a level of abstraction, he suggested. I replied – the are better ways! He said – well, then give them to us!

This exchange made me think that I am not explaining myself enough when talking about NORM. And if I know that successful implementation depends on cooperation with app developers, I should advocate for them.

Leave a comment

Filed under Development and testing, research, Systems

Introducing New Term

I’ve been thinking for a while how to name our methodology of data exchange between databases and applications. Several years ago, we came up with the term HDAT – Holistic Database Application Tuning.

This term didn’t get much recognition, partially because I didn’t really like it myself. It is pretty vague and implies that the subject we are talking about is something “nice to have” rather than entirely new technology.

In an attempts to highlight the most important feature of this technology – the fact that it eliminates a need in any ORM – I came up with the term which I am going to introduce today:  NoRM.

The letter ‘o’ stands for both ‘No‘ and ORM, to avoid double ‘o’ and any possible associations with “Noom”:). Also, it highlights the fact that this approach is natural, that this is “a new norm.”

From now on, I am going to call “this JSON thing” a NoRM approach or NoRM technology.  

How do you feel about it? Do you like this term? Will you use it?

1 Comment

Filed under Systems

About My Nomination, And How To Vote

First of all, a big THANK YOU to everybody who reached out congratulating me for becoming a finalist in the “Technologist of the Year” nomination. This nomination is especially important for me, because I’ve always strived to apply the best CS theories for the success of the business. I do not believe in approaches, which can’t be used in practice. However, I think that applying the right theoretical principles in the industry can have a tremendous impact.

Another aspect important to me is that all my innovations are related to PostgreSQL. If I were asked to name the three most important things which I’ve introduced at Braviant Holdings, it would be

  • The wide usage of FDW both in OLAP and OLTP
  • The usage of pg_bitemporal in both OLAP and OLTP
  • Abandoning ORM and using JSON -based data exchange between applications and databases

There is more in my blog about all of the above, but what I want to point now – each of these Top 3 is about using PostgreSQL in an innovative way.

The award descriptions say:

Presented to the individual whose talent has championed true innovation, either through new applications of existing technology or the development of technology to achieve a truly unique product or service.

Isn’t it precisely what I just said :)? Do I want to win? Absolutely! Do I think I can win? Yes! Can you help me :)?…

Several people reach out to me, telling me that they have difficulties casting their votes. I agree that the voting process is at least contra-intuitive. So let me explain it step by step.

First, you go to that link.

Then, click where it is said to CREATE LOGIN. It says that you can login with your Facebook account, but this does not work. So you will need to create a login. After that, you need to click on the large grey “Like” on the very top. Wait for a response to make sure your vote is counted.

Also, there are SHARE buttons, and unfortunately, the most important one – Share on LinkedIn – does not work. Others work fine, so you can help me by sharing with your network 🙂

And one more thing – this voting is only opened till August 16, so please don’t delay 🙂

Once again – THANK YOU!

2 Comments

Filed under events, news, Systems

PostgreSQL And Academia

Recently I’ve been thinking a lot about relationships between the PostgreSQL community and the DB research community. To put it bluntly – these two communities do not talk to each other!

There are many reasons why I am concerned about this situation. First, I consider myself belonging to both of these communities. Even if right now I am 90% in industry, I can’t write off my academic past and writing a scientific paper with the hope of being accepted to the real database conference is something which appeals to me.

Second, I want to have quality candidates for the database positions when I have them. The problem is more than scientists do not speak at the Postgres conferences, and Postgres developers do not speak at the academic conferences. The bigger problem is that for many CS students, their academic research and practical experience to not intersect at all! They study some cool algorithms, and then they practice their SQL on MySQL databases, which as I have already mentioned multiple times, lacks so many basic database features, that it hardly can be considered a database!

If these students practiced using PostgreSQL, they would have a real full-scale object-relational database, not a “light” version, but a real thing, which supports tons of index types, data types, constraints, has procedural language, and the list can go on and on.

It is especially upsetting to see this disconnect since so many database researches were completed on Postgres, for Postgres, with the help of Postgres; R-trees and GIST indexes, to name a couple. Also, the SIGMOD Test of Time Award in 2018 was given to the paper “Serializable isolation for snapshot databases”, which was implemented in Postgres.

I know the answer to the question “why they do not talk?” Researches do not want to talk at the Postgres conferences, because those are not scientific conferences, and the participation in these conferences will not result in any publication. Postgres developers do not want to talk at the CS conferences, because they do not like to write long papers :), and also, even if they do submit something, their papers often are rejected as “not having any scientific value.”

I know the answer. But I do not like it :). So maybe – we can talk about it?!

2 Comments

Filed under research, SQL, Systems

Let’s Go Bitemporal!

Dear friends and followers from the Postgres community! Today, let’s talk more about the bitemporal library (as if I did not speak enough about it yet!).

We have been developing Postgres functions, which support bitemporal operation for almost four years by now. We have found our initial inspiration in the Asserted Versioning Framework (AVF), first introduced by Jonson and Weiss nearly twenty years ago. There is nothing new in the concept of incorporating time dimension into data, and even the concept of two-dimensional time is not new. However, we believe that AVF approaches the task in the best possible way and that it allows making the time a true and integral part of data.

We believe that Postgres is suited the best to support a two-dimensional time due to the tow factors: the presence of the interval types and GIST with exclusion constraints. Having these two available made the process of implementation of the concept more or less trivial.

Implementation of bitemporal operations took some time, though, and we are still in the process of improving some of the functions. However, we are happy to share with the world, that Bravinat Holdings runs both OLTP and OLAP databases on the bitemporal framework with no performance degradation. Since we had an opportunity to develop as we go, we could address lots of issues in this implementation, which we initially did not even expect.
Recently we have uploaded several files into the docs section of the pg_bitemporal GitHub repo, including several presentations and short papers so that those who are interested can read more on the theory of bitemporality. We hope that people will give it a try – it works! Also, we are always looking for volunteers who will be interested in collaboration.

Please check us out at https://github.com/scalegenius/pg_bitemporal

Leave a comment

Filed under research, SQL, Systems

DB developers and App developers

Last week when I was at the PG Conf in New York, I met with my former co-worker, with whom we worked together while being on the New York Department of education project. She was a Java developer, and I was an Oracle developer.  I was implementing a solution, which was very close to the idea I am implementing now – building Oracle packages for each application endpoint, so that an application could only call functions, not separate SQL statements. Our collaboration was not always easy and peaceful those days, and when I was describing to her the specifics of the work we were presenting, I told her: see, if I would return to you the same type of objects back then, you won’t have any trouble with me!

She replied: Oh yes,  if that would be the case, I wouldn’t have any trouble with you! And the way she’d said that reminded me that she indeed had some troubles with me back then… And I thought to myself, that perhaps it’s not only the team I work with now is better than any other team I’ve worked with before (though it is still the case :)), but also that I myself had changed. And now I am way more flexible and willing to walk this extra mile, than fifteen years ago…  and our success is a product of collaboration, of everybody being able and willing to work together to find the right solution.

Leave a comment

Filed under People, SQL, Systems

If we need this data for reporting purposes only…

I always thought I have a well established opinion on separating application needs and reporting need. So whenever I was asked to add some attribute to the production system, which is “necessary for reporting”, I would always say: we will build this dada in the reporting system.

So first I could not believe what I just said, when I’ve said something to the effect “I will create this table, which application does not need, and I will use it to record data, which we need for reporting. Literally, after I’ve sadid this I thought – what in the world I am saying?! But then I was very positive that we indeed need this data to be recorded, and I’ve been keeping thinking about it…

And then I’ve realized, that the “reporting” I was referring to is not really a “reporting”. We just historically use the reporting system to go for this information, but in reality, those ones are not analytical reports, but operational reports, which raise alerts when some processing errors occur – mostly in the daily batch jobs. And then I started to think – probably we should treat those operational reports differently? They are not processing big data volumes, they are actually exception reports, and may be we even need to create an “exception processing service” in our production system?

Yes, just to prove, that I am right, of cause 🙂

and data, which is required for reporting. I think it makes lots of sense –

Leave a comment

Filed under Data management, Systems