Category Archives: Systems

About My Nomination, And How To Vote

First of all, a big THANK YOU to everybody who reached out congratulating me for becoming a finalist in the “Technologist of the Year” nomination. This nomination is especially important for me, because I’ve always strived to apply the best CS theories for the success of the business. I do not believe in approaches, which can’t be used in practice. However, I think that applying the right theoretical principles in the industry can have a tremendous impact.

Another aspect important to me is that all my innovations are related to PostgreSQL. If I were asked to name the three most important things which I’ve introduced at Braviant Holdings, it would be

  • The wide usage of FDW both in OLAP and OLTP
  • The usage of pg_bitemporal in both OLAP and OLTP
  • Abandoning ORM and using JSON -based data exchange between applications and databases

There is more in my blog about all of the above, but what I want to point now – each of these Top 3 is about using PostgreSQL in an innovative way.

The award descriptions say:

Presented to the individual whose talent has championed true innovation, either through new applications of existing technology or the development of technology to achieve a truly unique product or service.

Isn’t it precisely what I just said :)? Do I want to win? Absolutely! Do I think I can win? Yes! Can you help me :)?…

Several people reach out to me, telling me that they have difficulties casting their votes. I agree that the voting process is at least contra-intuitive. So let me explain it step by step.

First, you go to that link.

Then, click where it is said to CREATE LOGIN. It says that you can login with your Facebook account, but this does not work. So you will need to create a login. After that, you need to click on the large grey “Like” on the very top. Wait for a response to make sure your vote is counted.

Also, there are SHARE buttons, and unfortunately, the most important one – Share on LinkedIn – does not work. Others work fine, so you can help me by sharing with your network 🙂

And one more thing – this voting is only opened till August 16, so please don’t delay 🙂

Once again – THANK YOU!

Advertisements

1 Comment

Filed under events, news, Systems

PostgreSQL And Academia

Recently I’ve been thinking a lot about relationships between the PostgreSQL community and the DB research community. To put it bluntly – these two communities do not talk to each other!

There are many reasons why I am concerned about this situation. First, I consider myself belonging to both of these communities. Even if right now I am 90% in industry, I can’t write off my academic past and writing a scientific paper with the hope of being accepted to the real database conference is something which appeals to me.

Second, I want to have quality candidates for the database positions when I have them. The problem is more than scientists do not speak at the Postgres conferences, and Postgres developers do not speak at the academic conferences. The bigger problem is that for many CS students, their academic research and practical experience to not intersect at all! They study some cool algorithms, and then they practice their SQL on MySQL databases, which as I have already mentioned multiple times, lacks so many basic database features, that it hardly can be considered a database!

If these students practiced using PostgreSQL, they would have a real full-scale object-relational database, not a “light” version, but a real thing, which supports tons of index types, data types, constraints, has procedural language, and the list can go on and on.

It is especially upsetting to see this disconnect since so many database researches were completed on Postgres, for Postgres, with the help of Postgres; R-trees and GIST indexes, to name a couple. Also, the SIGMOD Test of Time Award in 2018 was given to the paper “Serializable isolation for snapshot databases”, which was implemented in Postgres.

I know the answer to the question “why they do not talk?” Researches do not want to talk at the Postgres conferences, because those are not scientific conferences, and the participation in these conferences will not result in any publication. Postgres developers do not want to talk at the CS conferences, because they do not like to write long papers :), and also, even if they do submit something, their papers often are rejected as “not having any scientific value.”

I know the answer. But I do not like it :). So maybe – we can talk about it?!

2 Comments

Filed under research, SQL, Systems

Let’s Go Bitemporal!

Dear friends and followers from the Postgres community! Today, let’s talk more about the bitemporal library (as if I did not speak enough about it yet!).

We have been developing Postgres functions, which support bitemporal operation for almost four years by now. We have found our initial inspiration in the Asserted Versioning Framework (AVF), first introduced by Jonson and Weiss nearly twenty years ago. There is nothing new in the concept of incorporating time dimension into data, and even the concept of two-dimensional time is not new. However, we believe that AVF approaches the task in the best possible way and that it allows making the time a true and integral part of data.

We believe that Postgres is suited the best to support a two-dimensional time due to the tow factors: the presence of the interval types and GIST with exclusion constraints. Having these two available made the process of implementation of the concept more or less trivial.

Implementation of bitemporal operations took some time, though, and we are still in the process of improving some of the functions. However, we are happy to share with the world, that Bravinat Holdings runs both OLTP and OLAP databases on the bitemporal framework with no performance degradation. Since we had an opportunity to develop as we go, we could address lots of issues in this implementation, which we initially did not even expect.
Recently we have uploaded several files into the docs section of the pg_bitemporal GitHub repo, including several presentations and short papers so that those who are interested can read more on the theory of bitemporality. We hope that people will give it a try – it works! Also, we are always looking for volunteers who will be interested in collaboration.

Please check us out at https://github.com/scalegenius/pg_bitemporal

Leave a comment

Filed under research, SQL, Systems

DB developers and App developers

Last week when I was at the PG Conf in New York, I met with my former co-worker, with whom we worked together while being on the New York Department of education project. She was a Java developer, and I was an Oracle developer.  I was implementing a solution, which was very close to the idea I am implementing now – building Oracle packages for each application endpoint, so that an application could only call functions, not separate SQL statements. Our collaboration was not always easy and peaceful those days, and when I was describing to her the specifics of the work we were presenting, I told her: see, if I would return to you the same type of objects back then, you won’t have any trouble with me!

She replied: Oh yes,  if that would be the case, I wouldn’t have any trouble with you! And the way she’d said that reminded me that she indeed had some troubles with me back then… And I thought to myself, that perhaps it’s not only the team I work with now is better than any other team I’ve worked with before (though it is still the case :)), but also that I myself had changed. And now I am way more flexible and willing to walk this extra mile, than fifteen years ago…  and our success is a product of collaboration, of everybody being able and willing to work together to find the right solution.

Leave a comment

Filed under People, SQL, Systems

If we need this data for reporting purposes only…

I always thought I have a well established opinion on separating application needs and reporting need. So whenever I was asked to add some attribute to the production system, which is “necessary for reporting”, I would always say: we will build this dada in the reporting system.

So first I could not believe what I just said, when I’ve said something to the effect “I will create this table, which application does not need, and I will use it to record data, which we need for reporting. Literally, after I’ve sadid this I thought – what in the world I am saying?! But then I was very positive that we indeed need this data to be recorded, and I’ve been keeping thinking about it…

And then I’ve realized, that the “reporting” I was referring to is not really a “reporting”. We just historically use the reporting system to go for this information, but in reality, those ones are not analytical reports, but operational reports, which raise alerts when some processing errors occur – mostly in the daily batch jobs. And then I started to think – probably we should treat those operational reports differently? They are not processing big data volumes, they are actually exception reports, and may be we even need to create an “exception processing service” in our production system?

Yes, just to prove, that I am right, of cause 🙂

and data, which is required for reporting. I think it makes lots of sense –

Leave a comment

Filed under Data management, Systems

The best things from 2017 and what I am looking forward to in 2018

When I am talking to people about the year 2017, and what was important, I would repeat over and over: everything I was striving for during my 30+ years of professional career had happened. All wishes have come true. Which would inevitably lead to the question: well, Hettie, what are you going to next then?

And there are plenty and plenty of things I want to do, but first I wanted to reflect on 2017 one more time. I’ve already listed multiple times all our technical accomplishments, all the wonderful things I was able to implement in a real production environment.  But when I look back at what was the best, it is definitely working with our new tech team. As I am reiterating over and over again, “the database is a service”, and whatever we are doing inside the database, can only make any impact, if our results can be utilized by “somebody” – by our end users.

And most of the time our end users are  application developers.  I can’t imagine any serious database development without continuous interaction with the rest of the development team. For years my “golden standard” for the teamwork has been my work for New York Department of Education, where I had a full support of the tech leadership, which helped me to defend my approaches and produce the results I am still happy about.

But what has being going on in the past several months is even better. The way we discuss the user stories. The way we make decisions on what should go into the application, and what – into the database. The way we debug. The way we discuss, what is critical, and what’s not. The willingness of each of the team members “to go extra mile”, to adjust in order to make others job easier. Basically, whatever I could imagine for the teamwork being perfect, is there. 🙂

Being in this industry for 34 years, I know that nothing lasts forever:). I know that every tea,, every organization evolve, and that nothing is granted forever. But I also learned to be thankful for what’s going on right now, and enjoy the moment.

Looking forward to 2018 – I hope that our team will continue to be the most amazing team ever. I am looking forward for the stress -test of our new applications with high data volumes. After all, I was designing the data storage and data access to be super-scalable, and I hope that it will work this way.

I am looking  forward to use the performance data we’ll obtain to improve our bitemporal library, and I already have some ideas of what I exactly I want to do. Actually, I have new technical ideas on almost everything I’ve developed through the last year. I still didn’t get notifications on whether any of the papers I’ve submitted for the next year conferences were accepted,  but I know that both are very good papers :), so even if none will get accepted… I will try for another conference!

 

 

 

 

Leave a comment

Filed under Development and testing, SQL, Systems, Team and teamwork

The joy of team work!

Last week I had to work a lot! I mean, A LOT! But work-wise that was one of the happiest weeks, and I wanted to share with everybody why it felt so happy.

Those of you who worked with me before know, that my favorite activity is working with  applications, because optimizing for applications is way more difficult and way more fun than optimizing reports. Yes, you can impress everybody, including yourself, reducing the report run time from one hour to one minute. But how much cooler is it to reduce the page load time from 30 sec to 0.1 sec?!  Especially when you have a power not only to write the best queries ever, but also to design the data model “the right way”.

When you do the application database work, the most critical part is to work in close contact with the  application developers. And depending on what kind of team you are lucky or unlucky to have, it may be the best or the worst part of your professional experience.

My IT team here in Braviant is one of the best I ever worked with, which was proved one more time last week. The most difficult part has always been connecting the db work and the app work, like: I’ve selected all this data for you, can you read it from the output I am providing? Or: we can give you all input parameters that way, can you process them? Our app developers have already made a huge step “in my direction” agreeing not to use ORM, but to read the output of the database function. Next step – we hit the  wall exactly where I expected. I’ve spent a half of Saturday writing my code, so that the app developers could start using in Monday morning… and now they are saying they can’t process correctly the embedded record sets! I’ve heard that many times before, and each time in a couple of hours I would hear: Hettie, there is no way! Let’s do it “the old way”, we know how… That time, however my team kept trying to find a solution, and watching these efforts made me to start thinking how I can change the output on my side. After several iterations going back and forward, we came up with a pretty neat way to return the records, which could be used right away, and even a better approach, which, however would require more work from me, and could not be done on the spot.

And you know – I totally understand, when people hate rewriting one piece of code multiple times, which makes me appreciate even more the willingness to rewrite later, when I will come up with the more automated solution from my side…

I have already written a lot in this post, and I am not sure whether it all make sense, but let me try to summarize.  I loved that everybody were willing to compromise, to make adjustments, that there were no “just because” statements, that the whole team was focused on the goal to build the application right from the very beginning, so that we won’t need to worry about performance six months down the road.

Hope it will continue that way.  Except of me working on Saturdays part 🙂

 

Leave a comment

Filed under People, Systems, Team and teamwork