Tag Archives: professionalism

2019 Highlights

Today, on the first day of 2020, I am looking back at 2019 – what an amazing year it was for me! Here is what I am especially happy and proud about:

  1. Undiubtfully, the most outstanding professional recognition I ever received was my ITA Technologist of the year award.  What made it especially significant for me was the fact that I ended up being the first female ever receiving this award. 
  2. After three years of rejections, I finally have another paper accepted for the academic conference. I will be presenting at SOFSEM 2020 in Cyprus
  3. I ran a full-day training for the first time in my life, at 2Q PG Conf in Chicago. That was way more work than I initially thought, but I am pleased with the outcome. 
  4. Also for the first time, I was a part of the talk selection committee (for the same conference) 
  5. It was also a fantastic year for the Chicago PUG. We are now the third-largest PUG in the Western Hemisphere, and I can’t wrap my head around it. When did it happen?! I am so thankful for all of the Chicago PUG members who were attending our meetups this year! Huge thank you for all the speakers, who made our year so memorable, especially our November Meetup!
  6. Last but not least: it is my fourth year at Braviant Holdings, and this is the first time in my life I can design and build the system from scratch, the way I believe is right, and to prove that it is indeed the right way – works as expected 🙂 Also, I am working with the most wonderful team ever. 

The only thing I can wish for myself in 2020 – for things to continue to be as good as they are. Also, I have two announcements to make. 

  1. For those of you who wanted to attend my training but didn’t get a chance – I will do it again in New York at the end of March. More details to come, stay tuned.
  2. I am looking for ways to further improve Chicago PUG meetups. Some people expressed a desire to have “PG lunches” in addition to the evening meetups. If you are interested and want/can help with the organization – please reach out to me!

One more time, big thank you to everybody: my co-workers, my managers, Chicago PUG, and Postgres community at large – nothing would happen without your continuous support!

1 Comment

Filed under events

I knew I won’t like this feature! About the parallel execution

Many years ago, when Postgres 9.6 was still in making, my coworker said to me with excitement: Hettie, Postgres will now have the ability to run queries in parallel! There can be only four parallel processes, but still, isn’t in nice?!

And I remember exactly what I’ve replied: no, I do not like this feature a bit! You know why? Because executing queries in parallel would rarely solve performance problems. In fact, if four parallel workers would solve your performance problems, they were not really problems! It can do more harm than good, because it can mask some real performance problems for a while, and then they will turn just to be more severe.

What happened then – I’ve started a new chapter of my life at Braviant, and we had Postgres 9.5 then, and then for almost 3 years it was like I never had time to stop and upgrade :). But now I have a team, so we’ve finally planned the upgrade, and since we were already four versions behind, we planned upgrade to 9.6 and immediately to PG 10.

We’ve started from our Data Warehouse. First – in the staging environment, we we’ve tested, and observed the execution for some time. And then on the mirroring instance, and only then – on production.

And then it started! Seriously, out of all data refreshes this one is only one, which is important for the OLTP instance, because it sends data to one of our client interfaces. It started to behave inconsistently, Sometimes it would be just fine. Other times, instead of about 50 seconds it has been running for an hour, and probably won’t finish if we won’t kill it. Yes, it was obvious, that something did change in the execution plan after the upgrade. But what?! From the first glance the execution plan looked the same, all HASH JOINS, which you would expect, when you join tables with no restrictive conditions.

But it was still painfully slow. What was more puzzling – I could take out of the equation JOIN to any table, and performance would be unpredictable. After several dozen of attempts to make things run decided to take a closer look at the execution plan. And you know what I saw?! Yes, parallel execution/! Four joins were parallelized, which resulted in the execution time been really horrible. After the issue was found, the only thing left was to figure out, how to turn this feature off 🙂

Leave a comment

Filed under Data management, Development and testing, SQL

Can you teach somebody to optimize?

I’ve got a lot of feedback on my last blogpost; one question was posted on another paltform where I’ve reblogged the same text, and this question was so interesting that I’ve decided to write a separate post in reply.

So tell me, Hettie, are these kind of discoveries being reported at the conferences, and would they later become a part of a common knowledgebase? Will this sort of technique be taught in colleges? Overall, in your opinion, are the nowadays CS graduates more knowledgeable in this area. And, by the way, is there any special knowledge which is necessary to be able to resolve problems like this, or it’s just a combination of basic knowledge plus experience?

Great question! I have being teaching optimization for almost 15 years, and in general my optimism on this subject is very modest. You can teach technique, you can show tons of examples, and still there is no guarantee that a student who has attended your class will be able to correctly identify a similar pattern in the real life and to recall what specific technique which was advised for a similar problem. But may be I am just not good in teaching this stuff.

It’s tempting to say, that all that matters are years of practice, but this is also not always the case, since you, and me, and all of us can recall the situations in which years of experience did not help. And to be honest, I do not want to wait for “years of experience”! I want to be able to hire a new grad, who can optimize at a reasonable level. And I am not saying I never met this kind of CS grads, but what I am saying is that whenever it happens, it is due to the person’s individual abilities and/or a desire to excel in this particular skill.

Let’s be clear: the kind of breakthrough as I’ve described in the previous post does not happen often. In fact, you might never get anything like this in your live. But there are still tons of optimizations which can be done almost every day.

I would still argue that knowing the basics is a key. For the thousandth time over – you need to know your math, your calc and your algebra in order to understand how databases work. You might not be aware of some sophisticated indexes, but you should be able to identify, looking at the query, what’s it about< whether it is "short" or "long". And if you try and try, and it does not become faster, you need all your convincing powers to convince yourself, that this query can be optimized. There should be a way.

Another big thing I am trying to teach – to write queries declaratively. This is an extremely challenging task, because most of the requirements are formulated in an imperative manner. So what I am trying to teach, is that even if you find something like “previous three occurrences”, or “return to the previous status”, or “compare the top two” in the requirements, you still can write a declarative statement, carefully using CASE, GROUP BY and sometimes window functions. And it’s amazing, how fast everything starts running right away. Most of the time being able to reduce the number of table scans to one does the trick, except of… well, except of the situation, when you should do exactly the opposite.

I didn’t figure out yet, how to teach to distinguish one from another :). But the more I think about it, there more it seems like that’s what signifies that somebody can optimize, and that skill is the most difficult to teach. Most optimization classes teach you how and when to use some indexes, and how to choose the join order, but they do not teach how to rewrite a query itself.

… For the original question: no, I do not think they teach it in school. But I am trying to promote this idea!

Leave a comment

Filed under Data management, SQL

Can you teach somebody to optimize?

I’ve got a lot of feedback on my last blogpost; one question was posted on another paltform where I’ve reblogged the same text, and this question was so interesting that I’ve decided to write a separate post in reply.

So tell me, Hettie, are these kind of discoveries being reported at the conferences, and would they later become a part of a common knowledgebase? Will this sort of technique be taught in colleges? Overall, in your opinion, are the nowadays CS graduates more knowledgeable in this area. And, by the way, is there any special knowledge which is necessary to be able to resolve problems like this, or it’s just a combination of basic knowledge plus experience?

Great question! I have being teaching optimization for almost 15 years, and in general my optimism on this subject is very modest. You can teach technique, you can show tons of examples, and still there is no guarantee that a student who has attended your class will be able to correctly identify a similar pattern in the real life and to recall what specific technique which was advised for a similar problem. But may be I am just not good in teaching this stuff.

It’s tempting to say, that all that matters are years of practice, but this is also not always the case, since you, and me, and all of us can recall the situations in which years of experience did not help. And to be honest, I do not want to wait for “years of experience”! I want to be able to hire a new grad, who can optimize at a reasonable level. And I am not saying I never met this kind of CS grads, but what I am saying is that whenever it happens, it is due to the person’s individual abilities and/or a desire to excel in this particular skill.

Let’s be clear: the kind of breakthrough as I’ve described in the previous post does not happen often. In fact, you might never get anything like this in your live. But there are still tons of optimizations which can be done almost every day.

I would still argue that knowing the basics is a key. For the thousandth time over – you need to know your math, your calc and your algebra in order to understand how databases work. You might not be aware of some sophisticated indexes, but you should be able to identify, looking at the query, what’s it about< whether it is "short" or "long". And if you try and try, and it does not become faster, you need all your convincing powers to convince yourself, that this query can be optimized. There should be a way.

Another big thing I am trying to teach – to write queries declaratively. This is an extremely challenging task, because most of the requirements are formulated in an imperative manner. So what I am trying to teach, is that even if you find something like “previous three occurrences”, or “return to the previous status”, or “compare the top two” in the requirements, you still can write a declarative statement, carefully using CASE, GROUP BY and sometimes window functions. And it’s amazing, how fast everything starts running right away. Most of the time being able to reduce the number of table scans to one does the trick, except of… well, except of the situation, when you should do exactly the opposite.

I didn’t figure out yet, how to teach to distinguish one from another :). But the more I think about it, there more it seems like that’s what signifies that somebody can optimize, and that skill is the most difficult to teach. Most optimization classes teach you how and when to use some indexes, and how to choose the join order, but they do not teach how to rewrite a query itself.

… For the original question: no, I do not think they teach it in school. But I am trying to promote this idea!

Leave a comment

Filed under Data management, SQL

The magic of the right answer

When you are in school, you think that the the most difficult questions are those on the final exam. I remember how I’ve anticipated this happy future when I will never-ever have to take any more test! Sure…

Besides of the fact that later I went on to get my PhD, and voluntarily subjected myself to the new battery of tests and exams, not mentioning the actual dissertation… besides of all this – the most difficult questions are not on the test. That just appear. Show up. In course of your daily work, when you take on a new project, or when you firmly believe that your project is almost done… and then comes a problem. This problem. And a cost of not coming up with a correct answer is way higher than getting B instead os A on your final.

Several days ago after a couple of back-to-back meetings I’ve got this feeling, that everything is falling apart. That my original design was only good in my head, snd that it did not survive the meeting with reality. And that felt like I’ve promised more than I can deliver, and that more than one project will need to be pushed back.

I left work a little bit earlier, biked from the train station home without any audiobook in my earbuds. The I called my mother so that this will be a checked box for that day. And then I sat down and started thinking. It might sound silly, but I’ve realized that the only thing I needed was to focus and to analyze all the inputs I’ve already had. I was not staring at the code. I was not even staring at the database schema. Not writing. Not drawing. Just imagining all of the things I need to fit into one picture in my head. “Imagine you are a database” :).

And then that was it! Before I could even analyze all the dependencies, I knew it was right! I’ve started going through all the difference scenarios in my head: yes, this will work! And in this case as well! And for the future, when we won’t need this particular functionality it will work as well! And it won’t require any additional data transformation!

The next thing I did – I’ve set up a meeting with my co-workers so that they could criticize me ideas and find the cases when “this” won’t work. (Fast-forward to the next day – we actually found more cases when “this” will work!)

And thinking about my 35 years in this profession – those are the best moments, the peaks of satisfaction. This very moment, when you can see the solution, and you know it’s the right one right away. It may be a super-cool optimization, when you improve a query execution time 500 times. It may be a new design, it may be a new way of writing functions. But this moment when you know that what you’ve just come up with is right – that’s the real magic.

The next nest thing is to call my husband and tell him what a cool thing I’ve just invented, and then run around the office with my laptop opened showing this cool thing to everybody 🙂

Leave a comment

Filed under Development and testing

What makes a manager awesome?

Recently I  was asked to submit my “Motivational inventory”. Among other questions there was one which made me think for a while: if I think about the best manager I ever had, what would make him(her) awesome? Here is what I’ve replied.

I was lucky to have several awesome managers,  and I can’t really rank them.  But there was definitely one thing in common for all of them- they were supportive.

They would trust my technical skills and would let me to do things “my way”, even when it was not “like people normally do”.

They would have patience to wait till I reach the desired results, even when I myself didn’t have patience .

They would help to resolve the conflicts between team members, and would make sure there is no bitterness left. It would often require multiple phone conversations, and sometimes felt like marriage counseling :).

They would recognize my achievements and make them visible to the upper management and clients. And they would support and motivate me at the moments when I would feel I am a looser and nothing works.

***

I think that being supportive is the most important role of any manager. Trust, openness, recognition of employees achievements – all of these things can be viewed like forms of support.

Leave a comment

Filed under People, Team and teamwork

What I am looking (and not looking) for

Since  I’ve been looking for  database developers and DBAs for quite some time now,  and since virtually everybody knows about this, people often ask me: what are you looking for? What skills and qualifications you are interested at? Who would be your ideal candidate?

Most of the time I reply: please read the job description. I know that all the career advisors tell you “apply even if you do not have some qualifications”, but as for my job postings, I actually need those qualifications which are listed as “required”, and I would really prefer the candidates, who have “is a plus” qualifications.

Also, there are definitely some big DON’Ts, which I wish I would never ever hear again during an interview:

  • when asked for the definition of the foreign key,  starting your answer from “when we need to join two tables”
  • when asked about normalization, starting from “for better performance”
  • when asked about schemas, saying that we use then for storage optimization

Today however, I was asked a different question: why you are saying that you are looking for skilled candidates, and at the same time you admit, that for anybody who will get hired there will be a long learning process? if a candidate does not know something, doesn’t it mean he does not have enough skills? Doesn’t it mean, (s)he is underqualified?

I thought for a while before I’ve responded. When I was first hired as a Postgres DBA, it was a senior position right away, although at that time I did not know any Postgres at all. But people who’ve hired me were confident that not only I can learn fast, but also that I can generalize my existing knowledge and skills and apply it in the new environment.

To build on this example, there are two pre-requisites for success: knowledge and the ability to apply it in the real-life circumstances.

I think, that a person who wants to succeed as a database developer or a DBA should possess a solid knowledge of the relational theory.  But it is not enough to memorize your Ullman or Jennifer Widom, you need to be able to connect this theory to the real-world problems. This is such an obvious thing, that I never thought I will need to write about it, but life proved me wrong :).

Same goes in the situation, when a candidate has a lot of experience with other database, not the one you need. Yes, different database systems may be different, and significantly different. Can somebody who is very skilled  Oracle DBA be qualified for a position of Postgres DBA? Yes, if this person knows how to separate generic knowledge from the systems specifics.

if you know how to read an execution plan in Oracle, and more importantly, why you need to be able to read it, you will have no problem reading execution plans in Postgres. If you used the system tables in Oracle to generate dynamic SQL,  you will know exactly what you want to look for in the Postgres catalog.  And if you know that queries can be optimized, it will help no matter what a specific DBMS is. And it won’t help, if the only thing you know is how to execute utilities.

… No idea, whether this blog post is optimistic or pessimistic, but… we are still hiring 🙂

1 Comment

Filed under SQL, Uncategorized