Tag Archives: teamwork

Why I like so much what I am doing

Many years ago, when I was about to graduate from the University, my not-then-husband asked me, what I wanted to do with my professional life: to write “smart” papers about how-everything-should-be, or to do something real? Because it was quite obvious, which answer he had expected at that time, I’ve answered: of cause, the latter one!

But speaking seriously, that was my goal through all my professional life. Yes, I do write the “smart” papers about how things should work, but all these discoveries are of little interest to me until I can make a practical usage out of them, and until I can prove, that what I think is right actually changes things for better.

I like to say, that “a database is a service”. There is nothing else in the world of information technology which is more remote from the end user, than the database internals. Our work manifests itself in a very not-so- straightforward way. And when the the absolutely theoretical approaches which I’ve developed, actually work the best possible way – there is nothing more exciting.

In the system which I am building right now, which is more than just an app, but the whole system, which includes interaction between different online services and the data warehouse(s) I am implementing all the ideas, which has been important for me for most of my professional career.

I am using the bitemporal model I talked so much about through the past two or three years, and it is fascinating to see that things I was hoping will work and have some value to the business, actually produce value!

I work with application developers to bypass the ORM, and to use the output of the database functions for the most efficient communication with the data storage. I did this many times before, but never before I’ve experienced that level of cooperativeness.

I am using the foreign data wrappers in a most extended manner, and literally eliminate the gap between the application databases and the data mart.

Everything I wanted to accomplish in different periods of my professional life – everything is coming together, and I can see that the results are coming out really … how I wanted them to be :).  And I can’t allow it to be different.

Leave a comment

Filed under Data management

The joy of team work!

Last week I had to work a lot! I mean, A LOT! But work-wise that was one of the happiest weeks, and I wanted to share with everybody why it felt so happy.

Those of you who worked with me before know, that my favorite activity is working with  applications, because optimizing for applications is way more difficult and way more fun than optimizing reports. Yes, you can impress everybody, including yourself, reducing the report run time from one hour to one minute. But how much cooler is it to reduce the page load time from 30 sec to 0.1 sec?!  Especially when you have a power not only to write the best queries ever, but also to design the data model “the right way”.

When you do the application database work, the most critical part is to work in close contact with the  application developers. And depending on what kind of team you are lucky or unlucky to have, it may be the best or the worst part of your professional experience.

My IT team here in Braviant is one of the best I ever worked with, which was proved one more time last week. The most difficult part has always been connecting the db work and the app work, like: I’ve selected all this data for you, can you read it from the output I am providing? Or: we can give you all input parameters that way, can you process them? Our app developers have already made a huge step “in my direction” agreeing not to use ORM, but to read the output of the database function. Next step – we hit the  wall exactly where I expected. I’ve spent a half of Saturday writing my code, so that the app developers could start using in Monday morning… and now they are saying they can’t process correctly the embedded record sets! I’ve heard that many times before, and each time in a couple of hours I would hear: Hettie, there is no way! Let’s do it “the old way”, we know how… That time, however my team kept trying to find a solution, and watching these efforts made me to start thinking how I can change the output on my side. After several iterations going back and forward, we came up with a pretty neat way to return the records, which could be used right away, and even a better approach, which, however would require more work from me, and could not be done on the spot.

And you know – I totally understand, when people hate rewriting one piece of code multiple times, which makes me appreciate even more the willingness to rewrite later, when I will come up with the more automated solution from my side…

I have already written a lot in this post, and I am not sure whether it all make sense, but let me try to summarize.  I loved that everybody were willing to compromise, to make adjustments, that there were no “just because” statements, that the whole team was focused on the goal to build the application right from the very beginning, so that we won’t need to worry about performance six months down the road.

Hope it will continue that way.  Except of me working on Saturdays part 🙂

 

Leave a comment

Filed under People, Systems, Team and teamwork

A major accomplishment

It’s almost the end of the year. And it sounds like the most improbable day to write something about work, especially voluntarily.  But I’ve being really busy working  for the past several weeks (and months, to that matter).

But now I have something really exciting to report. And that “something” happened on Wednesday last week. On that day we stopped the automated emailing of reports from the old datawarehouse. Which of cause means, that we have all needed reports set up to run and email results from the new datawarehouse, which I have being building for the past seven months.

On one hand it seems like it took too long – the data was there and available for a while. But it always takes lots of time to convince people, who are used to the old ways, that they need to switch to the new system, the the new system is better, more accurate, etc. Besides,as it almost always happens, on the day that we’ve announced that “the old reports will be discontinued”,  the whole bunch of reports never converted to the new system emerged out of nowhere…

We were able to convert all these reports, to validate their results with business, and to shut down the automated reporting as planned.  And that means a lot for me, because it’s a very practical result of all my work since I’ve joined Braviant.  I do not have any illusions regrading this “sunset”, I know that there may be still multiple cases of “I need to see this old report” and such. And even after all user access to the old system will be disabled, we’ll need to archive the data for auditing purposes. But in any case I consider the current status as a major accomplishment, and a very material result of my work.

Of cause I won’t be able to accomplish that task without my wonderful co-workers, and since we are a small company, the input of each individual made a huge impact on the successful achievement of that goal.  I am in a very good place now, in all possible meanings of this word 🙂

Leave a comment

Filed under Companies, Data management, Team and teamwork, Workplace

How I learned to love tests: both using and writing ones – part 2

Even when I would reluctantly admit I need to have tests on place, I never understood, why one might want to put the check for the number of tests you want to run in pg_tap. What’s the point? You know how many tests you want to run, so the only thing you need is to count the executions :). And when the number does not match it means that you didn’t count them correctly;)

That’s what I was absolutely sure about… until last week. A week before that I’ve discovered that I’ve mapped one foreign table incorrectly. Or, may be, it got changed and I didn’t notice – I didn’t have proper tests!

Nevertheless, after I fixed the table structure… yes, you are right, a number of tests failed! and since ai’ve added a whole bunch of newly mapped columns (21 of them, to be precise), I had to place 84 more tests… four for each column… and after I did it… and pg_tap reported that I ran less tests than I’ve planned. And my first inclination was to change the “number of tests I want to run”. And I almost did it… but then I thought: I remember I’ve counted! If there are less than 84 new tests, then there are two options: either I counted them incorrectly, or – I misses several tests.

It was not fun at all, going through this huge file with all the tests… but I found the missing ones! And I was so happy again, that somebody forced me to run the tests each time I am committing changes 🙂

 

 

Leave a comment

Filed under Development and testing, Team and teamwork

What I am busy with these days

I haven’t been posting anything for a while. I am still crazily busy at work, yet I’ve being feeling, that what I am doing at work right now is of almost no interest to anybody. And this feels paradoxic, because I have no doubt, that the work I am doing now is really important for company’s success. So I decides to try to break my silence and write a couple of short posts describing what I’ve being doing, hopefully it won’t be boring!

About three weeks ago we completed a very important project. I am saying “we”, although I personally did nothing, I didn’t write a single line of code. Yet all the people involved told me, that this project won’t happen without me.

The scope of the project was to clean up the objects ownership and permissions on all the database clusters which run our new platform. Doesn’t sound exciting, right? Could we continue operating without this cleanup? Definitely. But these discrepancies made many production deployments unnecessary difficult, because each time “something” might not work on “some” clusters. Besides, we could not set up the sating environment to look the same as production permission-wise.

This project called for the joint effort of DBAa, database developers and lead application developers. We did test the process the best possible way, but since we didn’t have an exact match of staging and production environment, there was still a possibility that something unexpected would happen. We’ve being switching ownerships and permissions one cluster at a time, with the system owners closely monitoring the execution of the apps. It took us three sessions to complete all the changes, and for most of the developers and the business users nothing really happened. Nobody noticed anything (which is a good thing, but then you feel like nobody really appreciate what you’ve accomplished:)).

I’ve got a moment of a real satisfaction when a couple of days later a very complex permission-sensitive deployment (one of those which would always cause pain for the application owners) went through without any issues!

Now we can model our staging environment from the correctly set production environment, and make all future permission changes even more safe and secure.

2 Comments

Filed under Data management, Systems

About the work I’ve done for New York Department of Education

Since people are keeping asking me about this, and since I myself often mention my work for the NYC DoE as one of the most successful projects in my professional life, I thought it would be a good idea to tell in more details about it, what it was all about, and why this project is so important for me.

In December 2003 I was a consultant working for the Spherion Consulting Group. My assignment at this time was a new HR/Payroll system in the City of Chicago Mayor’s Office. I’ve being working on this project for quite a while by that time. One of our the Spherion directors, Ed, who worked there in the beginning of my engagement left to lead another Spherion project in New York. A couple of times he would email me asking whether I have any ideas on how to optimize some queries…

Continue reading

2 Comments

Filed under Companies, Data management, People, SQL

What happens at the Hour of Optimization

I’ve being bragging about our “Hour of optimization” for a while, and now I want to talk about it in more details.

The original idea was that I will try to allocate some time for random optimizations: since people were coming to me with all sorts of optimization questions for a while, and a question could pop up in a chat at the most inconvenient time, like 5 PM on Friday, I’ve decided to have some “office hours”. This attempt was an epic fail, because literally nobody would come with their questions at that time, and everybody continued to bring me their questions at any other day/time of the week.

Then I’ve decided, since I already have this hour on my schedule, and since we’ve got so many newly hired database developers, to use this hour to show some optimization techniques, and how they can be applied for our tasks. This idea was definitely more productive, and I felt like people are learning something. On the other hand I knew that listening to the great optimization stories is not enough, that in order to master any optimization techniques you need to apply it at least ones.

So I’ve started to contemplate making other people talk at the Hour of optimization. First I’ve begun to approach those people about whom I knew for sure they performed some cool optimization task during the week. I’ve started to ask them to talk about these tasks – no formal presentation, just a conversation, and please show your code. After each of those presentations I’ve tried to initiate some exchange of opinions, some discussion: which technique was used, did we talk about it during my tuning classes, did we use it somewhere else recently? I liked it a lot when almost everybody started to participate. I’ve also started to invite people from other departments, since I’ve learned that many optimizations are going on all over the place 🙂

After some time I didn’t even need to know for sure that “somebody did something”. I was just asking: who would like to talk at the next Hour of optimization? Sometimes it’s even better: a developer may approach me and say: I have something for our optimization hour!

Yes, one can say that that’s a job of a database developer – to optimize queries. But I think it’s extremely important to reflect on your work, to think about the patterns you are using, to connect the current problems with the previous similar ones. Also, it gives the new database developers this feeling of almost visible growth of experience. We are definitely growing our database developers on the fertilized soil!

Leave a comment

Filed under SQL, Team and teamwork, Workplace