I’ve just looked at my stats and realized, that I haven’t being posting anything for an entire month (till yesterday :))! In my defense – I didn’t have a single day off for the past 3 weeks, so this weekend was the first time since the beginning of June when I’ve started to get back to normal, back to life. More or less:)
I’ve spent the whole month of June building a new Data Warehouse for my company. I had a pretty good idea what I want to build, but the timing was really-really tight. I know – not only theoretically, but from my experience, that no matter how carefully you plan, there will be unplanned things which will slow you down, and that basically you need to plan for unplanned 🙂 – but it’s always hard. Especially when you know exactly, what you want to do, and you feel really frustrated when you are limited by the speed with which you can type, and by the time it takes to recompile a function or to execute a query.
That was the first time in my professional career, that I had to build the whole system from scratch, and to build it exactly the way I wanted. Ironically (or, may be not!) it was also the first time I had to sacrifice the “purity” to “what business needs”. No, do not take me wrong, actually, the at the end the business needs things to be done right, but I always hated when I was told: we need to build this thing fast now, and later we will optimize – make it right, etc. I used to always scream, that “later” will never happen, and now I made a conscious decision myself to do exactly that! And I do not regret, because now it’s me who has to control, that “later” will actually happen.
This morning I am looking back at the previous three weeks and it’a amazing. There are so many things I did for the first time! The first time I’ve actually integrated the heterogeneous source systems, and not just one-time sync, but scheduling it on a regular basis. That was the first time I’ve used the Postgres foreign data wrappers with materialized views over them, and it works unbelievably well! The process is so simple and straightforward, with close to zero chances of failure. Although not the whole system is functioning now, it’s a 100% reliable source of information, and if there will be a need to extract any data “out of schedule” I will be able to do that even if the actual report is not completed yet.
And the last thing – I do not want to give an impression, that I am the lonely warrior – I won’t be able to get as far as I did without my amazing co-workers! With organization as small as ours’ at that time everybody’s contribution is clearly seen, and I can’t thank come each and single person working alongside me. In short – I am really happy with what we’ve built so far!