About the work I’ve done for New York Department of Education

Since people are keeping asking me about this, and since I myself often mention my work for the NYC DoE as one of the most successful projects in my professional life, I thought it would be a good idea to tell in more details about it, what it was all about, and why this project is so important for me.

In December 2003 I was a consultant working for the Spherion Consulting Group. My assignment at this time was a new HR/Payroll system in the City of Chicago Mayor’s Office. I’ve being working on this project for quite a while by that time. One of our the Spherion directors, Ed, who worked there in the beginning of my engagement left to lead another Spherion project in New York. A couple of times he would email me asking whether I have any ideas on how to optimize some queries…

At the end of December I took several days of vacation and went to Russia with my kids, to celebrate the New Year with my family. When I returned, my then director Bob told me: Ed is asking, whether he can borrow you for a couple of weeks to help with some optimization problems they are having over there in the NYC DoE…

This was a start of the most amazing project I ever had. The problem our consultants in the DoE were facing was indeed a big one: the “match algorithm” worked fine on a relatively small subset of students. However, started on a half of the whole student body it would never finish! “Never” – I mean, they would let the program run for several days! So here I was: facing the system I never worked with, all new database I never saw before and 5,000 lines of PL/SQL program – and I had no idea what it was doing! And “this thing” was not working…

… I’ve spent two weeks working literally 16 hours a day; sleeping for 5-6 hours, getting out of the house for short walks twice a day… my kids were cooking our meals and bringing me food to my desk… I still could not reach a visible progress! By the end of week two, at about 11 PM I’ve emailed Ed: I am sorry! You brought me in in hope I will save the project, and I’ve being working for two weeks by now, and every day I feel like the success should be somewhere near, but nothing works… I am a failure… And he replied immediately (because that’s what he always did): Henrietta, I know you are doing your best, that’s what I am telling the people in DoE every day, and no matter what the outcome is going to be, I will never-ever consider you a failure!

This email gave me such a boost of confidence! And three days later, at the same late night hours, we were sitting at our desks – me being at my home in Palatine, Ed being in his hotel room in New York – holding our breaths and watching the first run of the full “match algorithm” – I knew, this time it was going to work, but I won’t believe it till I see it… And it finished! At 11-45 PM Chicago time. Being executed for 4.5 hours… that was a victory!

I ended up spending years, not two weeks on this project. Together with other team members, including Bill, we did quite amazing things, and there is not a single part of this project I am not proud of… but this night when the match algorithm worked for the first time, was something really special. Something I will never forget:).



Filed under Companies, Data management, People, SQL

2 responses to “About the work I’ve done for New York Department of Education

  1. It never ceases to amaze me how the best, most rewarding projects in our careers seem to start out as innocuous little assistance efforts. It’s almost as if rewarding success intentionally hides in plain sight!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s