Yes, it has a name: Toil!

From Google SRE Book, chapter 5 – Eliminating Toil:

So what is toil? Toil is the kind of work tied to running a production service that tends to be manual, repetitive, automatable, tactical, devoid of enduring value, and that scales linearly as a service grows. Not every task deemed toil has all these attributes, but the more closely work matches one or more of the following descriptions, the more likely it is to be toil:

This includes work such as manually running a script that automates some task. Running a script may be quicker than manually executing each step in the script, but the hands-on time a human spends running that script (not the elapsed time) is still toil time.

If you’re performing a task for the first time ever, or even the second time, this work is not toil. Toil is work you do over and over. If you’re solving a novel problem or inventing a new solution, this work is not toil.

If a machine could accomplish the task just as well as a human, or the need for the task could be designed away, that task is toil. If human judgment is essential for the task, there’s a good chance it’s not toil.21

Toil is interrupt-driven and reactive, rather than strategy-driven and proactive. Handling pager alerts is toil.We may never be able to eliminate this type of work completely, but we have to continually work toward minimizing it.

No enduring value
If your service remains in the same state after you have finished a task, the task was probably toil. If the task produced a permanent improvement in your service, it probably wasn’t toil, even if some amount of grunt work—such as digging into legacy code and configurations and straightening them out—was involved.

O(n) with service growth
If the work involved in a task scales up linearly with service size, traffic volume, or user count, that task is probably toil. An ideally managed and designed service can grow by at least one order of magnitude with zero additional work, other than some one-time efforts to add resources.

Also read:

Is Toil Always Bad?

Toil doesn’t make everyone unhappy all the time, especially in small amounts. Predictable and repetitive tasks can be quite calming. They produce a sense of accomplishment and quick wins. They can be low-risk and low-stress activities. Some people gravitate toward tasks involving toil and may even enjoy that type of work.

Toil isn’t always and invariably bad, and everyone needs to be absolutely clear that some amount of toil is unavoidable in the SRE role, and indeed in almost any engineering role. It’s fine in small doses, and if you’re happy with those small doses, toil is not a problem. Toil becomes toxic when experienced in large quantities. If you’re burdened with too much toil, you should be very concerned and complain loudly. Among the many reasons why too much toil is bad, consider the following:

Career stagnation
Your career progress will slow down or grind to a halt if you spend too little time on projects. Google rewards grungy work when it’s inevitable and has a big positive impact, but you can’t make a career out of grunge.

Low morale
People have different limits for how much toil they can tolerate, but everyone has a limit. Too much toil leads to burnout, boredom, and discontent.

Additionally, spending too much time on toil at the expense of time spent engineering hurts an SRE organization in the following ways:

Creates confusion
We work hard to ensure that everyone who works in or with the SRE organization understands that we are an engineering organization. Individuals or teams within SRE that engage in too much toil undermine the clarity of that communication and confuse people about our role.

Slows progress
Excessive toil makes a team less productive. A product’s feature velocity will slow if the SRE team is too busy with manual work and firefighting to roll out new features promptly.

Sets precedent
If you’re too willing to take on toil, your Dev counterparts will have incentives to load you down with even more toil, sometimes shifting operational tasks that should rightfully be performed by Devs to SRE. Other teams may also start expecting SREs to take on such work, which is bad for obvious reasons.

Promotes attrition
Even if you’re not personally unhappy with toil, your current or future teammates might like it much less. If you build too much toil into your team’s procedures, you motivate the team’s best engineers to start looking elsewhere for a more rewarding job.

Causes breach of faith
New hires or transfers who joined SRE with the promise of project work will feel cheated, which is bad for morale.

Replication Hell

Ever had an already bad day, and just right when you’re heading home you had to substring some strings up and do some data formalization for some lazy dude who needs not to look at row data for some reason that he always relates to your competence? I bet you can relate to. So, let’s move it, substring this, a little bit of that,, oh you need to lpad that? tough luck! It’s Sybase baby! Our lovely software from 1995. The reason you hate DBMSes so much. Should I buy Oracle license for the client? Or that’s too much reaction?

Okay, today’s not the day they see me sweat. I will REPLICATE the amount of padding needed and SUBSTRING it. Lazy devs at DBMS companies yields creative devs in companies using those. Now for the sake of this lovely evening, can I just head home?

Now to the real blog: I’ve encountered a very strange behavior for the REPLICATE function. Take the example of when you just want four zeros:


The result is obvious:


Now, a db savy will ask: what’s with the single quotations? Simply, dear, the replicate function takes a character to replicate. It’s just like they are preparing that to be used for a substring. For an LPAD! You had the scenario in mind, lazy bastards. Complete it and make us an LPAD. Oh.. never mind.

The funny behavior I encountered, in a very non-funny manner, is when you forget to put the single quotation, i.e. give a number to REPLICATE:


it results in:


WTH! (That 32 chars for those who does not enjoy their eyes getting hurt). The trick is, REPLICATE does convert numbers to hex before replicating them. I have no idea why and for what, but I just noticed it:

REPLICATE(1,2)   —>  0100000001000000

REPLICATE(10,2) —>  0a0000000a000000

REPLICATE(15,2) —>  0f0000000f000000

I really hope there’s an explanation or use case for that, not just that’s how they get chars replicated and they couldn’t manage the exceptions. I really do wanna feel I had some value from my money in this crappy software.


Too much to ask? Okay!

iReport Top N

If you had the [unfortunate] chance [or punishment] to make a Java report using iReport tool, and that report should’ve get a parameter to use in a select statement (the most popular example is select top n …), you might (most probably) flew into the wall of this error:




Now the only logical question you get when this error occurs is: What the hell? There is no @p1 in the goddamn query!!  Well it turns out that you cannot insert a parameter into the SELECT part of the query (you know, the part before you start WHERE conditioning the hell out of the query).

Now, to the silver lining: How to hack that up. Turns out, there’s a little hack you can do:

  1. Create your parameter as you would, name it and define it, and use it as prompt as you normally would.
  2. Create another parameter, let’s call it Controller, and define its default value as the query (which include the first parameter in wherever place you need). Do not forget not to use this controller as a prompt.
  3. Now in the query box in iReport, put the P!{Controller} parameter. Just type it in the box and leave.

What will happen is that, the report will get the prompted parameter, substitute it in the controller parameter, and takes the controller (now full query) and execute it as the query. Just remember that if your [web] app using this report is validating the query before executing (that is, substituting values), it will not be happy! Smart right? This is one of the coolest hacks I’ve encountered lately!

A small problem may occur: Formatting. Let’s take the following example: if your query has another parameter that is a date. When defining it as a prompted parameter, it will be substituted into the controller like the other parameter, and that is fine. The problem is that it will be substituted in an ugly format that you cannot control (Sun Feb 03 00:00:00 AST 2013 as opposed to 3/2/2013), and it will be a string so you cannot parse back into date in the query. The only way you can work around that is to convert whatever you are comparing it to in the query to the same format, or parse the text somehow. Await for my edit on this post with some examples…

Happy Again!

Off the grid is a goal I can never seem to be able to achieve…

I can’t afford being off the grid. Family responsibilities, work issues, catching up with friends, and various social interactions that for a while are inevitable…

Aside from sleeping time, I was offline for a total of 14 hours last six months. All were forced (12h on-flights, 1h internet cutoff and roaming problems abroad, and 1h battery death). But what exactly I mean when I say offline? Glad you asked…

(actually, since I am my only reader, I’m glad I asked 😀 )

Offline is the state of me not having any cellular, landline, or internet connection to my tier-1 people. The state of not being able to hear the Mayday from people who would direct their emergency call for me. Most of the time I love to silence all communications around me, but I never close the gate for my tier-1 people. I even have another number named the mayday number, which I change whenever someone calls me there for a non emergency issue, and never give it to him ever again.. But, that’s another topic…

The tier-1 people. My people. Those whom I answer for their urgent needs. The number at any given time of those never goes beyond ten. Those I welcome 24/7 and won’t ever be offline for (ehm, except for that 0.31% downtime!). How are people who are there, are there?  That’s usually by birth. But some people got there without sharing any name with me, and they can testify that’s not a privilege at all 😀

I had a business trip last week. I wanted to add to the offline hours of mine. The plan was: get to the movies, airplane mode, and just enjoy those 2 hours (or four if you watch two movies in a row) of cinema art with no worries, no checking for any thing. Just you, the screen, and your thoughts. I couldn’t for various reasons, but only the thought of it made me happy. I am totally alone. And boy have I missed that!

It’s only when you travel alone could you realize how crowded it is. It’s only there where you realize how much chaos you’ve been into. How much you’ve swayed from your path. How much you hate your current life that you would prefer cutting all ropes relating you to it at any chance. It’s only my tier-1 people that keeps me tied to that life, and would cut it all together if not.

And yeah, It’s only when you’re alone that you remember how happy you are when you are alone…




The New Old

Cold outside. It’s not like it’s freezing, but that’s well above average for this country. Just the way I like it. I’m in a good mood…

Slid the screen to unlock. Social media? let’s surf. All those minds out there that I can talk to without talking to. Imagine being able to know all about someone without recognizing him/her if you meet! That’s social interaction the way I like it, and I’m in a good mood.

Let’s skip the small talk and talk about her. Do you know her? Oh poor you. She is what every poet ever tried to say, what every rock band ever rocked and rolled and screamed for. She is where those beautiful piano notes take you. The land of ever-sunset and ocean beach. The muse, hermosa.. Helen of Troy.. The face that launch’d a thousand ships, and burnt the topless towers of Ilium. You know her. Everybody know her. She’s all your happiness and all the lovely moments of your life. Your sweetest dreams, your light in the end of the tunnel. She is just the way I like it, and I’m in an awesome mood…

The sweetest are the beginnings, they say. But she’s different. Starting to know me, she was afraid of me, disgusted of me, and pretty much felt a trap in me. I never blamed her. That’s exactly how I felt when I knew me. Actually I never had that look of regret on her face the first time she saw me, but it’s alright. I never liked me too.. That’s one more common thing between us (Yay!). But let’s not talk about what happened between us, and focus on her. I can sing all poems about her all day everyday.. My favorite topic indeed, so let me pour myself a good cup of tea just the way I like it, and continue talking about her cuz today I’m in a good mood.

Who was she?
I could tell you everything about her.
Who she was, how we met.
The color of her eyes, and the shape of her nose.
I can see her, right in front of me.
She’s more real than you are.

Everyone tell her how beautiful her smile is. She knows it damn well. How do you tell a girl with a beautiful smile that her smile is beautiful? That’s a bit tricky. Let me guide you through this. You’re lucky cuz I’m an expert -I got to tell the one with the most beautiful smile how beautiful her smile was. You start small by knowing yourself: Are you telling her that to make her fall for you? to make her recognize you? Think again buddy. If so, you better off not telling her that because every.single.guy in the planet already said that to her. Are you telling her that for yourself -because you love saying nice things about people? Not good enough. Are you saying that because Oh God that’s a beautiful smile? Now we’re talking! Her smile actually didn’t give you the luxury of why and how to say what, words just come out of your mouth. But let me warn you: if she’s in a good mood, if she likes you (lucky you!), or if she is feeling generous that day, she’ll give you a wide smile when you tell her that, and oh boy does she heal a broken soul! Be ready to witness the greatness, the definition of beauty, the awesomeness that can’t be put to words, and the recalling of the feelings you had in all fabulous moments in your entire life. Her smile and you, that’s the entire universe right now. Just the way I like it, and I’m in a good mood today…

So why don’t we just play pretend
Like we’re not scared of what is coming next
Or scared of having nothing left

Look, don’t get me wrong
I know there is no tomorrow

A pretty smile isn’t only about the lips, it’s all about the eyes. You remember that old rainy summer night, with the after-rain fresh air, roaming around with your best friends? You remember how the sky cleared and you saw the stars, and you sat down and whispered glory to God the Almighty.. You remember the serenity and holiness? That’s her eyes.. Remember your last night in Beirut, when you ditched your boring travel mates and went to Hamra cafe, ordered a chocolate cake and a splendid cup of tea, and sat towards the street, looking to those group of young people and making up stories to all of them: the tough looking guy that has a crush on the red dressed that seem to like the shy one to the right that only looks to his MacBook and smiles when they say stupid things. He seems good looking but doesn’t dress well, so he is overlooked and all the attention is on the third one with glasses who says the meanest things about his family and I can swear knows nothing about “Why American business took over the world” thing that he’s been yapping all night about… You take out your phone and send her the “I miss you” text that your carrier refuses to let through, and go on with your chocolate cake and the waitress’s kind flirting.. The amount of peace you felt that night, my friend, is her eyes. The happiness you felt on your parents faces the day of your exam results, that’s her eyes. Utter happiness and complete beauty.. Only talking about them makes me remember that feeling that I totally missed. Another cup of tea? You know how I like it.. And I’m in a great mood today…

If this is my last night with you
Hold me like I’m more than just a friend
Give me a memory I can use
Take me by the hand while we do what lovers do
It matters how this ends
‘Cause what if I never love again?

Then life happens…

And by life I mean a lot of things. Crappy decisions, damaged souls, wrong countries, realizations of how I am not making her happy enough, financial issues, family issues,  screenshots, ordinary misters (screw you, mister!), statues, terrible first impressions to parents, and lies that cannot be recovered…

“Didn’t I tell you not to talk about what happened?”

Here comes my trust issues kicking..

I’d forgive anything but lies.

I’d forget anything but betrayals..

Except her eyes…

But when I see those eyes look to some other mister the way I saw them, I curse the day I saw them at all…

You think, twice and thrice.. But it boils down to only one thought that you can never shake off your mind..

I thought I was special!

I thought it was me whom you’re interested in, not the state of being with me..

But oh boy was I wrong…

Still in a good mood? No? that’s exactly how I like you from now on…

Hold me like I’m more than just a friend
Give me a memory I can use
Take me by the hand while we do what lovers do
It matters how this ends


The new reality is your life as it is now. You damaged that beyond repair and now all you can do is to be nostalgic. It’s all memories now. This is the new old…




About her: The Blood of My Blood!

To the lovely gathering, the tireless hours we spend talking, chit-chatting, and moving thoughts around…

To the smile of the little baby, the pure laugh of her, and the joy she spreads around the place..

To the brother whom is one of the few I love unconditionally without sharing a last name..

To mom’s smile when you’re around.

And to her.. My world is a better place with you..

You’re gonna be missed, a lot.. May ALLAH be always with you.. yi3dilha 3laik m7l ma tgabbili halool…

You are never alone, never far away from our hearts…

About her: The Drama Queen Child

I have been struggling with some childish behavior lately.. I really dunno how to deal with such behavior, and it seems every step I take, every word I say, and every word I don’t say would trigger some stranger behavior..

Ever been in similar situations?? Walk away and never look back.. Trust me, you’ll regret every single moment you struggle with this craziness.. And however hard you try, you will never be rewarded.. Don’t you ever fool yourself that you’ve got it under control. You are never in control of drama queens.. Drama queens cannot be tamed.. They are forever in drama, and they love it.. They will never let you take that from them however hard you try, however lovely and sweet world you make for them.. It’s either you live in their drama and fuel it, or you step outta their lives.. There’s no staying in their lives without getting ignited with drama flames..

Just walk away, pal…