Amr Awadallah Blog

Cloudera just launched a Hadoop and Big Data blog, you can access it at the following URL:

http://www.cloudera.com/blog

The purpose of the blog will be to post tips and tricks on how to tune Hadoop system performance and how to work with big data in general.

Jeff Hammerbacher just posted his latest “Open Source Data Management Software” presentation on the blog. In the presentation he covers Thrift, Scribe, Hive, and Cassandra, you should check it out.

Cloudera also launched Hadoop Support, Hadoop Resources and Hadoop Training sections, we are still in the process of beefing them up.

Cheers,

— amr

The new company I am starting is called Cloudera, it will be offering support for Hadoop, think RedHat for Hadoop, but that is just the beginning.

My co-founders are a bunch of really interesting folks:

• Mike Olson was CEO of Sleepycat, which commercialized Berkeley DB, the open source embedded database engine. Mike spent two years at Oracle after they acquired SleepyCat in 2006. You can read this article about Mike from InformationWeek to find out more.
• Christophe Bisciglia just left Google, he created and managed their Academic Cloud Computing Initiative, which is a public hadoop cluster for academia/research. You can read this article about Christophe from BusinessWeek, his photo was featured on the cover of that issue.
• Jeff Hammerbacher just left Facebook, where he started and managed the Data Team. The team made a lot of contributions to Hadoop under his stewardship, most important of which is Hive (a SQL structured data layer on top of Hadoop). You can read Jeff’s Mainly Data Blog here, and you can also watch a video of him giving a talk about Facebook at Yahoo.

In addition to these folks, we also started building out our founding engineering team: Aaron Kimball joined us as a full-time software developer, and so did Tom White. Alex Loddengaard joined us as an intern, and Matei Zaharia joined us as a part-time consultant. We are hiring a couple more engineers, but that’s it for now. The skills required are a solid understanding of large-scale distributed systems and strong C++/Java programming experience, we also have one position for an outstanding front-end developer, specifically Javascript/AJAX.

We are in the process of wrapping up our funding (we don’t need any more cash at this point), and will soon announce our investors and technical advisors.

Cheers,

— amr

I spent 8.5 years at Yahoo, and grew up the corporate ladder from a senior engineer to VP engineering. I had many wonderful bosses at yahoo (Qi Lu, Tim Cadogan, Andrew Braccia, and Venkat Panchapakesan), whom helped mentor and grow me. I also attended many career development training classes (an excellent benefit at Yahoo!) and read books like “The Seven Habits Of Highly Effective People” and “First, Break All The Rules”.

Below I list some of the key lessons I learned and tried to practice while at Yahoo:

• Lead By Example: Don’t ask your troops to do something that you aren’t willing to do, if you want them to work hard, then you gotta work even harder than them.
• Keep Your Hands Dirty: Don’t be a pure people manager, roll up your sleeves and jump in to help the troops. This helps you earn the respect of your team, but also allows you to call BS when you see it :).
• Eat Your Own Dog Food: To really work on a product and make it better, you need to use that product at least once. I saw many people working on Yahoo products that never used those products, that is wrong.
• Think Solutions, Not Problems: I saw many people complaining about problems and getting completely hung up. As soon as you encounter an obstacle you should move on to think what is way around it, how to resolve it, as opposed to getting locked up on fact that the problem is there.
• Never Be Shy to Ask And Learn: It is ok to admit that you don’t know something and ask your reports to educate you about it. I sometimes saw people with egos that forced them into pretending they knew when they didn’t, that made them look even more foolish.
• Be Honest To Yourself: You gotta have your own opinions about things, even if they contradict what your boss or company wants to hear. You shouldn’t be rude, or just contradict for the sake of it 🙂 just be honest and stand by your principles and passion.
• We Are Statistically Insignificant: Reaching product decisions by using proper analytics of user traffic is much better than going with your instinct because you’re statistically insignificant in the grand scheme of things, no matter how smart you are. I’m not saying never use your instincts, you should, but if the numbers say otherwise then be comfortable changing your opinion.
• Be Self Motivated, Go The Extra Mile: If you want to grow career wise, you have to be self motivated, you have to come up with ideas on how to improve things without being asked to do so. You also have to go the extra mile when needed, work extra without being asked to, work on something that other neglected even if it was not your assignment to do it, etc.
• Don’t Just Go With The Flow: It is very important you know where you want to go, and what you want to be working on, don’t just sit there and wait for things to happen. Similarly, know what projects and what things you want to accomplish in a given quarter and have a score sheet to measure your self against.
• Multitask and Prioritize: It is important to know how to work on more than one thing at the same time, keep a todo list and highlight the important things that need to be done first. Nothing is more satisfying than scratching done items off a sheet of paper.
• Listen, Reflect, Think, then Talk: When somebody tells you something that you don’t necessarily agree with, don’t be impulsive and jump back directly with a counter-response. You should digest what they said, repeat it back at them in your own words to make sure you understood what they really meant (this also helps them feel you heard them), think about your response, then finally express your opinion. This was one of the hardest lessons for me to learn, took a lot of mentoring in the early years, even at the end still :).
• Take Care Of Your Peeps: A good manager should know all of his reports by name and take good care of them, he should remove all obstacles in front of them so they can excel. He should realize their strengths then put them in roles that leverage that, conversely, he should realize their weaknesses then help them improve or move them out of roles that depend on those skills. That said, if after all that they do not deliver, then they don’t deserve to be one of your peeps :).
• Rally The Troops: It is very important that your group has a clear mission that every body is excited about and is building towards, even if they are only contributing little towards that goal. Clear goals like that draw people together, motivate them, and give them a sense of pride (e.g. “The Moon Landing Mission” 🙂 ).

Cheers,

— amr