Presentation on Map Reduce Merge

I gave a presentation yesterday (here is the powerpoint slides) and I am writing this blog to just jot down things I have learnt.:

1. I need to improve my grammar. The best thing to do is give a practice talk before your presentation.
2. It is very important to add all the references in your presentation.
3. If the paper is about a system, always add big picture describing the modules and their interactions.
4. If possible try to prepare for a demo to get people excited about the topic.
5. Always be sure of what you tell in the presentation. If you don't know something, say that "I don't know or I am not sure of it".
   

In the flow of talk, I said "As a database researcher" when I meant to say "As a database person". It seemed pompous at that time and somehow I preferred not to correct it at that time. But in retrospective as I think about it, I am a database researcher, may be not a good one and definitely not an established one. But still, I spent considerable time of my daily time doing research and I believe that qualifies me to be a "researcher".

SACHIN TENDULKAR - The Greatest (Video Collection)

What is your motivation ?
Its the passion. I care about playing for India. Its being a dream and I am absolutely honored to do that for last 20 years. - Sachin Tendulkar.

First some of his great innings:










Second, his interviews:




Third, Documentaries:














Fourth, Best Quotes on Sachin:


And lastly, his bowling:

How to set up Trac + SVN on Ubuntu

sudo apt-get install trac
sudo apt-get install libapache2-mod-python libapache2-mod-python-doc libapache2-svn

# Create a directory where all your Trac projects will live
sudo mkdir /var/lib/trac
sudo mkdir /var/lib/svn

# Usernames
sudo htpasswd -cm /etc/svnauth urname
sudo htpasswd -cm /etc/svnauth nextuserrname

# Add following to /etc/svnaccess
sudo vim /etc/svnaccess
##################From here#########################

[groups]
developers = yourusername, nextusername
[ / ]
@developers = rw
* = r

##################Till here#########################

# Add following to /etc/apache2/sites-available/default
sudo vim /etc/apache2/sites-available/default
##################From here#########################



DAV svn
SVNParentPath /var/lib/svn

AuthType Basic
AuthName "Subversion Repository"
AuthUserFile /etc/svnauth
Require valid-user

AuthzSVNAccessFile /etc/svnaccess




SetHandler mod_python
PythonHandler trac.web.modpython_frontend
PythonOption TracEnvParentDir /var/lib/svn
PythonOption TracUriRoot /trac

AuthType Basic
AuthName "Trac"
AuthUserFile /etc/svnauth
Require valid-user



##################Till here#########################


# Statically set the users in envvars
sudo vim /etc/apache2/envvars
export APACHE_RUN_USER=www-data
export APACHE_RUN_GROUP=www-data

# Reload the configuration
sudo /etc/init.d/apache2 reload


# Say ur project name is PhDProject
sudo svnadmin create /var/lib/svn/PhDProject

# Assign permissions
sudo chown www-data:www-data -R /var/lib/trac
sudo chown www-data:www-data -R /var/lib/svn

# Create Trac with SVN
sudo trac-admin /var/lib/trac/PhDProject/ initenv 'PhDProject' 'sqlite:db/trac.db' 'svn' '/var/lib/svn/PhDProject'

# Add urself to developers group
sudo trac-admin /var/lib/trac/PhDProject/ permission add urusername developers

# Optional: Some cool plugin
sudo easy_install http://trac-hacks.org/svn/accountmanagerplugin/trunk

# Re-assign permission
sudo chown www-data:www-data -R /var/lib/trac
sudo chown www-data:www-data -R /var/lib/svn

# Reload the configuration and restart
sudo /etc/init.d/apache2 reload
sudo /etc/init.d/apache2 restart

References:
http://trac.edgewall.org/wiki/TracUbuntuMultipleProjects
http://anantgarg.com/2009/03/25/subversion-trac-multiple-projects/

Meta-PhD

Why did I decide to do a PhD ?
When Chris asked me to do PhD, I was dead sure about one thing: I want to be in US for at least next 4-5 years of my life. So, I had two choices in front of me: either accept the SDE job offer from Microsoft SQL Server Data Mining team or accept Chris' PhD offer. Somewhere in back of my mind, I wanted to do a PhD after 3-4 years of my job. Also, Microsoft SQL Server Data Mining team was a dream job at that time. However, I decided to go for PhD for following reasons (ranked in the order of importance):
1. Why PhD?: For rest of my life, (except the daily chores and some personal issues), I want to THINK and DO certain work. I believe PhD sharpens (if not develops) the unbiased factual decision making process. It also helps you identify and enumerate some choices which you probably would not had thought of. (To be brutally honest, you don't need to do PhD for this, you just need to spent some time regularly acquiring this skills. PhD is one of the well-known an hopefully the simplest way to acquire them.)
2. Why Chris?:
a. I don't think I would find a better advisor than Chris (for "me"). There are two reasons for it. First, I am very comfortable communicating my ideas with him (both technical as well as non-technical). Second, Chris is an "awesome" researcher and I will learn a lot (and not just Database Research topics) from him.
3. Why PhD then and not afterwards?: If I jumped into job, I don't think I would ever return to PhD. (May be I think once I am settled down with good income, I would not have the motivation to start studying).

Though I have talked about skills that I would like to acquire in general, I would like to break it into my so-called mini-goals. The list below is a result of an hour long discussion with Ravi Jampani.

Mini-Goals for PhD (ranked in the order of importance):
1. Problem Identification Skills: Before I complete my PhD, this is the most important skill that I would like to acquire. For example, I should be able to identify potential problems in a given area after doing critical literature study.
2. Critical Literature Study or Building your base: This includes accumulation, simplification, contextual categorization (eg: Mindmap with Mnemonics) of the knowledge in my research field. This skill should also facilitate a way of frequently revisiting this knowledge base and also doing periodic "Lateral thinking" on it.
3. Multi-domain expertise: This means iterating skill 2 for multiple fields. It will also help for skill 1. The fields that I am interested in are Database, Data Mining & Machine Learning, Algorithms(with emphasis on Approximation and Randomization) and Mathematics (with emphasis on Statistics).
4. Indexing: Finding out who are good researchers in your field and always knowing what they are working on (kind of like fan-following).
5. Improve my communication skills and then writing skills. As of now, I ramble a lot and am almost never articulate in any discussion (though I believe that I am very persistent and continue the discussion atleast until I have communicated my ideas).
6. Collaboration of research with other people: This is least important skill as of now.

Useful advice from different researchers for new graduate students:
1. Chris:
- Indexing skill (i.e skill 4)
- Do not do exactly what the good researchers are doing.
2. Steven Weinberg:
- Start doing research (instead of writing this blog post ;) ), and pick up what you needed to know as you went along.
- Forgive yourself for wasting time. As you will never be sure which are the right problems to work on, most of the time that you spend at your desk will be wasted.
3. Richard Hamming:
- What are the most important problems in your field?, Are you working on one of them? and Why not?
- 'Knowledge and productivity are like compound interest.' Given two people of approximately the same ability and one person who works ten percent more than the other, the latter will more than twice outproduce the former. The more you know, the more you learn it is very much like compound interest.
- Luck favours the prepared mind (originally for Louis Pasteur). This essentially means that acquire skill 2 and 3 and you will soon find an opportunity to identify a 'good' potential problem and may be even solve it.
3. Others:
- There is not substitute for hard work. (Most important of all the advices)
- Be Persistent: "I have not failed. I've just found 10,000 ways that won't work" - Thomas A. Edison
- Explore as many areas as you can find. Read papers from different areas, not just your area.
4. Self:
- Be modest. The ego destroys the process of learning. Never disregard anyone who does not agree with you. Hear what he/she has to say and then try to think it through.
- Be committed to your problem. even if you think its not important. Finish it first and then start working on what you think is important.
- Think clearly, write clearly and talk clearly. Always talk with right amount of abstraction.

Mistakes you can make in your first job

1. Not taking time to study the fundamentals.
Eg: If you are expected to write C++ program, first learn C++. Take time to understand classes, inheritance, composition, etc. By understand, I mean you not only need to know the syntax, but also imagine the scenarios in which they are useful (an which they are not). If you don't know STL or smart pointers, believe me you would be re-inventing some of its code inefficiently :) and your code might be prone to bugs.

2. Google search before you ask your team lead or team members. This protects you from asking stupid questions or worse committing stupid mistakes.

3. Under-estimating the importance of "requirement gathering" and "design" phase aka Start coding immediately. A good programmer always takes time to understand the problem and create a big picture of the project in his/her head.

4. Trying too hard to present what you are not. It is extremely difficult to live a lie. Remember honesty should be coupled with extreme desire to improve yourself and also hard-work.

5. Blaming others to improve your chances of staying in the job. Believe me nobody likes a snitch. If you commit a mistake, be brave and accept the responsibility. Be proactive and ask the mentor for what you can do to improve your chances instead.

6. Disregarding "Murphy's law" and not clearing your expectations with team members.
If you think there is slightest possiblity of any team member to commit certain mistake or make certain wrong assumption, clarify it.

7. Delving into details without getting the big picture (kind of re-statement of point 3). But this is so common that I need to mention it again.

8. While using new software library, not differentiating the interface part and core problem part of the project.

9. Ignoring the coding standards of company. If you company does not have one, try to follow the coding standards given in the book C++ Coding Standards: 101 Rules, Guidelines, and Best Practices.

10. Reinventing the wheel in your design aka not knowing what are design patterns. If you don't know what design pattern mean read Head First Design Pattern. If you have already taken course in design patterns, then revising Gang of Four book won't hurt.

11. Not using a version control. Chances are that you would break the working code when you are trying to add new functionality. If you are using a version control, then remember to add messages like "Working version 1.4", etc. and learn how to revert back to certain version. If you are not allowed to use version control, make copies whenever you implement and test new functionality.

12. Not discussing your design with others aka school-children syndrome. Think of it this way rather than sharing your trade-secrets; by explaining or helping others, you improve your knowledge base.

13. Not respecting chain of command. Always give credits to your mentors, even if you think they have not helped you in it. It is even worse, if you complain against your mentors to their managers. Remember habit 4 (Think Win-win) of 7 habits of highly effective people.

Two books you must read and re-read before joining the job,
1. 7 habits of highly successful people by Stephen Covey. (Summary)
2. Getting things done by David Allen. (Summary)