Memories of Masdar: 2011 – 2013


I wrote a memories of Ife piece after completing my bachelors and it would be just as well to write another post about my time at Masdar. Masdar is a great school, has an international student community, extremely nice people, an awesome nearly unbeatable welfare package and a great beautiful environment.

I arrived at MASDAR in the fall of 2011 as a young naïve man with some basic development experience. Something unexpected happened just before I resumed; that event, which left an unpleasant indelible mark, propelled me to give my all and Alhamdulilah I emerged as a better person in the end.

The first few months were somewhat lonely – it was an unfamiliar emotion as I was used to being alone and far away from family all my life. The feelings of loneliness soon vanished when lectures went into full gear. I had thought everything would be super easy however I soon realized I had grossly underestimated the coursework involved as I struggled to understand some of the material and complete projects.

My interest in socio-mobile computing got me involved in the joint SCAIlab-MIT social fitness project. It involved extending a Pedometer app with social networking support, data collection, accuracy tweaking and influence mechanism design. Although the project was not completed, I got to learn more Android programming as well as Git.

I spent my first break solving Project Euler puzzles (a way to simultaneously hone my new Python skills), polishing my résumé and hunting for internships ( I applied to about 10+ software firms).

During my second semester, I got to learn more about Algorithms, distributed systems (hadoop, Skype, networking etc) and social computing. In my spare time, I learnt vim (and I am glad I did) using the excellent byte of vim book. Alhamdulilah, I spent the summer as a SDET intern with the Microsoft TFS Agile team.

In my last year, I finally took the dreaded software engineering ( got to learn about C, valgrind, make etc.) and sustainability courses. I also had to work extremely hard and at an insane pace to make up for the time I lost duing my internship. My first attempt didn’t turn out well so I had to start afresh.

Gains at MASDAR

Sometimes the loneliness and extreme levels of hard work got to me – the emotional drain and stress were just too much at times. I had to force myself to persevere ( giving up was easy but too costly) and I repeatedly prayed to Allaah to bless my sacrifices and efforts. Looking back, I am glad I got to improve on some of my weaknesses.

I improved my time organization, goal setting and progress tracking skills. I learnt to focus ( after getting burnt out a gazillion times) and also realized that I had a limit and couldn’t do everything. It was good to learn that being consistent was much better than alternating between periods of high and low productivity.

It was also at MASDAR that I started making conscious efforts to improving the quality of blog posts. The goals were quite clear: I wanted to create a knowledge repository for myself (something I can always refer to when I am in doubt), improve my writing and inform people. Now, when I write, I ask the ‘so what?’ question :).

My academic training equipped me with lots of computer science / software engineering knowledge. I got to conquer my fear of public speaking, honed my leadership skills and teamwork contribution. Oh, and I picked up the habit of continuously enrolling in online courses. Learning never ends…

I made a couple of true friends: the camaraderie, the teasing, the play, the talks and the sharing actually enabled me to come out of my shell. I met people who were extremely nice, friendly and wonderful. I couldn’t have asked for better companions and I do hope we’ll keep in touch for life.

It wasn’t all fun at MASDAR though – there were a couple of unpleasant experiences as expected. However I am glad because I learnt from such experiences too. In the end, I think I became more mature and discovered the meaning of self-worth.

I have left MASDAR now (I actually felt bad leaving MASDAR because I had become so attached) however I hope the lessons I learnt there would stay with me, that I have friends for life and have become a better person for it.

Advertisements

MOOC Review : Machine Learning


So I completed the Machine learning course on Coursera recently; it’s my umpteenth MOOC Course and I actually do not know how many I have taken or want to take. Despite this,  I have only gotten three certificates so far; all my attempts at integrating Coursera courses into my schedule have not been successful. I have had to settle for auditing courses at my pace: at least I get to fulfill my goal insha Allaah: learning and understanding.

I took a couple of data mining courses during my master’s degree and have some experience with a variety of tools and programming languages (Octave, Weka). As grad students, we had to write our own classifiers and performance evaluators (kNN, Naive Bayes, ROC curves). Using Weka or similar tools/libraries would have saved us much trouble however our lecturer did not agree. I believe his approach was great as we were ‘forced’ to learn and got to really understand what the mathematics was all about.

Back to the Machine learning course, I think Professor Andrew Ng (Stanford) is simply awesome at what he does – everything is explained in really simple terms. He is also the guy behind the awesome autonomous helicopter video, haven’t seen it? Here is the link, go watch it!

It was refreshing to use Octave again, its elegant approach to numerical computation is mind-blowing – it makes it possible to get a lot done in a few lines. It also seems to handle the floating-point issue well (0.1 + 0.2 = 0.3). These features are impressive when compared to other languages; however every language has its strengths, weaknesses and application domains.

Alhamdulilaah I learned a great deal and refreshed my background in a variety of machine learning techniques and applications. It was nice to relearn the concepts of gradient and stochastic descent, clustering, overfitting and underfitting (bias and variance). There were a couple of new topics such as principal component analysis (beautifully explained), logistic regression, anomaly detection, recommender systems and support vector machines.

I struggled with the neural networks section and at times found the programming exercises quite challenging. It was a surprise (albeit a humbling one) to realize that I didn’t know as much as I thought I knew. The suggestions and advice on data mining are invaluable, he gave suggestions on setting up a processing pipeline, ceiling analysis, learning curves, error analysis and regularization and creating artificial data and dealing with massive datasets.

Overall I think it is a great course and Alhamdulilah I am glad to have taken it. The next challenge is to find some way to use these techniques – there is no better way to fully understand than to practice, not so?

Interested? Head over to Coursera and sign up. It is currently not running however you can either view the archives at your own pace or wait for a new run. I have also written on some other MOOCs too: Databases and Social Networks. Do check out this cool infographic too.

 

Job Hunting 101 for Students


Yes, I am still continuing with my posts on Grad School, research and all-of-that. The big question for most of us towards the end of school is ‘Whats next?’, a lot of us want to get jobs immediately afterwards but realize too late that it’s much more difficult than we thought. Here are a couple of tips.

1. Get the skills you need while you’re still in school

Learn as much as possible: leadership, teamwork, computer literacy, and relevant industry skills. Once you acquire these skills, relentlessly hone them till you stand out. There are always ways to excel: software packages to learn, online courses to take and certifications to earn.

Ideally the learning process should start years before you graduate. However, if no one ever told you this before, then start now! No, graduation is not so far away; it’s much closer than you think.

2. Network, Seek mentors

The easiest way to get a job (in my opinion) is through referrals – Mr. A knows Mr. B and refers him. Be bold and confident, create a really polite email asking to be a protegé and tell your potential mentors about your aspirations and career goals. You’ll be surprised by the amount of help mentors can offer through their network, experience and wisdom.

I have gotten referrals through my friends and mentors, so I know it works, try it out too.

3. Prepare a strong CV and cover letter

Your CV is probably the first measure of your skills that your potential employer will see, so make sure you dedicate lots of time to it. Sanitize your online profiles, review your LinkedIn profile (create one if you don’t have any) and create accounts on job portals.

Ideally your CV should be one page long; if you have more pages you probably should cut some insignificant detail. Jump start your rewrite with these templates here. Create a cover letter too, this might make the difference.

4. Apply to Jobs, internships, everything

Apply for every promising opportunity: the more applications you send out the higher your chances of success. I am not advocating applying to jobs requiring 90 years of experience in alien technology for Saturn – you’ll end up wasting your time and that of the recruiters.

You should have an application strategy, create a list of organizations you like and apply to them. If you don’t meet their requirements, go to step 1 above. Since you have nothing to lose (and a lot to gain), you should apply to other firms too; if you don’t get an offer, at least you’ll have learnt more about the application process.

Important! It’s always better to reach out to the recruiters and/or people in charge of recruiting directly. I’ll give you two stories; someone I know applied for a job and he noticed that the HR manager at that firm looked up his LinkedIn profile. However, when he got no response, he found the email of the HR manager through Google, forwarded a strong cover letter + CV and got an interview request in about 48 hours! Amazing, huh?

I have also met a recruiter who told me they sometimes have to go through 1000 CVs/week, eek! What are your chances out of 1000?

5. Go to as many interviews as possible

You’ll become more confident and better at ‘selling’ yourself and your skills. Another added benefit of this is that you’ll learn how to answer non-technical questions (questions like ‘how much do you want?’, ‘why do you want to work with us’).

It also gives you an external perspective of your skill level and allows you to find the gaps in your knowledge (this might be a humbling experience if you’ve overestimated your skills). If you get the offer and decide to turn it down; do it nicely and make sure you keep in touch. You can always reach out to them or refer people to them – it’s always good to network.

6. Improve, improve, improve

After each interview, schedule some time to review your performance and take note of your weaknesses. Make sure you never make the same mistake twice if you can help it.

Also review the questions, did you answer the tricky questions right? Did you impress the interviewers? Go through the experience again and make notes of what not to do at your next interview.

7. Never Give up

If you don’t get the job, send them a nice email thanking them for their time. Politely ask about why you weren’t chosen and use this information to grow.

Never give up! Also remember to pray to Allaah for help as this is very essential.

Thesis Stories Ep 3: Research is Hard!


Alhamdulilah I completed my thesis about three weeks ago; if you’re interested, you can check out my thesis and presentation. Looking back at the two years I spent at MASDAR, I have a couple of thoughts: Alhamdulilah I learnt a lot, met a couple of wonderful people and matured significantly. There were a couple of not-so-pleasant experiences too but I believe I emerged stronger ultimately.

So I switched to the complexity analysis of road networks after my stack overflow adventure ended unsuccessfully. It was a fresh start but I had no alternative since I wanted to graduate. In the end, I defended all my hard work in about 75 mins – imagine! Nearly 6 months of work translating into just 75 mins!!

Research is difficult! As difficult as any other endeavor; I think most researchers don’t know how their efforts will turn out (as most start-ups do at the beginning too). There is usually some hunch about a model, some experiments and then eventually they have to figure out what the ‘right’ result is. Also, ‘big data’ appears to be fun and cool but it requires unbelievable and prodigious amounts of grunt work.

I built JIZNA, a custom Python framework for complexity analysis. JIZNA can parse openstreetmaps XML dumps of cities (the parser was an open-source utility I found and modified), create dual graphs of these networks, merge discrete roads, exclude outliers and calculate the desired metrics. These metrics were used to predict how difficult it would be to search the city. The JIZNA platform is available here.

The Cool Stuff

I think I wrote much better code: the framework was modular, nicely designed and flexible; I was able to write some really cool algorithms for the complex computations and I learnt how to use Sphinx, the Python documentation tool. Sphinx, in my opinion, is a lovely tool once you grasp its basics.

The Not-so-Cool

I got a couple of interesting results however I think they were not so spectacular. I guess further work would reveal some new insights.

I had to throw away some of my code (a complete simulation framework had to be discarded when the approach changed) and my writing (again! This is the umpteenth time I’d be chopping off my writing).

So what did I learn? Lots more Python, algorithms, software design, documentation, writing, latex, vim and some maths (mostly matrix algebra). However, more importantly, I came to appreciate the value of grit, determination and perseverance while working towards goals. Don’t ever give up, even if all appears to be lost.

Next plans? I don’t quite know fully yet; one thing for sure: research is hard! :)

Did you like this post? Check out my other posts on Grad School.

Exploring my Facebook Network


So I finally completed the Social Network Analysis course by Lada Adamic today and I learnt quite a few things Alhamdulilah. Some of the MOOCs are really good but there are so many options that I sometimes get overwhelmed.

One of the cool things about the course is that students can get exports of their entire Facebook network (export tool available at this link). Of course, I got mine – I have always wanted to and I am fascinated by social computing. Analyzing my network in Gephi led to quite a few unexpected discoveries. Now, don’t get any ideas, I ain’t talking about those here ;). I’ll only talk about the ‘safe’ stuff which is already obvious public anyway.

So, I have about 1051 friends who share 23915 edges who can be segregated into four communities – these communities kind of map the friends I have made in different places and times. The average path length is about 3 while the network diameter is 8 (quite close to the much-touted 6 from the classical work of Milgram). The average degree is about 45, so my friends have on average about 45 friends? I dunno…

I generated a visualization of the network using the Fruchterman-Reingold algorithm (it provided the ‘nicest’ picture); the colours indicate the degree (i.e. number of connections to other friends) while the sizes of the nodes shows the betweenness centrality. This betweenness centrality can be explained as a measure of how important a friend is in bridging/connecting separate parties or groups of people.

My Facebook Network
My Facebook Network

The cluster on the top left shows the friends from my high school (Federal Government College, Kano). The person with the largest betweenness value (the big yellow circle) is the only person I spent about 10 years of my life with in the same institutions! (High School and University) It is quite unsurprising the person connects these two communities together.

On the top right is the community of friends I met at the MASDAR institute while the lower right (that appears to be two really close communities) are the friends I made in university – Obafemi Awolowo University. The left half is something of a sub community – it consists majorly of my colleagues in the Computer Science and Engineering Department while nearly every other person falls in the right half. Lastly, my family appears on the network too but I ain’t mentioning :).

Twitter next? Maybe.. Maybe not.

Applying to Grad School


Masdar Institute
Masdar Institute (Photo credit: Michael Keith Photography)

So you decide to apply to grad school – maybe you think research is the best thing to do, need more time to make up your mind or see it as the way to reach greener pastures. Oh, maybe your dream is to invent cars that run on water…

This post is for you then! There are many advantages (and disadvantages too) of grad school: you meet awesome people, broaden your scope and improve your analytic skills. However, the process of applying to grad school is long, demanding and at times frustrating.

Here are a couple of tips on the process: most of the suggestions are based on my  experience and things I have learnt about the workings of the grad school system.

FIRST STEPS

1. Boost your credentials

There is fierce competition for positions in top schools so you need to stand out. Diligently acquire all the skills that will give you an edge: research experience, coding skills, work experience and writing skills etc.  Make the most of your time: your holidays (both official and unofficial à la ASUU style) and cut frivolous activities.

This is also the time to build your academic credentials; make sure you get high grades and understand course content properly. Cultivating good relationships with lecturers is also essential as they’re going to provide recommendations for you.

Now don’t go seeking recommendations from the lecturer who taught 10000 students like you an elective course. It’ll be best to go for people who taught you and know you well. For example, your research supervisor, your course advisor or your favourite lecturer. You’ll most probably never see the recommendations they write, so make sure you are in their good books or you might be shooting yourself in the foot…

One last thing, if you need to write exams, then it is also essential to start looking for funds – the exams are quite expensive and if you have to save towards them, then the earlier the better.

Summary: Work on your CGPA, get funds and cultivate great relationships.

2. Identify your research interests

Identifying a research field of interest is quite difficult because you have to know what you love doing. This takes some time, so do extensive soul-searching and find out what stimulates your curiosity. Identify your passions: do you spend endless hours trying to solve a particular problem? Do you say to yourself: ‘I’ll do just one more fix and that’s all’ and yet spend 6 extra hours on a particular problem? If yes, then that field is probably a good research area for you.

A couple of tips include identifying what courses you enjoyed in undergrad, reviewing the various research areas in your field and asking older people for help.

Summary: Discuss with mentors, carry out independent findings and find out what areas of your field piqué your interest.

APPLICATION STAGE

3. Research your proposed Institutions

Create a ranking of the institutions you’ll like to study at. Know the merits and demerits of each institution, ongoing research and collaborations, career prospects after graduation, student life, study facilities and learning environment.

Reach out to the lecturers at these institutions, discuss your research interests and study goals with them. Some schools actually encourage this; however if you get no responses, don’t give up hope but continue striving.

Make sure your interests match ongoing research in your target institutions. Looking for a computer engineering position in a machine learning research lab is just as efficient as fetching water with a basket.

4. Take the exams

Most graduate institutions require some exams – TOEFL, GRE, GMAT or IELTS. Make sure you prepare for them thoroughly, it’s always best to ace exams at the first go so leave no stone unturned!

One of my friends actually made a good suggestion – he said he’ll only consider himself ready if he could solve all the practice GRE quantitative questions in 30 mins (normal GRE timing is 45 mins). Consider challenging yourself similarly too.

5. Prepare your statement

This is probably the most difficult part of the graduate school application: it involves a lot of time, thinking, reviews and feedback.

Read the statement guidelines for your school and follow them to the letter. Some schools ask for specific information, make sure you provide all these. Talk about your research plans and aspirations, why you are a good fit and show that you’ve done your homework.

Get many reviewers to go through your essay, they’ll see things that you don’t see and should offer great feedback. It’ll be great to get in touch with people studying at your prospective school and discuss with them too, maybe they’ll even help review your SOP; after all, they are students in that school and must have written a statement too.

Summary: Read it, find flaws, rewrite it and when you think it’s perfect, repeat the process again. When you think you can’t flaw it in any way (and have probably annoyed all your reviewers with your incessant appeals), then and only then you can start to think of  submitting it.

AFTER APPLYING

6. Pray to Allaah

I’m a Muslim and I think this is the most important aspect of it – I prayed a lot and Alhamdulilah I got admitted to MASDAR. It is the right thing to do, it strengthens you and helps you to cope with disappointments (and yes they do happen). So do your best, pray to Allaah and hope in Him.

I quite forgot to mention doing istikharah; this is a necessary step when you make decisions – choosing to go to grad school, selecting institutions etc.

And that’s all, share with people who are applying to grad school and do feel free to contact me: I’ll be willing to help insha Allaah.

The language series: C


I finally took the compulsory software engineering course notorious for its very difficult course project – writing a bitcoin client in C. Alhamdulilah, we successfully completed the project: about 18k lines of code, automated builds/documentation/tests and lots of other stuff. I figure we rank around 7 or 8 on the Joel 12-point scale even though some don’t apply to our project. :D Big UPs to the team!

I decided to do a review of all the languages I have used or been forced to use while taking the course; the story behind learning these languages, their strengths and weaknesses; quirks, advice for beginners and some wisecracks too :).

C is first on the list. Here goes!

How I learnt C

I somewhat got forced to relearn this language this year but my first attempt at C was self-study in 2007 or 2008 as an undergrad. Despite my dreams of building the most AWESOME program ever, my C adventure ended abruptly after I read about 3 to 5 chapters of a C book. I was discouraged by apocryphal reports which insinuated that C was no longer relevant; so I left C for C++ and then Java. That story is here.

Well, this year I had no choice but to learn it. Well, there was another choice: getting a poor grade in the software engineering course.

Likes

  • C packs a powerful punch, who doesn’t like power and speed?
  • It has a concise grammar and you can learn the language fast.
  • Purity: its simplicity forces you to think.
  • I think function pointers are kind of cool too.
  • Forces you to learn how low-level computer stuff like stacks, heaps and memory allocation work.

Dislikes

  • It doesn’t support as much abstraction as I want.
  • Bah… why do I have to call free() all the time? Can’t the language help me with this? I already know and agree am spoilt but why make programming harder?
  • No hashtables? No string support? Beats me… every language seems to have these.
  • There is some redundancy in the methods available in the C library; e.g. strtol and atol; seems PHP got a predecessor in C.
  • Pointer tangles; what does this point to or mean? ***a.
  • Uninitialized values can hold all sorts of values; woe betide you if you make the mistake of using them straightaway; C won’t raise any errors.

Writing code in C

It’s one of two things: you’ll either learn code purity and write pretty nifty code or massacre lots of innocent computer bits à la segmentation faults, memory overwriting and stack overflows…

I think everyone starts out in the latter group and moves to the former :).

Recommended For Beginners?

C is pure and has a small grammar (makes it easy to learn) but a bit challenging for a beginner to start with. I think Python or scheme will be easier.
You’ll probably find OOP difficult to grasp if C is your first language however you’ll find other languages really easy.

C Quirks

7[a] == a[7] if a is an array; it was even on my exam! :P

while(*s++ = *t++) copy a string t to a string s.

Rating

6/10

Pretty powerful, compact and small although lacks a lot of expected features and development is sometimes painful. There are a couple of libraries that you can use though.

I hear C++ is more challenging… Do the ++  signs signify difficulty? :)

Read my reviews of PythonJavaPHP and JavaScript too.

Thesis Stories Episode 2 : Adventures in Ginormous Data


What could be worse than trying to understand ginormous data? Not finding what you’re looking for in it! The original plan was to use stackoverflow data as a proxy for my data mining experiments and after battling the databeast into submission (with the aid of ‘weapons of mass data analysis‘ like Python, SQLServer and sqlite3); it pulled another fast one on me.

I started exploring the murky depths of the subdued dataset and plotted the data distributions (they were mostly heavy-tailed as expected although one was unexpectedly normally distributed). Plotting the distributions was a task in itself – I had to learn about the awesome matplotlib, numpy and scipy (installing them was ‘wahala‘) – and  then the plots were so skewed that I had to spend hours on end finetuning and tweaking before they finally agreed to appear properly on my plots.

Plotting data distributions was child’s play compared with the next set of challenges : getting the candidate posts. Having defined the five features I was interested in; I set out with gusto to find the ‘offending’ entries. I got a surprising outcome – the first three characteristics (deleted answers, converted answers and flagged answers) didn’t exist in the dump; I got the same outcome when I ran my queries on the online data explorer. Finally, I asked on stackoverflow meta and it was confirmed.

You bet I was disappointed right? Sure! I’d spent hours on end writing convoluted SQL queries (subselects, joins, aggregations and what-have-you) and wrapping my head around the data. Heck! Some queries took me about an hour to write, run, verify and tune. Do you know what programming in a declarative non-Turing complete language with lots of constraints (geek-speak for SQL) feels like? It feels like fighting Mike Tyson with one hand tied behind your back. :P (Alhamdulilah,  I took the MOOC DB course).

When man fall down, no be the end of hin life… (Nigerian proverb; language: pidgin English)

So I listed out my alternatives : getting a new dataset, using the same dataset or starting on a new topic (most disliked obviously ;) ). My search for a new dataset was not fruitful – I find other datasets ill-suited to my research and going through the potentially painful process of transforming them does not appeal to me. I went back to my dataset and extracted three other features but the nagging feeling in my mind is that I might have to fall back to the third option.

So do I concede defeat? Nah, nat at all – am a Nigerian remember? We never say die; we’re way too optimistic for our good even :).

Lessons Learnt

  • Never write a lot until you’re really really sure that you’re gonna get something.
  • How to extract information from papers and critique them, know what they are all about.
  • How to read and write continuously for a long period – how do I do it? Pomodoro of course!

Next Steps

I might go back to the SO data; or start all over again but I just pray it turns out all fine – I now have about 4 months left.

Ohh; I forgot to talk about the platform – that’s just been about as good as the experiments.

I am using EmberJS, a MVC framework and it’s being really challenging as I am new to it. I’ve had to fix issues with performance and page load times, integration on Amazon EC2; and all sorts. It’s been so difficult that I’ve started entertaining un-Nigerian thoughts of giving up on EmberJS – plain old vanilla JavaScript is much simpler.

Ok. Magana Yakare (“The discussion is over”, language: Hausa).

Have a great weekend – I just wanted to go at it the Naija style today and not write the same old normal way I do. I hope you enjoyed it; if you did – drop a nice comment or share some of your grad life experience.

N.B: If you’re a grad student having issues with your thesis; don’t worry be happy :D