Regex hammer time

When you have a hammer, you see everything as nails.
A very true message from Dive into Python 3,

Regular expressions are extremely powerful, but they are not the correct solution for every problem. You should learn enough about them to know when they are appropriate, when they will solve your problems, and when they will cause more problems than they solve.

But it is easy to fall into the trap of using regex for everything.
I need to find the last string from a path, where

path = "d:/some/directory-with dash-and space"

So here is how I try to get the last string in a path, with regex

path.match(/\w*$/)

trying to capture any word characters at the end of the string with \w*$, but only captured the last word “space”, not good.

path.match(/(\s|\w)*$/)[0]

to match either whitespace (\s) or word characters (\w) at the end of the string, the result is “and space”, dashed is not captured, not good.

path.match(/[^\/]*$/)

[^\/] to take in everything except “/”, result is “directory-with dash-and space”, it works!

Only later I found out there is a really simple way to do this

File.basename(path)

will actually give you “directory-with dash-and space”. Plus, it will ignore ending slash if there is any.
Duh!

How this started

I think it was late 2005, my wife used to work really late during those days, 8, 9 even 10pm at night, I have experienced a few even longer wait until 1 or 2 am, literally the next day. Sometimes I use this spare time to finish up my works, but I was still quite junior then, there aren’t much work that I can’t finish in normal working hour.

As a system administrator for the department, one of the tasks that I dratted most is to update a long list of computer inventory. Physically go to a computer, check on serial number, OS installed, computer name, application installed, etc etc. If the user was not around and the screen locked, then I am out of luck and have to find another time for a check. So I punched in words like “hack”,”hack remotely to check hardware specification” into google, hope it will yield some useful dos-prompt command that let me remotely checking out the hardware specification. Instead, I came across Eric Steven Raymond epic article “How to become a Hacker”. One thing lead to another, soon I find myself reading every Paul Graham’s essay feverishly, Great Hackers, Hackers and Painter, The Word Hacker, How to do what you love, Python paradox , the list go on.

Reading “How to become a Hacker” and most of the PG’s essays is mind blowing, like open up a whole new world, at a time when hacker normally carried a negative connotation in the mainstream; suddenly I was shown the true meaning of a hacker.

“Hackers solve problems and build things, and they believe in freedom and voluntary mutual help”
“hacker” connotes mastery in the most literal sense: someone who can make a computer do what he wants.
You do something so clever that you somehow beat the system, that’s called a hack

Wow, that sounds cool. First thing in the basic hacking skills listed in “How to be a Hacker” is to learn how to program. As a Computer Science Graduate, sadly all I have learned during my study was some asp for coursework, html and really tiny bit of java. I still remember when we learn html, I put up a page with <marquee> tag showing off a blinking words moving across screen then proudly showoff to my classmate. Serious.

So I have to pick up a language and brush up my programming skills again. Since both Eric Steven Raymond and Paul Graham touted Python as one of the language to go for, that was what I did. Learning python with A Byte of Python, Dive into Python and The Python Tutorial in IDLE make those waiting hours gone past so much easier.

Feeling excited with the new found world, I continue to play with django, ruby on rails, Processing, Fedora Core, Ubuntu, wordpress, radiantcms, to name a few. I was fascinated with the newer shinny things churning out from the web. It is always the same pattern, I started out really eager, then either something happen in real life, change job, married, have kid, etc, or some newer toys show up, then I shift my attention. I spent a lot of time learning, but with no end result to show. It is a cycle that dragged for a few years and brings me to nowhere.

Until a casual chat with my childhood friend some time last year, we rant about the meaningless job that we are doing, how I really long for works that require my creativity, be in coding or design. That must be the hundredth time we went thru the same rants. But this time, my friend said, “You are in your thirties, your life is almost fixed. You are done. You can forget about your bloody hacker’s dream and just think about your son’s future instead.”

The thinking that my life is done, and I can’t do anything about my future, that I am gonna be the same mindless working robot in corporate working life until I retire suddenly strike me.

Is that what I really want? Admit defeat without even try hard enough? Then what is the lesson I am teaching my son here? How am I supposed to tell him that you can be all that you want to be as long as you work hard enough? What kind of example I am setting for him?

Being in my thirties is exactly the reason I should really buck up and deliver; I have no time to waste. So to force myself to deliver, I looks for excuse to rollout Ruby on Rails application to the department, eventually I find two, and commit a date that I don’t really feel comfortable with, then push myself to build it, sometime even drop my job on hand totally to make sure I delivered the features and meet the rollout date. It took me about 6 weeks working on and off, eventually I delivered both applications to about 50 users. Railscast, stackoverflow, AWDWR 4th Edition were my most frequently used references throughout the development. You really learn a lot more building up real things compare to just reading tutorial or books and creating bogus websites.

I enjoy every minutes of it, coding in rails, solving problem, changing layout, adding in a bit of jQuery. Everything. But then every minute that I have to perform my normal task become so much more unbearable. I really need to do more real works in rails.

So I search almost every job sites. There are not a lot of rails opening in my area (Penang, Malaysia), and even if there is, it might be difficult for me to find a permanent job in ruby on rails that could pay my current salary. After all I am experienced (7 years) in my current field as a system admin, but in term of rolling out rails apps, I have only the experience of developed two rather small rails apps for 50 users to show, as for rails apps that facing the real world …none.

So when I saw a remote rails developer job opening, I just have to give it a shot. There is a test project as a prerequisite for the job. I thought it could be completed over a weekend, but it took a little bit longer than I wished, end up I decided to take a day off from my work to complete it. Don’t want to regret for not trying hard enough. I learn something even with this small little test project, deploying to engineyard, brush up my regex, pick up some jquery.

After completing the test project, although the employer commending on the effort I put on, he couldn’t take me as he is looking for a permanent fulltime remote employee, not a part-timer. And he doesn’t have the budget to match my current fulltime salary.

I tell him frankly I might not have a lot of real world rails experience, but I really like web development and would love to have the opportunity to work on the project. So we discussed about the possibility of a part time arrangement on hourly rate. We come to a rate that is much lower than if I were to divide my fulltime salary by the hour. But heck, I am starting out and just glad to get my first freelance rails gig.

So far I have been in it for about 2 months, I work on the project by waking up early at 4 or 5 a.m. before going to work on weekday, then a total of 8-10 hours on weekend. Learn a lot of things like version controlling in git, deploying to heroku, which I wouldn’t have pick it up as fast if I am not developing a real apps. Really love every minute of it, and ya, that make going back to my fulltime work even more difficult, just wish to spend more time on the project.

It is tough for me to make a sudden switch of career with all the commitment I have, as I need a stable income. But I intend to keep pushing and not submit to the ideas that my life is done at thirties. Now I am just looking for ways to add experiences and repertoire to my web development skills, while pilling up my github profile. I believe the best way to do that is by doing real work. Hopefully that will open up more doors for me. I know this must be cliché to quote this words now, but still,

“Stay hungry, stay foolish. Keep looking, don’t settle”.

staticmethod in python

Going through byte of python 3, I tried to make sense of staticmethod in the section talking about class and Object variables.

In the Robot class, there is a howMany class method, declared as below

def howMany():
  print("we have {0:d} robots".format(Robot.population))
howMany = staticmethod(howMany)

or using decorator, it will be

@staticmethod
def howMany():
  print("we have {0:d} robots".format(Robot.population))
howMany = staticmethod(howMany)

It was then called by the Robot class,

Robot.howMany()

I couldn’t really understand what is the staticmethod role here though.
According to the Python documentation,

A static method does not receive an implicit first argument.
It can be called either on the class (such as C.f()) or on an instance (such as C().f()). The instance is ignored except for its class.

That explain a little bit more, seems a staticmethod can be called either by class or instance.
Then I try something like this,

class Human:
  def communicate():
    print("I talk")
  def move():
    print("I walk")
  move = staticmethod(move)

So a Human class have two class methods, communicate and move, but move is a staticmethod.

Steve = Human()
Steve.communicate()
Traceback (most recent call last):
  File "", line 1, in 
TypeError: communicate() takes no arguments (1 given)
#communicate() is a class method, when object Steve try to call it, an error message is saying communicate take no arguments, but 1 is given. I guess that is object Steve trying to pass self to communicate()?
Steve.walk()
#It duly print out "I walk"

Seems to me, staticmethod is a way for object to access class method that otherwise wouldn’t be available to them. Is that the correct perception?

Openshot 1.4

I tried to get Openshot working again. There are suggestion that locking libcairo2 at version 1.8 would fix the problem, but select libcairo and Ctrl + E in Synaptic didn’t give me option to choose and lock it at any particular version. Maybe I miss out some steps. So I figure maybe I should try uninstall and reinstall libcairo, big mistake. After the installation done, it just wouldn’t let me go into the desktop environment.

Maybe there are ways to fix this without reinstalling Ubuntu, but since I will be having problem anyway with Openshot even I fix this issue, I just go for a clean installation instead.

I pull out the 11.04 installation disc put into the CDROM, and at about 2 hours later, I got Ubuntu back, sure enough, installing and running the spanking new Openshot 1.4 without a single problem.

A noob journey on video editing

Due to the really diversified nature of my job, somehow I ended up being assigned to create a video for someone in higher management for a presentation which gonna happen pretty soon, in 2 weeks time to be exact.

I do have some very basic experience with video editing stuff before, but nothing too complicated really. So what follow is more like a journey into searching for suitable video editing tool from the point of view of a noob.

MovieMaker

Seeing that the only tools that I can use immediately within the office environment is movie maker, so I don’t really have much choice. Hey, it comes bundled with Windows, what can I do. The editing interface is easy enough to start with, dropping in simple effect like fade in fade out, sweeping from the left, fly out, fly in, almost make you feel like working in a PowerPoint environment (that might not necessary be a good thing for a lot of people though)

The problem arise when I try to publish the project into a proper video format after almost half a day of effort, it start to complain that it could not locate some of the source files!! Reason could be some of the files are moved, or renamed while I am working on the project file.

Strange thing is I can still view all my editing in Movie Maker, just couldn’t publish it. If I copy the project file and all the source file I am working with to another computer, it couldn’t even play out in Movie Maker, I think that is even stranger.

Worse is, as I search thru the net, most of the suggestion is that there is no way to fix it, other than redo everything.

Lightwork Beta

I tried the open source Lightwork Beta, the interface look really professional, it packed with lot of advance features, which of course means the interface is a lot more complex, and the learning curve is steeper. But the show stopper for me is the limited choice of video format, especially when it wouldn’t work with flv and mp4.

I am not sure if there are plugins or add-on or any other hacks around for the conversion, or I could actually use another conversion tools(video transcoder like handbrake for example) before using Lightwork Beta to deal with the editing, but then again using yet another tools create another layer of complexity. And giving the short timeline I have for the work, I really don’t wish to spend my time doing that.

 After Effect

Then I think of trying out Adobe After Effect, CS5.5 only work with a 64-bit machine, so I started to download the whooping 1G trial version of After Effect CS4 (for some reason I couldn’t find the download direct from Adobe site, and ended up download the trial from ProDesigntools, here). While I am sure After Effect will handle the tasks without any issues as it is a tool used by a lot of professionals, but eventually I have been turned off by the complexity of After Effect as well.

Looking at all the buttons and menus available, I just couldn’t stop feeling lost. I am still hugely interested in learning about After Effect, but then again this video editing thing, according to the higher management, “should not take up too much of your time”, but still “it has to be completed within week”, and with all the bells and whistles required.

Avidemux and Openshot

That leave me thinking, maybe I should do it at home in Ubuntu since I have some experience working with Avidemux before, a rather simple video editing task I must admit. So I come home and set out to look for good video editing tools.

I couldn’t remember why I didn’t use Avimux, maybe it seems too basic, I read through reviews in Ubuntu Software Center for video editing apps, it seems Openshot get quite some rave reviews. So I quickly installed Openshot, and the reviews are not too far off, pretty easy to use with some nice effect library. While I am busy editing the videos on hand, the higher management come with another order, to have all video in the same view size. So I try frantically looking for the option to adjust view size in Openshot, I am not sure if it is bad luck that all of a sudden Openshot just refuse to start.

I did a brief research on the net, seems that it might be related to some package being updated that break Openshot. Uninstall and install may or may not help, or updating some packages. But then, I decide to try another apps instead.

update, as I wrote, I read that there is already a newer version of Openshot 1.4 available, from the announcement video, it looks great, I will certainly give it a try soon.

Pitivi

Eventually, I tried Pitivi. Pitivi come with even lesser features than Openshot, or MovieMaker. I couldn’t find any effect or transition library around, adding layer or fading in and out video depending on adding in keyframe and adjusting the opacity accordingly, which is surprisingly easy to use after you get the hang of it.

One neat feature in Pitivi is the mini preview sort of thumbnail picture along the timeline, so it give user a better hints on what scene is on which part.

But still, I am looking for way to adjust view size, and then I found in the net that there is an update version of Pitivi around, and there are tons of new features added in, re-sizing viewer being one of them.

To get the updated version, just do this

sudo add-apt-repository ppa:pitivi/stable
sudo apt-get update
sudo apt-get install pitivi

Then you should get the latest stable version, 0.15. So far I only encountered very few problems with editing, and rendering in Pitivi. I got this useful tips for getting the optimal setup for video rendering in Pitivi, from the ubuntuforum, comment from theSuperman

Set Container to MP4
Audio Codec set to Ffmpeg ALAC encoder
Video Codec to Ffmpeg MPEG-4 part 2 encoder
Change the video codec bit rate to “10300000”

I try similar setup with container change to AVI, and overall quite satisfied with the quality.

Summary

For a lighter and simpler, and of course free video editing tools, so far my experience favor Pitivi, it is easy to use, stable, and in the latest version, it come with quite a lot of useful features. I will definitely try out Openshot latest version 1.4 when I have time, hope it is even better than Pitivi, healthy competition will benefit everyone.