So, it was good to have found a good source which described the process in four simple steps using the shell. It worked.
Step 1: Backup the existing database.
I would not want to lose my work, specially the blog, so against my natural tendency, I did backup.
[me@mywebserver]# mysqldump -u lazyinvestor -p
Step 2: Get the latest wordpress .zip and unzip
This would generally work, but you should be smart enough to know there are other ways.
[me@mywebserver]# wget http://wordpress.org/latest.zip
[me@mywebserver]# unzip latest.zip
Step 3: Overwrite all the new files onto your old ones
[me@mywebserver]# cd [to_whereever_your_wordpress_files_are]
[me@mywebserver]# cp -avr [from_the_unziped_latest.zip]
Step 4: Open http://yourblog_url/wp-admin/upgrade.php in your favorite browser.