Category Archives: Programming

Quora answers “Brilliant” problem

Came across this question on Quora (Quora is an infinite time sink BTW!) The original question is at https://brilliant.org/i/ge9jJs/

Wonder why people post puzzle-type questions on Quora!

Anyway, the highest ranked answer gives a nice way to approach the problem. I would not be able to think that way and just used the brute-force:
Here is the ruby code to solve it:

def testit(a)
  (x,y,z)= a/100%10,a/10%10,a%10
  return a == x*x+y*y+z*z + 543
end

q=(111..999).select{|q| testit(q)}
q.reduce(:+)

The answer is 2626, the last 3 digits being 626.

HTML5 Up and Running

HTML5 Up and RunningHTML5 Up and Running is Mark Pilgrim's book on HTML5. The entire text of the book is also available at DiveintoHTML5.org. Just like the other books written by Mark (Dive into Python, Greasemonkey Hacks), this book will serve as a very comprehensive introduction to the topic.

The book begins with history of HTML specification starting from conversations on www-talk mailing lists to formation of W3C and to WHAT Working Group. Then it moves on to Feature Detection, high level view of new features - canvas, video, Storage, Web Workers, Geolocation. The next chapter is a dive into details of the Document elements, new semantic elements. The next few chapters cover in detail each of the new features - Canvas, Video & Audio, Geolocation, Local Storage, Offline applications, Form semantics, Microdata. Each of these later chapters can be read stand-alone without depending on others.

There are some open source projects mentioned in the book - Modernizr for HTML5 feature detection, geo.js for smoothing out differences over gelolocation APIs. These pointers should be of great value to developers.

The book website is itself a great study in HTML5 with its very detailed attention to live examples, typography. Great work by Mark once again and kudos to O'Reilly for allowing full version (which is in fact more up-to-date than the printed book) online and also for selling the ebook in DRM free formats!

Disclaimer: I am writing this post as a part of Blogger Review program. I am not being paid to write this review. But I received the ebook free for doing this.

twitter auto refresh

Google chrome - CTRL+SHIFT+J to open javascript console. Type the following line there...

function updateIT(){$('#new-tweets-bar,.new-tweets-bar').click();setTimeout(updateIT, 1000);}updateIT();

Update: Here is a small extension if you are on google chrome (or using Firefox with greasemonkey installed).

Update (2013-01-15): Playing with twitter is a cat and mouse game! Here is what works as of today:
window.setInterval(function(){$(".js-new-tweets-bar").click();},10000);

First impressions on Google Wave

I was really looking forward to get a chance to play with Google Wave. If you have not heard about this, Google is experimenting with a new means of communication and collaboration (to destroy any remains of your unused time!). Watch the video of the demo from Google IO conference to get a feel for the technology.

The catch line is - How would email be if it is designed today ?

Being Google, they are fairly open about the entire technology.Google Wave is a collection of several components which work in concert to bring us this amazing way to collaborate and communicate. There is the wave server (which hosts the waves. Google provides an implementation and others are free to implement it in their own control), federation protocol (which is open specifications protocol and allows the servers to talk to each other), the client (typically your web browser which you use to interact with the wave server, but there is a sample text client and emacs based client in development as well!), the gadgets (small pieces of code that are embedded in documents and provide rich look and feel and additional functionality to the wave) and the robots (robot participants in the wave which can do cool things like correct spelling as you type, syntax highlight code while it is being pasted in the wave, translate language etc.)

I have spent some time in developing a robot called Nokar (meaning assistant or servant in Marathi/Hindi) which can do several things when invited to a conversation - Insert images based on specified keywords, translate text between a set of 20 languages among some other geeky functions. The intention was to learn about the robot protocol. I also created some pages which use the embed API. This allows any web page to embed a wave conversation (or a subset of it). I am also going to experiment with the Gadgets in the next few weeks. I will try to document my process in next few posts.

RSS Feeds for Indian Columnists

Frustrated by Indian Express's inability to provide individual syndication feeds for its columnists, I have written scripts to parse the HTML pages and generate the feeds myself.

Here are the feeds for
Shekhar Gupta
Tavleen Singh.
R. Jagannathan (DNA India)
Arun Shourie
Sudheendra Kulkarni
Ila Patnaik
Pratap Bhanu Mehta

If you want this for another columnist, let me know and I will add that too. This is very easy to do for Indian Express columnists as I already have the script, but I can also help with other websites.

P.S. The script is in ruby and I will release the source after I fix some things and clean it up some more.

Update: May 29, 2009
Added new feeds for Arun Shourie, Sudheendra Kulkarni and Ila Patnaik

Update: June 22, 2009 - Added columns feed for R. Jagannathan of DNA India

Update: August 11, 2009 - Feeds for C. Rajamohan , Harsha Bhogle , Shailaja Bajpai.

Update: Sept. 10, 2009 - You can use my shared page from Google Reader to see all the new posts from all of these columnists on a single page.

Update: July 18, 2011 - Added Karan Thapar.

opml to csv converter

This is a first step in being able to make all  my  planets    configurable from anywhere. The following ruby script parses the opml file specified on the command line and generates a comma separated file with XML feed URL and feed title. In case of nested outline elements, it just picks the elements which actually have xmlUrl attribute (this will flatten the opml hierarchy which is used by google reader - for implementing labels and bloglines - for implementing folders)

require 'csv'
require "rexml/document"
include REXML
if ARGV.length >=1
fname = ARGV[0]
else
fname = "opml.xml"
end

doc = Document.new File.new(fname)
CSV.open('csvfile.csv', 'w') do |writer|
doc.elements.each("//outline[@type='rss']") {|element|
writer < <  [element.attribute("xmlUrl").value, element.attribute("text").value]
}
end

django unicode integration: fix for venus djando template

I just upgraded django tree which recently merged in the unicode support. This immediately broke django templates for venus. Here is what you need to change in planet/shell/dj.py to account for new django changes:

43c43,46
< f.write(t.render(context))
---
> ss = t.render(context)
> if isinstance(ss,unicode):
> ss=ss.encode('utf-8')
> f.write(ss)

This is probably due to render returning unicode strings which need to be converted to byte-streams.

Update: I found out that my changes broke it for people using older version of django. I have updated the patch above to account for that.

Dear Lazyweb

I am looking for a flash media player which can play audio/video content in browser. I am currently using del.icio.us's playtagger for now, but it is limited to mp3 only (I had to change the code a little bit to prevent it from adding multiple inclusions - my changed code is here) and it does not allow rewind/forward.

I also looked at this post which suggests using google video player, which would be nice as it could then also play video files, but it consumes a lot of real estate on the pages. There is also this website mentioned which is nicer but that player is bigger than I want too... And it is probably not a fair use as it is undocumented use...

I also checked out the odeo player, which is nice but you need to host the files on odeo, which is a no-no.

Here are my requirements for this with a scorecard for Playtagger and Google:

  • Easy to use. No need to add tag soup for each file. (Del.icio.us + Google -)
  • Small size on the page (need to collapse the buttons until user clicks on play button) (Del.icio.us + Google -)
  • Ability to rewind/fast forward (Del.icio.us - Google +)
  • Play different media formats (mp3/wav/ogg/avi/mpeg video, possibly realmedia) (Del.icio.us - Google +)

Any suggestions ?

Thanks!

Fix maharashtratimes.com font problems on firefox

Maharshtratimes.com (or maharashtratimes.indiatimes.com) is a marathi news website using unicode fonts. But it does not display correctly on firefox browser. The problem is because of a single HTML div which uses justified font style. It displays correctly on IE (which is why it is not getting fixed) - this could be because of firefox's buggy implementation of "align: justify" or that IE simply ignores that style (likely).

Anyway, here is a javascript one liner that you can bookmark and once the page is loaded, click on it to fix your font problem.

Fix Ma. Ta. -- Drag this link to your bookmarks.

I tried to create a greasemonkey script to do this automatically, but it's not working for some reason...

Update: Apparently it *is* a mozilla/firefox bug open for 4+ years. See here and here.

Update2: Here is a greasemonkey script by Saravana Kumar to fix this issue. Caution: you might want to change the included domains carefully (it by default runs on all http and https sites!)

Update3: Here is my greasemonkey script specific for maharashtratimes. Enjoy!

Update4: (2009-02-16) The original site seems to have removed this style attribute now. So the above post is now only for posterity.

Using Ruby for integer format conversions

Here are some recipes for interpreting stream of bytes as different C types. I will keep adding more to this as I go...

Convert a byte stream (embedded in a string) into 1 byte signed integers:
"\xfc\xfd\xfe\xff".unpack("c*")
=> [-4, -3, -2, -1]
Convert a byte stream (embedded in a string) into 1 byte unsigned integers:
"\xfc\xfd\xfe\xff".unpack("C*")
=> [252, 253, 254, 255]

Check if a site is phishing site.

Here is the updated bookmarklet: Phishy? (tested on firefox 2.0 only!)

1. Drag this link to your bookmark. This checks if the site you are currently on is a phishing site.
2. Drag this link to your bookmark. This prompts for a URL and checks if it is a phising site.

Uses phishtank's check URL API.

If this does not work try turning debug to true above if you want to see the encoding.

Update: This still uses the GET method for checking the URL. Phishtank recommends using the POST interface (which will remove limitations on URL length: base64 inflates the length by 33%). Implementing that would need some kind of xmlhttprequest hackery. Stay tuned...

Update2: I got the AJAX bookmarklet ready, (thanks!)but it hits the infamous "uncaught exception: Permission denied to call method XMLHttpRequest.open" bug. i.e. you cannot do cross-domain xmlhttprequests. To solve that I think I need to convince PhishTank to host the javascript code, so the bookmarklet will insert a hidden iframe into the current page which will load the javascript from phishtank page, which will eventually make xmlhttprequest to phistank and display the result back. Are you listening PhishTank ?

Update3: Thanks to "till" who commented below, here is the bookmarklet using the POST method so now the solution will also work for really long URLs. Till's solution is good, but it makes users trust his site (in addition to phishtank). So basically user has to trust that he is not trying to filter the results being presented..

I have also merged the two earlier bookmarklets so that the current site location will be autopopulated in the prompt, so that user can easily change it if he wants to check a URL different from the one he currently is on.

Fixing the geocoder gem.

I wanted to use the ruby geocoder library on the windows machine, but the installation of the gem failed due to some weird error. I checked the rubyforge project page to see if someone else had a similar problems and someone actually had, but the bug was open for a long time. I decided to fix this issue and found that the problem was due to the fact that windows platform does not allow characters '?' and '&' in the filename with any escaping, period. The said files were used (in a very innovative way I must say!) to test the library by modifying http.rb to return the test datafile contents instead of fetching the URL from the net. (yay open classes in ruby!). The way I fixed the problem was to change the filenames to use '_' instead of '?' and '__' instead of '&'.

I wrote to the developer, but there was no response. Anyway I managed to create a new GEM with the changed files so that this should be installable on windows now. Here are the files if you want to try installing the gem. (Also including the tgz because... it got generated anyway!)
geocoder-0.1.1.gem
geocoder-0.1.1.tgz

More about Phishtank API

Here is what will be good-to-have from phishtank.com API:

  • Good documentation about each interface e.g. how is callback_url used by auth.frob.request API ?
  • Description of all possible fields in return response (all possible XML elements and their possible values)
  • Some test URL's and emails which will return known responses (i.e. phishy URL, good URL, not in the database etc.)
  • Developer mailing list/wiki
  • Response should always honor the responseformat parameter if specified and valid

Phish Tank

http://www.phishtank.com is a new service which aims to help weed out phishing URLs and email addresses using wisdom of the crowds. Users can submit emails/URLs which they suspect of fraud and others can vote if they really are fraudulent or not. I think it is a great concept. There is a REST API using which applications can embed this webservice within them. So for example, there could be a outlook plugin which will display "phishy" email addresses in a special way in order to alert the user immediately. Same for web browsers which can render phishing websites in a special stylesheet. The applications can also add interface for the user to submit suspect pages and email easily without using web browsers.

I checked out the API and it does not feel like it is fully baked! There are interfaces for authorization and checking email/url status and submitting new emails/urls. Some things that stand out immediately are:

  • Exclusive use of SSL for the API access.
  • Parameter authentication (i.e. including cryptographic digest of all the parameters to ensure that parameters are not changed using man-in-the-middle attack)
  • Choice of xml or php output.

The api calling sequence works like this:

  1. User registers on the web for API access and gets api key and shared secret
  2. Using the API, application gets a frob (what is behind the name ?) and authorization url using auth.frob.request
  3. User has to authorize the frob using the authorization url specified in the response. (optionally you can specify callback url which the server will call for authorization, I will need to check this from home when I have access to a server -- the docs are very thin about the mechanism)
  4. Once authorized, app uses the frob and gets a token for short time API access (30 minutes in my tests) (auth.token.request)
  5. App can check token status which tells remaining time on token.(auth.token.status)
  6. App can revoke the token when it is done using it. (auth.token.revoke)
  7. The APIs for check.url, check.email, submit.url, submit.email then use the token.

I did not understand why there is a need for FROB in this, why can't you just get the token from api key and shared secret ? What problem are they solving by this indirection ?

Anyway, here is the ruby script that I used for testing this... I am planning to turn this into a module, but providing it here for early access...
phishtank.rb
config.yml

P.S. the check_url interface is not working, I am getting invalid token error. and the same token can be revoked successfully.
P.P.S. The API uses SSL (no cleartext api available) and ruby's open-uri library insists on checking the server SSL certificate which always fails (probably because signer needs to be trusted by openssl), I had to change it locally to ignore ssl verification in order to proceed.

Update (Oct/12/06): the check.url interface is finally working. For this API, the signature needs to be calculated before escaping the url. I refactored the ruby script a bit to remove redundant code and moved the configuration to a seperate file. I still need to work with the response parser and make it general for all types of responses. XML parsing gets so ugly so fast, it's amazing!

Ruby script to get a list of all mp3 files in a directory sorted durationwise.

Here is some ruby practice...

def mp3len(dirname)
require 'mp3info'
summary=[]
Dir[dirname].each do |f|
Mp3Info.open(f) do |info|
summary.push([f,info.length])
end
end
summary
end

puts "Enter directory containing mp3 files"
dirname=gets.chomp
dirname = File.expand_path(dirname)
puts "Searching #{dirname}"
dirname += "/**/*.mp3"

s=mp3len(dirname).sort {|a,b| a[1] < => b[1]}
s.each{|q| puts "#{q[0]} => #{q[1]} seconds"}

Update: Here is the link to the file on github (for updates) or the git repository