Tag Archives: Internet

Why doesn't amazon allow me to add a donation to some charity when the items that I want to buy qualifies for "Free super saver shipping", but do not quite add up to $25. It makes me search for some other junk item that I don't want just to qualify for free shipping.

Sony Bravia Internet Link Unofficial API

Some notes from experimenting with my new toy!

The device (DMX) plugs into HDMI port of the TV and has its own HDMI input allowing pass-through. The control of DMX is done from a USB cable which is attached from the TV to DMX (Wonder why they do not use the HDMI for that!). DMX has an ethernet port which can get autoconfigured using DHCP (or can also be configured for manual settings from the TV). Sony has licensed a lot of content providers (slacker for radio and tons of other video providers like youtube, Amazon, Yahoo, Blip.tv). They have a portal (http://internet.sony.tv/) which allows you to add your own video links and some browser extensions which allow you to link your favorite videos which the DMX can then play for you. The firefox extension unfortunately requires older (2.X) version of firefox and does not work with newer versions (3.X). I tweaked that extension and it now works okay without significant changes. Here is a link to it. This extension adds a context menu to firefox and you can use it to direct the DMx to start playing the linked video, bookmark it etc. The caveats are that the link needs to be a raw video file (mp4, mov, avi, divx etc.) and not HTML file or any other kind.

Here is what I learned from reverse engineering the extension code.

The DMX runs a primitive web browser on port 9784. The server is reportedly called "Callisto Debug Server v0.2". There is a php script running which responds to URLs like the following
Send commands using the following REST API:

http://192.168.0.101:9784/renderer.php?method=play&url={encoded URL}

The firefox and internet explorer generates these URL's.

Here is what I gathered from the extension code about the API:

Available commands and arguments:

  • play (url => encoded URL to mp4, AVI, MOV file to play)
  • pause
  • stop
  • addbookmark (title=> encoded bookmark title, description=>encoded description, icon => ???, source =>encoded URL)

It seems to play the mp4 files quite well (I had good success with HD mp4 links from youtube and dailymotion websites which have a lot of bollywood movies). The streaming is very smooth and video quality is acceptable. You can use a website like http://keepvid.com/ to see the hidden video links. These links are then directly playable by DMX. I will experiment with AVI, MOV, DIVX files next...

Next step for me is to create a simple webservice which does all of these steps and posts the link to the DMX! Can't wait till I can manage to do that...

django unicode integration: fix for venus djando template

I just upgraded django tree which recently merged in the unicode support. This immediately broke django templates for venus. Here is what you need to change in planet/shell/dj.py to account for new django changes:

43c43,46
< f.write(t.render(context)) --- > ss = t.render(context)
> if isinstance(ss,unicode):
> ss=ss.encode('utf-8')
> f.write(ss)

This is probably due to render returning unicode strings which need to be converted to byte-streams.

Update: I found out that my changes broke it for people using older version of django. I have updated the patch above to account for that.

Search the web for devanagari words.

I created a small form using which you can search the web for unicode devanagari words. It is very cumbersome to actually enter unicode devanagari characters using qwert keyboards, so I have adopted the phonetic transliteration scheme from Manogat website. Do give it a try:

http://amit.chakradeo.net/search/ (Link now removed, please see update below)

Start typing devanagari words phonetically and you will see unicode characters in the input area. When you hit enter, the phrase will be submitted to google.

Update: (2009-07-14) I have taken down my link now, as there are many more effective alternatives to do this. Check out the following pages from google:
Google Indic Transliteration (Marathi, Hindi)
Bookmarklets to transliterate any text element on any webpage.
A simple form for transliterating any text.

Check if a site is phishing site.

Here is the updated bookmarklet: Phishy? (tested on firefox 2.0 only!)

1. Drag this link to your bookmark. This checks if the site you are currently on is a phishing site.
2. Drag this link to your bookmark. This prompts for a URL and checks if it is a phising site.

Uses phishtank's check URL API.

If this does not work try turning debug to true above if you want to see the encoding.

Update: This still uses the GET method for checking the URL. Phishtank recommends using the POST interface (which will remove limitations on URL length: base64 inflates the length by 33%). Implementing that would need some kind of xmlhttprequest hackery. Stay tuned...

Update2: I got the AJAX bookmarklet ready, (thanks!)but it hits the infamous "uncaught exception: Permission denied to call method XMLHttpRequest.open" bug. i.e. you cannot do cross-domain xmlhttprequests. To solve that I think I need to convince PhishTank to host the javascript code, so the bookmarklet will insert a hidden iframe into the current page which will load the javascript from phishtank page, which will eventually make xmlhttprequest to phistank and display the result back. Are you listening PhishTank ?

Update3: Thanks to "till" who commented below, here is the bookmarklet using the POST method so now the solution will also work for really long URLs. Till's solution is good, but it makes users trust his site (in addition to phishtank). So basically user has to trust that he is not trying to filter the results being presented..

I have also merged the two earlier bookmarklets so that the current site location will be autopopulated in the prompt, so that user can easily change it if he wants to check a URL different from the one he currently is on.

Phish Tank

http://www.phishtank.com is a new service which aims to help weed out phishing URLs and email addresses using wisdom of the crowds. Users can submit emails/URLs which they suspect of fraud and others can vote if they really are fraudulent or not. I think it is a great concept. There is a REST API using which applications can embed this webservice within them. So for example, there could be a outlook plugin which will display "phishy" email addresses in a special way in order to alert the user immediately. Same for web browsers which can render phishing websites in a special stylesheet. The applications can also add interface for the user to submit suspect pages and email easily without using web browsers.

I checked out the API and it does not feel like it is fully baked! There are interfaces for authorization and checking email/url status and submitting new emails/urls. Some things that stand out immediately are:

  • Exclusive use of SSL for the API access.
  • Parameter authentication (i.e. including cryptographic digest of all the parameters to ensure that parameters are not changed using man-in-the-middle attack)
  • Choice of xml or php output.

The api calling sequence works like this:

  1. User registers on the web for API access and gets api key and shared secret
  2. Using the API, application gets a frob (what is behind the name ?) and authorization url using auth.frob.request
  3. User has to authorize the frob using the authorization url specified in the response. (optionally you can specify callback url which the server will call for authorization, I will need to check this from home when I have access to a server -- the docs are very thin about the mechanism)
  4. Once authorized, app uses the frob and gets a token for short time API access (30 minutes in my tests) (auth.token.request)
  5. App can check token status which tells remaining time on token.(auth.token.status)
  6. App can revoke the token when it is done using it. (auth.token.revoke)
  7. The APIs for check.url, check.email, submit.url, submit.email then use the token.

I did not understand why there is a need for FROB in this, why can't you just get the token from api key and shared secret ? What problem are they solving by this indirection ?

Anyway, here is the ruby script that I used for testing this... I am planning to turn this into a module, but providing it here for early access...
phishtank.rb
config.yml

P.S. the check_url interface is not working, I am getting invalid token error. and the same token can be revoked successfully.
P.P.S. The API uses SSL (no cleartext api available) and ruby's open-uri library insists on checking the server SSL certificate which always fails (probably because signer needs to be trusted by openssl), I had to change it locally to ignore ssl verification in order to proceed.

Update (Oct/12/06): the check.url interface is finally working. For this API, the signature needs to be calculated before escaping the url. I refactored the ruby script a bit to remove redundant code and moved the configuration to a seperate file. I still need to work with the response parser and make it general for all types of responses. XML parsing gets so ugly so fast, it's amazing!

Yahoo Geocoding in ruby…

Find lattitude and longitude of any address

Yahoo just released a new beta of their maps webservice. Here is a small ruby script (inspired by Rasmus's PHP code ) that I wrote that returns Lattitude, Longitude of the address provided...

require 'open-uri'
require "rexml/document"
include REXML
url='http://api.local.yahoo.com/MapsService/V1/geocode?appid=yahoomap.rb&location='
puts 'Enter Location: '
address=gets
address=URI.escape(address)
result=URI(url+address).read
doc = Document.new result
r=doc.elements["/ResultSet/Result"]
print "Precision: ", r.attributes["precision"],"\n"
r.children.each { |c| print c.name, " : ",c.text,"\n"}

Update: Here is a link to the script in github or to the .

Anonymity on Internet

On the internet nobody knows you are a dog
On the internet nobody knows you are a dog
Unfortunately it's only true in cartoons! Basically you are leaving a surprisingly easy trail of the websites you visit. Visit Test anonymity if you want to find what web servers can know about you. A determined person can find out about the websites you browsed, what you did on each of them etc.

There are some commercial services like anonymizer that insert a random proxy between you and the destination web server. There are also a number of HTTP/Socks proxies that you can use. But then all of your traffic is subject to monitoring by these people.

Freenet project takes anonymity to other extreme and you can access content that you may not access otherwise, and also provides anti sensorship / banning features. But it has always been very slow, prone to protocol changes. (i.e. sites working the previous day do not work the next day because of release of new protocol and peer software).

Tor project takes another approach for this. The endpoints are still the same, but all your packets are routed using random combinations of tor routers. The routing technology is called onion routing where the encryption is only between hops in the route and none of the intermediate hops know either the contents of the packet or the sender. There is a provision for hidden services(any TCP protocol), which are not accessible from regular internet, which comes close to achieving what freenet does. I have been using tor for some time now and noticing some things:
* The performance is improving a great deal (as more and more tor nodes are commissioned, it will yield better performance)
* You can get routed through completely different continent, so going to google might open their german page (because they send you german page if they detect your IP address is from germany)
* This service might be easily abused by spammers who will definitely want to route spam through tor, child pornographer who can host "hidden services", illegal content downloaders. (Though I believe many tor nodes block SMTP and peer-to-peer traffic). I guess there is a price to be paid for "really free speech"