Face-ing your future

A lot of you are absolutely aware of what Web 2.0 means and what change that has brought about. Among the plethora of changes that it continues to offer, the rise of Social Networks and the way it permeates into each of our lives is no longer something new. To be fairly honest, articles talking about the rise of Social Networking and the way people interact on Facebook these days sounds trite and contrived. Yes they exist, but how do they transform computing in general? I haven't found a lot of articles that talk about what is in store for the next two years in terms of computing and in terms of what to do with the wealth of information that these networks dish out to us and what techniques are used to tackle them.


While a single glance at a Facebook page, one can really predict the type of the user. *This applies to any human who just looks at the profile and the Wall updates, along with the frequency of updates*. While I am not going to discuss on the updates that focus on how a single great looking female's inanity has been appreciated or how a certain celebrity's foolishness has been applauded, I am certainly looking at how these obscure status messages can turn into potential business goldmines.

Coming to the crux of the situation. A seemingly innocuous status message like "My dream!" along with a picture of a "Porsche Cayenne" gives an indication that the poster is concerned about or even likes that car. If Chaos Theory can argue that the fluttering of a butterfly's wings can cause an earthquake on the other side of a world, surely a status message can prompt a lot of businesses to latch on to the opportunity. While not everyone might want a "Cayenne" but surely a lot of us might want Xboxes, Wiis, Cameras, Mobile Phones, Kindles and what not? Enter Natural Language Processing. For decades AI researchers have talked about how NLP can be used to understand text. While most of these projects were really academic and served as interest to a lot of scientists and researchers, their true test comes out in this age of the unstructured data. A system with even 50% of accuracy, when run on a set of 100,000 status messages can give you a substantial idea on what people are talking about. Why then, these businesses can utilize this to directly talk to the user, probably come up with offers and go one step ahead in the marketing of the product.

Even as we are talking a lot of people are looking at this problem and are figuring out ways to solve them. One of the greatest challenges in building such a system has been the way in which such a system can be show-cased to the client. While Human Beings can be inane, nothing can take away their brains from them. So where are the stumbling blocks to processing this?

1)How do you understand the domain that the user is speaking( Even if the domain is resolved, how can the machine evolve and learn? Can parsers co-exist with ontologies? Is an ontology based parser probable? If we claim that the brain has an internal wiring and then assuming that ontology is the poor cousin of the brain, can we parse?

2) How do you present it as a sell-able product? What kind of examples do you show case the client? This is definitely the biggest challenge. The challenge arises from the fact that the client here is definitely presented with a virtual goldmine hidden in some treasure chest and the problem is that he does not know how do get to the goldmine. More often than not the client might get the wrong assumption while looking at an application. The way he tests it should be on the same lines that the program was tested. How can both the business user and the application developer align their thoughts? Very critical as it makes or breaks your product.

3) Finally how do you scale the system?. Scalability is always the question in many applications, but in my case I'd rather say, how easy is it that the system learns new patterns. Can it apply the patterns from one domain and use the same to extend it to another domain? Would that be a worthwhile route to take?

Loads of questions, very little and diverse answers, the future, probably is not bleak, but then there are tough times ahead for products which don't embrace the power of Social Networks.

A verbal thought (Literally)

Wrote this in my company blog today and since this space keeps languishing every once in a while, I thought I'll update it here

How does the human brain process speech? How does it understand the semantics of language? If there are ambiguities, what mechanisms does it use to resolve? For example lets have a look at these sentences, He is a man of simple wants and He wants money.

Admitted this might be more for people who are interested in Artificial Intelligence or probably wonder how to build systems that understand text. In one of the scenarios that I was working, I came across something fascinating. The words need and want are very interesting. As a noun they really don't require an object action to finish the sentence. They themselves become synonyms. For example look at the sentence He is a man of simple needs. Assuming a machine can process that needs is almost a noun here and can identify the underlying meaning behind needs, you have an absolutely great semantic engine.(I'd use the word semantic as it makes more sense :-) ).

Now here are two interesting sentences :

I need a camera.
I want a car.

In both the cases the verb is a simple verb and requires an object i.e camera or car to complete the sentence. Processing them is a breeze for some of the systems today. Heck! even a rudimentary system developed by us can process these kinds of sentences.

Lets get into a more specific scenario. Now the goal is to understand how verbs like need and want behave. Lets introduce a bit of anarchy (literally) and see if these words behave just the same?.

I need to purchase a car
I want to buy a house in Chennai.

Notice anything different? At first glance you see that there are two new verbs introduced purhcase and buy. Well both of them essentially means the same. You could also add construct to the second sentence. Now you have two verbs in the same sentence. How does your brain resolve? Do these steps happen?

1) Decide that need takes in an object to complete it's meaning.
2) Looking for an object action, come to purchase a car, buy a house.
3) Realize that purchase and buy are two verbs which in turn require an object action to complete their sentence. One cannot literally say I want to buy. There is always the question of What action that follows the verb.
4) Then decide that want to purchase actually points to a desire to verb. i.e desire to purchase.

Essentially when faced with words like want, need, a system cannot just process them alone. It has to look for the other verbs in the sentence which aid in their meaning. So the secondary verb is of more importance or is the deciding factor in the sentence. If any of you can help me figuring out much complex situations, you are welcome. I just am pouring out my thoughts here.

One should marvel at the wonders of the human brain. All of us could handle these sentences even in our first standard. In trying to mimic the brain, I just am stumped at how the brain handles complex things in such simpler steps. As for me, I think finishing the Wren and Martin book again, with a deeper understanding of each of the parts of speech is something that cannot be put down for ever :-).

Correction:I had written this in my company internal blog and Rb corrected me at the parts where I have crossed the text. This is his comment :

In my view it is "I need to act" and that action could be "run" or "purchase a house". The What is not mandatory. The "house" to me belongs to the action of "purchase" and not of "need".

Though "I want to purchase a house" is semantically similar to "I want a house"; they cannot be treated the same. Examples of the form "I went to purchase a house" or "I want to sell my house" show the tie-in of the "house" to the "action" and away from the "want".

That sense of Awkwardness

Awkward : socially uncomfortable; unsure and constrained in manner; "awkward and reserved at parties"; "ill at ease among eddies of people he didn't know"; "was always uneasy with strangers"

Have you ever been in an awkward social situation? How do you react to that? Do you get carried away by the moment? Do you really reflect on it? Do you later on try to view the situations with your inner eye? How much of what people say do you really care about? What do you do in the following situations:

The Sudden meet: So you have great friends. As friends you hang out together, you make plans for weekends, you make fun of each other and all the normal things that friends do. There cometh a time when the friends move on especially when they find potential life partners. Their lives become different then. The luncheons become progressively lesser, meets rarely do happen. The times become hectic. Suddenly out of a blue you meet each other. Both of you have nothing to say. What happens next? Awkwardness

The Repartee : So you for a long time bear the brunt of teasing in your group. One fine day you just happen to joke on one of the others. The person receiving it takes it so badly that sometimes the accusation of negativity drops on you. What do you do to reply? Every other day the person would have been taking their liberties on you, making fun of you with all and sundry. All this takes a negative turn when you give the joke back to them. The next moment all you feel is plain Awkwardness.

The Worker : You bug your best friend to do a lot of work for you. He might be better than you at certain computing tasks. You get all your work done from him. You have the audacity to disturb him and go to the extent of even getting upset since he does not finish your tasks. After everything gets over, suddenly you vanish. You become more important. You have everything now. The next time when he calls you, you ask your office colleague to pick the phone for you. How would your friend react next? Awkward

The Accusation : So you have your certain set of interests. You have your ambitions. You love doing certain things. What happens when somebody says that you do the very things to satisfy somebody else or to be approved by a group. The next moment your whole life flashes behind you. You look at yourself moving through time and then take a moment to get back. Another gem of an Awkward moment.

Life is a roller coaster ride of little moments. One cannot be his best at every given point in time. One aspires to be. Sometimes, the best of them would lose track in the constant barrage of negative cannons thrown at them. Obviously one has to take it. Because only in falling can we rise up again

PS : I really don't know what to write on a rainy day. These were just reflections of a mind which is tired.

The varied streams of reasoning

A daily quiz question mail-list sent a question with three pictures, asking us to give a connect between the three. I found that the first two pictures just gave away the connect. The first was a picture of Stephen King, second was a picture of Rita Hayworth. To anybody who has watched The Shawshank redemption, this would be a piece of cake. I guessed the third picture to be Frank Darabont, which was probably inferred from the fact, that I know Stephen King and Rita Hayworth connect only for this movie and the other closest connect I can find is Frank Darabont. It turned out to be right when I verified my thoughts.

What exactly triggered my brain to give the answer as Frank Darabont. At a mathematical level I'd say that the intersection of {King,Hayworth,The Shawshank Redemption} can lead me to certain connects like {Morgan Freeman,Bob Gunton, Frank Darabont,...}. Now I know that the third picture is not Bob Gunton or Morgan Freeman because I know how they look like. So the only possible connect here was Darabont. While we can mathematically eliminate a set of choices, the visual recognition also plays a huge part in the final elimination here. Yesterday's question was similar. I had to look at an epitaph of a young poet to figure out whom the epitaph belonged to. The young poets I knew, who died early in life were {Keats,Wilfred Owen,...}. I decided to look at the date of his death. Now that leads me to only Victorian Era poets. Here the only Victorian Era poet who died and closely resembled the epitaph was Keats. That turned out to be the answer. What amazes me is the fact that so much stimuli are hit upon when our brain is asked to process or deduce an answer. The cognitive processes that follow deal with visual recognition, context understanding, elimination of other choices.

These are minimal observations from somebody who also is a budding practitioner of Artificial Intelligence, but the questions these observations throw are mind boggling. How do I for example build a simple reasoning engine which can also accumulate knowledge. From the previous examples the accumulating knowledge part was how Frank Darabont and Keats get added to the brain. Now they are stored permanently in my brain and probably would serve for future knowledge gathering.

As Steven Pinker put it, The main lesson of thirty-five years of AI research is that the hard problems are easy and the easy problems are hard. The mental abilities of a four-year-old that we take for granted – recognizing a face, lifting a pencil, walking across a room, answering a question – in fact solve some of the hardest engineering problems ever conceived.... As the new generation of intelligent devices appears, it will be the stock analysts and petrochemical engineers and parole board members who are in danger of being replaced by machines. The gardeners, receptionists, and cooks are secure in their jobs for decades to come

The analogy of a Human Brain

No great content ahead folks. So I just did a Brain.rand(MyBrain) to come up with something.What turned out were highly unusual. I thought my brain was just another empty shell but it's seemingly decent storage capacity did return a myriad of content. So here goes a few random thoughts.

I was leafing through A Study in Scarlet by Arthur Conan Doyle where for the first time he introduces Dr.John.H Watson and Sherlock Holmes, the greatest fictional detective ever. Watson is piqued at the fact that Holmes knows nothing about the earth or the sun or even anything about Poetry and Carlyle. Did a google for the exact text and I got it...

"His ignorance was as remarkable as his knowledge. Of contemporary literature, philosophy and politics he appeared to know next to nothing. Upon my quoting Thomas Carlyle, he inquired in the naivest way who he might be and what he had done. My surprise reached a climax, however, when I found incidentally that he was ignorant of the Copernican Theory and of the composition of the Solar System. That any civilized human being in this nineteenth century should not be aware that the earth"


This really set me on the contemplative mood. How can Sherlock Holmes to whom knowledge and perfection is of paramount importance can not be aware of this basic fact? Why was he not too keen to update his knowledge?So many questions just raced through my mind with each one trying to win the race. The next few paragraphs cleared my questions. Surprisingly enough they made me look at learning in a different perspective. Just read the following extracts from the novel and you'll get a wind of it.

Sherlock Holmes : "You appear to be astonished," he said, smiling at my expression of surprise. "Now that I do know it I shall do my best to forget it."

Watson: "To forget it!"

Sherlock Holmes : "You see," he explained, I consider that a man's brain originally is like a little empty attic, and you have to stock it with such furniture as you choose. A fool takes in all the lumber of every sort that he comes across, so that the knowledge which might be useful to him gets crowded out, or at best is jumbled up with a lot of other things, so that he has a difficulty in laying his hands upon it. Now the skillful workman is very careful indeed as to what he takes into his brain-attic. He will have nothing but the tools which may help him in doing his work, but of these he has a large assortment, and all in the most perfect order. It is a mistake to think that that little room has elastic walls and can distend to any extent. Depend upon it there comes a time when for every addition of knowledge you forget something that you knew before. It is of the highest importance, therefore, not to have useless facts elbowing out the useful ones."

Watson : "But the Solar System!" I protested.

Sherlock Holmes : "What the deuce is it to me?" he interrupted impatiently: "you say that we go round the sun. If we went round the moon it would not make a pennyworth of difference to me or to my work."

Looking back, is Sherlock Holmes not right? Should we also limit ourselves to what is useful for our line of work? Or the skeptic in you says the brain has every right to store any kind of information. I reflect on the above paragraph and feel Mr.Holmes was probably right. What do you think?

The travails of being a Programmer

Wrote this in my company internal blog sometime back. Had nothing to post and just thought that I could post it here.

The last traces of a human company in my floor just vanished now. I just stood up to see that I was the sole occupant of an entire floor. Studying for certifications can be a boring task. Especially if you have a running nose the entire day. I still don't know why I should be studying something called ITIL, when there is a pressing need for me to learn Computational Linguistics. But what I have here is an incident that happened to me yesterday. I'd really like to share it with you folks. The great Larry Wall once wrote about the three qualities of a programmer. So why do I bring this up now?

Yesterday, I had to meet my Big Boss to clarify certain things. Well it was something entirely personal and nothing to do with the official work. He was immersed in his work and asked me to wait for a while. I Hung around** for a while. My eyes scanned the space emptied by the occupant there and so I had to trouble the the other team mates. This is the place where I realized, how important those prophetic words of Mr.Larry Wall were. One of the senior folks was actually working with a XML file. The XML file was generated with some blank spaces between tags. Her task was to remove these blank spaces and generate a clean XML. To illustrate the problem, her XML tags looked like StartTag(loads of blanks)some_string EndTag . Now all she had to do was to remove the loads of blanks. To folks well versed in notepad++ and TextEdit, this would have seemed trivial. A simple find and replace would have worked. TextEdit for some weird reason was not working. I looked at her and asked if she would consider writing a program to do it. She was not interested as ..it would take more time to write a program. So she was manually removing the spaces. I said I'll write a java program.(These days I am familiarizing myself with Cygwin and so I thought I'd as well try the powerful tools in the Unix shell). She was very apprehensive and inspite of her sweet denial I could figure out that she was really not so interested in the task.Well it would have required only two lines in java. Assuming there were only blanks or spaces. All she had to do was this :

String a = //The string from the XML file
String b = a.replaceAll("StartTag\\s+", "StartTag");

Probably that's not the best way to do it either. There might even exist a simpler way. Thankfully today I realize that there might be some traces of the laziness that Larry Wall spoke of. Oh! Btw you should have also thought about the immutability of Java strings if you viewed the above two lines of code.

This might be just me. Some folks would still love doing the manual way. I have no qualms about that. But, I wonder if the creative spark in me would die a slow death. Will I start imitating my seniors and shed the remnants of any creativity that I have? I get reminded of the movie Idiocracy where the protagonist lands in a future where the humans have become totally dependent on machines and cannot think for themselves. Hmm.. I really don't have an answer to the question now.

I just remembered a beautiful poem by P.B.Shelly called Ozymandias. It talks about the crumbling of mighty empires and the ultimate destruction of any civilization, however great it might be. Here is the poem:
I met a traveller from an antique land Who said:
Two vast and trunkless legs of stone Stand in the desert.
Near them, on the sand, Half sunk,a shattered visage lies,
whose frown And wrinkled lip, and sneer of cold command
Tell that its sculptor well those passions read
Which yet survive, stamped on these lifeless things,
The hand that mocked them and the heart that fed.
And on the pedestal these words appear:
"My name is Ozymandias, king of kings:
Look on my works, ye Mighty, and despair!"
Nothing beside remains.
Round the decay Of that colossal wreck, boundless and bare
The lone and level sands stretch far away

Yeah! I just hope the creative thinker in me doesn't have the fate that Shelley has put forth in fine verse. I hope that he lives on for at least 20 years from now.

(**Pardon me,I think the purists would not agree to such a usage. I'd rather use it for the descriptive purposes :D**)

**Goes back to listening songs from the evergreen movie Abhimaan and decides to call it a day**

For whom Time stands still

Time! on whose arbitrary wing
The varying hours must flag or fly,
Whose tardy winter, fleeting spring,
But drag or drive us on to die

-Lord Byron

This weekend, as is the case with any other weekend, I was cooling my heels at home. Words can be misleading for Chennai is essentially a hot cauldron for the first few months of the year. It would be rather apt to say “I was burning my heels at home”. Unmindful of the sweltering heat, I chose to sit down and from the heap of books strewn across, picked one and was immersed in it. My reverie was rather abruptly broken by a familiar voice in the streets. Why would I not recognize it? After all it’s been the same distinct loud voice, which I have heard for ages and which mouths the same two lines over and over again.

Paper, Old paper...

Unmindful of my observations, my mother called out to the owner of the voice. She asked him to come over. This was my cue. Over the years I have dedicated myself to making sure that my mother is not made to work hard for every little thing in house. This makes her feel happy and most of the times it is her smile and her silent acknowledgment that matters. I ran over to the other room and immediately began stacking the old pile of newspapers in a jute bag. The door bell rung and I was ready. The man looked at me. The glint in his eyes suggested a trace of faint recognition. A small smile ensued then in his lips. I acknowledged it and we both were into the act of shifting papers in his weighing machine. Just when a little stack remained, he looked at me, and asked me, “Son, what do you do now?”. In a low tone I told him I work. Me naming the company, which most of the educated urban lower, middle and upper class would instantly recognize would not bring the same sense of recognition in his mind. I spared him the name and tried to keep the conversation going on. I asked him how things are going on for him. He looked at me and said, “Son, it’s all the same, I do the same old work, collect papers and try to eke out a living. That has been my life for almost 20 years now”. My heart went out to him. I could not reply back. The words might have stung me. Here I was in my home making rapid strides in my life moving step by step to a place of higher recognition, whereas there was a fellow human being, who in all fairness has struggled more than me but life never has changed for him.

He gave me two soiled fifty rupee notes for the newspapers. I gave back one fifty rupees to him saying he could use it to buy something cool and drink. I guess he was overwhelmed with surprise for there were tears in his eyes. He thanked me profusely, blessed me and carried on with his work. I sat down again with my books. My mom, used to my ways asked me to keep the fifty rupee note. She was apparently happy for the house looked cleaner without the strewn newspapers around. My mind was rather engrossed in the inequalities of our lives. How many times in our lives would we hear elders asking us to work hard, for the fruits of hard work are sweet. Wasn’t that man doing the same thing? Every day he had to shout till his voice grew hoarse. The fury of the Sun will not stop him from venturing out for the pangs of his hunger are a much bigger driving force in his life. Time is a deceptive illusion I realized. For me it was the measuring factor of my achievements and age, whereas for him it was all about counting yet another grey hair while his life always remained the same. I suddenly realized I was sweating and switched on the fan. A cool wave of air spread through the room giving me temporary relief from the Sun’s fury. The voice outside became fainter. I realized he would soon move on to the next street. I wondered if I would see him ten years from now, would he still be the same. I wished he would go to places. I wished somebody would give him Redemption for the hard work he has done all these years. But these I realized were Utopian thoughts of a mind which has some solitude and loneliness. For that man though “Time will always stand still”.