Download Android App


Alternate Blog View: Timeslide Sidebar Magazine

Friday, December 30, 2011

Time series and OpenTSDB


OpenTSDB is a distributed, scalable Time Series Database (TSDB) written on top of HBase. OpenTSDB was written to address a common need: store, index and serve metrics collected from computer systems (network gear, operating systems, applications) at a large scale, and make this data easily accessible and graphable.

This article will help you gather metrics and send it to OpenTSDB server. The detailed instructions on setting up OpenTSDB can be found here.

First, register the metric.

./tsdb mkmetric order.count.1m

A simple MySQL collector.

cat > orders.sh <<\EOF
#!/bin/bash
set -e
while true; do
  echo "Querying orders...."
  mysql -u user -ppassword -Ddatabase --batch -N --execute "select count(orderdate) from ORDERS where orderdate > date_sub(now(), INTERVAL 1 MINUTE)" \
  | awk -F"\t" -v now=`date +%s` -v host=`hostname` '{ print "put order.count.1m " now " " $1 " host=" host }' \
  | nc -w 5 tsdb.host.name tsdb.host.port
  || exit  # To handle SIGPIPE properly.
  sleep 60
done
EOF

chmod +x orders.sh
nohup ./orders.sh &

This shell script will poll the MySQL database every 1 minute, get the order count for the last 1 minute and publish it to the TSDB server. Open the TSDB screen, select "order.count.1m " as metric and you should see a graph like the following:


Most of data collection work can be accomplished by writing simple shell or python scripts. Here is a code snippet that can help you publish metrics using Java.


public static void main(String[] args) throws Exception {

   Socket echoSocket = null;
   PrintWriter out = null;

   String host = null;
   int port = 0;

   while (true) {
       try {
           if (echoSocket == null) {  // Only open the connection if necessary.
                echoSocket = new Socket(host, port);
                out = new PrintWriter(echoSocket.getOutputStream(), true);
           }

           int orderCount = getOrderCount(60);//Orders in last 60 sec
           out.println("put order.count.1m " + (new Date().getTime() / 1000) + " 100");
           out.flush();
       } catch (UnknownHostException e) {
           System.err.println("Don't know about host: " + host);
           return;
       } catch (IOException e) {
           System.err.println("Couldn't get I/O for the connection to: " + host);
           out.close();
           echoSocket.close();
           out = null;
           echoSocket = null;
       }

       System.out.println("Sleeping for 60 s");
       Thread.sleep(60000);
   }
}

public int getOrderCount(int interval){
   .........
   return orderCount;
}

More collectors are available on github.

There are many good things about TSDB like:
  • Simple to setup and get started.
  • Easy to build dashboards. Plot and compare multiple metrics on the same graph.
  • Simple HTTP API to query the time series database using "/q" option. It can produce results in HTML, JSON, PNG or plain text. This way, you can use the data for numerical analysis or use a different plotting library than the default Gnuplot used by OpenTSDB.
It will be great to see some enhancements like:
  • More aggregation functions like percentiles etc.
  • Better support for counters, sampling and debugging.
  • User guide.
  • Export to more formats like CSV, Excel etc.

StatsD is another great tool for metrics aggregation and plotting. More about it in my future blog post.

Tuesday, December 20, 2011

Globalization!

Shared by my friend:

My wife wanted knitting needles as aluminum needles, being relatively heavy, were proving to be painful for her. After some research, she decided on buying bamboo needles which are light weight. She could not find it online in India and so she ordered it on a UK based E-Commerce website and shipped it to her friend in London.

She was super excited after her friend arrived and handed her the needles. She quickly opened the parcel and looked at the plastic wrapper to realize the needles were "Made in India"!


Well, the needles flew all the way from India to UK and back!

Sunday, December 18, 2011

The Principles!

Robustness Principle: "be conservative in what you do, be liberal in what you accept from others".

This principle encourages excellence in design and maximizes interoperability. It's practical meaning is:
  • when designing software, be careful to be compliant with the protocols specified in the RFCs and STDs (Internet-related standards documents).
  • do not introduce features or proprietary extensions to protocols that will be incompatible with other products or systems that are compliant.
  • design your software to be tolerant of products that may not be completely compliant with Internet, Unix and Linux standards.
  • where the standards are vague, accept as wide a range of reasonable operation as possible.
The principle is still honored today and is frequently mentioned in modern day Internet standards documents (RFC 2015 is just one example). Compatibility is a tradition in the world of the Internet, Unix and now Linux.

Douglas Engelbart's law of maturity: "The rate at which a person can mature is directly proportional to the embarrassment he can tolerate."

Excerpts from the book "The Engelbart Hypothesis: Dialogs with Douglas Engelbart"

"And then I’d realize, “Boy, that’s just the way I often sound.”When I was in the service I had time to think through a lot of things. I generated a sort of algorithm: the rate at which a person can mature is directly proportional to how much embarrassment he can tolerate. And I realized that embarrassment didn’t seem to bother me very much, because of my upbringing and the perspective I had about the world. Something Benjamin Franklin wrote was so beautiful, “You wouldn’t worry half so much about what other people thought about you if you realized how seldom they did,” and I’d say, “Oh, that’s right.”I seem to have a lot of intuitive capability. I just don’t mind at all not being able to explain to people how I reached something. It doesn’t bother me."

Principle of locality, also known as the principle of locality, is the phenomenon of the same value or related storage locations being frequently accessed. There are two basic types of reference locality. Temporal locality refers to the reuse of specific data and/or resources within relatively small time durations. Spatial locality refers to the use of data elements within relatively close storage locations.

Wednesday, December 14, 2011

New age and electronic music

New Age music is music of various styles intended to create artistic inspiration, relaxation, and optimism. It is used by listeners for yoga, massage, meditation and reading as a method of stress management or to create a peaceful atmosphere in their home or other environments, and is often associated with environmentalism and New Age spirituality.

'Beautiful' by Ryan Farish is pretty special. Listening to this album is like taking a tour through new age and electronic music. There’s some of Enigma’s arpeggio bells; a few chanting voices from Adiemus; some pleasant “whoosh” sounds from Jean Michelle Jarre; and them all topped off with some tastefully applied pan flute and simple Jim Brickman piano melodies. And yet, I think this is pretty special stuff. The compositions are nice and elegant; the melodies are simple but enriched by some intricate electronic noodling. It's really kind of intoxicating.

Enjoy 2 beautiful videos from this album.



Friday, December 9, 2011

The wifey confusion!

Friend 1: How is wify doing ?
Friend 2: do not know.. its data card...
its uses different network... something like UMTS or HDPS
Friend 1: hahahaha, wify means wife dear, not wi fi!
Friend 2: Oh!


You may also like: Thirst!, Software Humor

Thursday, December 8, 2011

Interesting times in Indian E-Retailing

These are interesting times for E-Commerce in India.

Companies are getting funded by the dozens, spending money on TVCs and burning investor money, distributing gift coupons as if it were cheaper that water, and what not. In short, e-commerce is catching up!

So, an e-commerce company X, famous for selling books online, recently started offering replacement guarantee for electronic products.

Company Y, another e-commerce player in electronics space, responded with:

In these fascinating times, Taggle.com shut shop, stating competition to be too cut-throat for building a profitable business.

One e-commerce website selling electronics and gadgets since inception, entered into another category - "Toys"!

A milestone day for India’s ecommerce industry was observed recently. While most of the sites are selling mobile, tablets, perfumes etc. Ebay went a step further and sold ‘milk giving black young buffalo 20 liter per day’.

Apart from free shipping offer (national courier), the deal also comes with 7-days exchange offer!


Deal websites are not just about deal anymore. They have entered mainstream e-retailing.

Very recently, Bollywood felt left out. Yes, Bollywood. After all, who would want to miss the e-commerce money making race.

Karishma Kapoor recently invested in an e-commerce startup that operates a retail website for kids.  Ajay Devgan launched an e-ticketing and film entertainment site Ticketplease.com in January this year. Other investors in this venture include Sanjay Dutt, Bollywood producer Nitin Manmohan, Vatsal Seth, Ramakant Tibrewal, Vijay Jain and Amit Sharma of Roha JP Group. Seth has also set up Celebwears.com with actor Sohail Khan. Surely, Indian celebrities are now reaching beyond endorsements and turning into full-fledged angel investors focused on start-ups.

Finally, not all is well. dearflipkart.blogspot.com will tell you why? Interesting name for a blog, isn't it? Read the article titled Dear Flip****, Your Customer Service Has Started Sucking. Shoppers are unhappy to the extent that they have dedicated a blog and websites to voice their frustration. 

And it is just the beginning!

Thirst!

I wake up in the middle of the night, thirsty. I drink water. It doesn't quench my thirst. I drink more, and more, and more. But the thirst refuses to be quenched, like a wildfire refusing to be doused by the gallons of water dropped over it by a helicopter.

It finally dawns on me: I'm thirsty for real, but I'm drinking water in my dreams. I sigh, relieved. I decide to wake up, take pick up the bottle, and quench my thirst for real this time.

-- From Rajbir Bhattacharjee blog

You may also like: BBA entrance interview
You may also like: The wifey confusion!

Sunday, December 4, 2011

NoSQL - Right tool for the right job!

First of all, the name: NoSQL is not "Never SQL" or "No To SQL". It is generally expanded to "Not only SQL".

There are a crazy number of softwares out there that are part of the 'NoSQL' movement. Some of the popular ones are listed below:

Tokyo Cabinet/Tyrant, Redis, Voldemort, Oracle BDB
CouchDB, MongoDb
Neo4J, InfoGrid, Infinite Graph
Cassandra, HBase, Riak
Oracle Coherence, db4o, ObjectStore, GemStone, Polar

NoSQL softwares can be categorized broadly into 4 categories:
  1. Key-value stores
    • Based on Amazon's dynamo paper
    • Collection of key-value pairs
    • Example: Voldemort,  Tokyo Cabinet
  2. BigTable clones
    • Based on Google's BigTable paper.
    • Big table, column families
    • Example: HBase, Hypertable,  Cassandra
  3. Document database
    • Inspired by Lotus Notes
    • Collection of key-value pairs
    • Example: MongoDB, CouchDB
  4. Graph Databases
    • Inspired by Euler and graph theory
    • Nodes, relations and key-value on both.
    • Neo4j, Sones, Allegrograph
NoSQL is primarily about scalability. Generally, when we talk about scalability, we think about the data size - from gigabytes to pentabytes and maybe more. There is another dimension to scalability - complexity of data. The two axes to scalability: Size and complexity.

Scaling size: Dealing with more and more information that is roughly similar in nature.
Scaling complexity: How do you deal with data that is more messy and more semi-structure. The categories above are mapped in the graph below.


As you can see, the focus of these softwares are very different.
Key-value stores: Simplistic data model that can massively scale.
Graph database: Capable of handling complex connected data; difficult to achieve horizontal scalability.

Typical applications:

Key-value store: Storing the customer shopping cart of a customer in an E-Commerce application. Since all lookups are based on key (customer id), and in most cases, ad-hoc query is not needed. This makes key-value an ideal storage for shopping cart data.

BigTable clones: You have a news site where any piece of content: articles, comments, author profiles, can be voted on and an optional comment supplied on the vote. You create one store per user and one store per piece of content, using a UUID as the key (generating one for each piece of content and user). The user's store holds every vote they have ever made while the content "bucket" contains a copy of every vote that has been made on the piece of content. Overnight you run a batch job to identify content that users have voted on, you generate a list of content for each user that has high votes but which they have not voted on. You then push this list of recommended articles into the user's "bucket".

Document database: Documents are semi-structured data. Modeling product attributes using RDBMS has always been a challenge. All products will have a common set of attributes e.g. title, sku, price and a category specific set of attributes. Document databases effortlessly solve this problem. MongoDB is one step away from MySQL providing ad-hoc queries, transactions, indexes and more.

Graph database: Great for cases where data is connected e.g. social applications like twitter, facebook and blogs.

Sunday, November 20, 2011

The daily deal nerve discovered!

Daily deals brought to you by companies such as Snapdeal, Crazeal etc., has brought “coupon-clipping” to an entirely new generation and it’s made it cool. It’s social. It’s a community of people working together if for nothing else but the simple joy of finding a spectacular deal. It joins the masses for the greater good. Bargain hunters unite!

Daily deal sites count on the fact that the limited time frame will pressure people into making a quick decision. You see that timer ticking (a clever visual pressure tactic) and there’s a limited number of the specific deal left, N number of people already bought it and maybe you want to be part of group that’s getting in on a great bargain. After all there’s nothing worse than feeling left out and possibly it’s something you’ll actually use…maybe. But it’s not long after that impulse shopping’s close relative, buyer’s remorse sets in.

Recently, an elderly relative visited me and showed his new Android phone. During our conversation, he mentioned he had been getting spam SMSes about deals, sometimes in the middle of the night. I offered to help him. I said I can install a SMS blocker on his phone and he can have his peace of mind. "No", he promptly replied, "sometimes the deals are interesting and I can't miss those!".

I was taken aback but I know how that feels. It is like “Shaadi ka laddoo – jo khaye pachtaye, jo na khaye pachtaye” or "Every coin has two sides".

So what is the way out? To have the cake and eat it too? Well, there are many preachy 'dos and donts' articles out there that help you decide on what deals to buy. And there is another way - don't succumb and subscribe to every daily deal website but have some friends in your group who do that and have their number in your phone's favourite list! After all - "har ek friend zaroori hota hai!".



Stunning Video of Earth from 240 Miles Up




Here's a beauiful space video: 18 time-lapse sequences of photos taken from the International Space Station, gorgeously processed and edited by Michael König.

It is like sitting in heaven and orchestrating the magic called Earth - the mesmerizing aurora borealis or the Northern and Southern Lights, the lightnings in the clouds, the cobwebs of settlement light stretching across continents, the blue oceans. A list of the locations being passed over are mentioned below.

Shooting locations in order of appearance:

1. Aurora Borealis Pass over the United States at Night
2. Aurora Borealis and eastern United States at Night
3. Aurora Australis from Madagascar to southwest of Australia
4. Aurora Australis south of Australia
5. Northwest coast of United States to Central South America at Night
6. Aurora Australis from the Southern to the Northern Pacific Ocean
7. Halfway around the World
8. Night Pass over Central Africa and the Middle East
9. Evening Pass over the Sahara Desert and the Middle East
10. Pass over Canada and Central United States at Night
11. Pass over Southern California to Hudson Bay
12. Islands in the Philippine Sea at Night
13. Pass over Eastern Asia to Philippine Sea and Guam
14. Views of the Mideast at Night
15. Night Pass over Mediterranean Sea
16. Aurora Borealis and the United States at Night
17. Aurora Australis over Indian Ocean
18. Eastern Europe to Southeastern Asia at Night           

Wednesday, November 9, 2011

Understanding the European Union Financial Crisis

If a picture is worth 1,000 words, a video is worth 10,000 words.



But why is Germany letting Greece default by cancelling 50% of latters debt?

Before Euro became a common currency across Europe, German exports stood at around $487 billion in 1995. In 1999, the first year of the Euro being used as a currency the exports were at Euro 469 billion. Next year they increased to Euro 548 billion. And now they stand at Euro 1 trillion.

Using Euro as a common currency took away the cost of dealing with multiple currencies and thus helped Germany expand its exports to its European neighbors big time. Also with a common currency at play, exchange rate fluctuations which play an important part in the export game, no longer mattered and what really mattered was the cost of production.

Since the beginning of the Euro in 1999, Germany has become some 30 per cent more productive than Greece. Very roughly, that means it costs 30 per cent more to produce the same amount of goods in Greece than in Germany. That is why Greece imports $64 billion and exports only $21 billion.

So the way it works is that German banks lend to other countries in Europe at low interest rates and they, in turn, buy German goods and services which are extremely competitively priced as well as of good quality. This, of course, helps Germany.

And that is why Germany is interested in rescuing these countries or at least showing that it is trying to do something about it. Because if these countries in Europe collapse, then German exports will collapse as well.

One solution bandied around is that these countries which are in severe debt to Germany should be asked to stop using the Euro as its currency. But if they stop using the Euro as a currency, then the huge export advantage which Germany has had because of the Euro will also end.

Moral of the story? Germany is jammed in from both sides!

How the whole thing happened explained in a picture:


Understanding the European crisis
 

Who will lose money?

More data and numbers?

http://www.guardian.co.uk/news/datablog/2011/nov/07/euro-debt-crisis-data

Sunday, November 6, 2011

Personalization and E-Commerce

Strong customer relationships are important to the success of an e-commerce business. That's why the e-commerce personalization features are so important.

I went to a neighborhood grocery store, of which I am a regular customer, and asked for a bottle of juice. He gave me a bottle of orange juice with lots of pulp, exactly the one I like.
You are standing in a digital electronics store, wondering which video game to buy for your son - Sony PS3 or XBox 360 or Wii? You are inclined to purchase Sony PS3 but minutes later, a store executive walks up to you and points to XBox 360 console and motivates you to give it a try. Nothing more is said and you walk out of the store with XBox 360 in your hand, passing by the smiling store executive.

These are just two forms of personalization out of many. Personalization is easy to achieve in the physical world. In the virtual world, it is a tad hard. There are no store executives to help, yet you must understand the preferences, needs and intentions of the shopper and help him transact. Sounds easy? I will say may be yes but mostly no.

Let's get to product recommendations, a very interesting form of personalization. The phrases that e-retailers use goes something like this:
  • Similar products
  • Recommended products
  • People who viewed this product also viewed these products
  • People who bought this product also bought these products and more
While searching for a dual SIM mobile phone, the following products are recommended for your consideration. If you have guessed by now, recommending relevant products is very important.
What is relevancy? To a shopper, it could mean related products around same price point, similar features and possibly a fast moving item.

Viewed product
Recommended products


This mobile phone costs Rs 2999 and the recommended mobiles are around the same price point. Since the shopper is looking for dual SIM phone, 2 dual SIM phones are recommended. All these 4 products are liked by the users as inferred from reviews and ratings.

Let us take another example where an attempted product recommendations goes bad.

Viewed product
Recommended products
 

Anomalies are easy to spot - Samsung laptop is similar to Sony PS2 and gaming laptop worth Rs 76,099 does not come close to Sony PS2 costing Rs 5690. And most interestingly, people who bought Sony PS2 also bought a geyser. That is very insightful!

Mostly, product recommendations are generated using rules or computer algorithms. If algorithm driven recommendations are not meaningful, consider manually linking related products or use paid service. It is a safer way to tread. Today's shoppers are smart and they can smell bad recommendations from miles away.

Product recommendations are very useful tools for personalization. If done correctly, it can significantly improve customer experience, conversion rate and repeat purchase.

There are many more decision to make before serving good recommendations. I will cover these in my future blog posts.

More blog posts on E-Commerce:
Social Validation and E-Commerce
Social Commerce: Part 3
Gamification

Monday, October 31, 2011

SOA learnings from Amazon and Facebook

I came across this interesting and very frank article written by Steve Yegge. You can read is here.

Having worked at Amazon for close to 4 years, I could not but agree with Steve's take on Amazon, Platform, SOA and 'Eat Your Own Dog food' concept. Thinking and designing in terms of services comes naturally to SDEs at Amazon.

It is interesting to understand what platform means. Platform is a term for technology that enables the creation of products and processes that support present or future development.

If we talk about software platform, it is a software program that makes services available to other software programs through Application Programming Interfaces (APIs).  e.g. the Java platform is a generic platform that runs programs written in the Java language.

Two notable characteristics about software platforms that I want to quote from the book "Invisible Engines":

Software platforms are at the heart of “ecosystems” that consist of mutually dependent communities of businesses and consumers that have a symbiotic relationship with the platform. 

"Software platform businesses have at least two sides. Software platforms consist of services that are often made available to developers through APIs. They are also made available to computer users, but those computer users typically avail themselves of API-based services by buying applications that in turn use APIs. It is only a slight exaggeration to say that all software platform makers all the time invest in getting both developers and users to use their platforms."

Look at the platform offerings from Amazon via Amazon Web Services(AWS) and Facebook. They provides developers with:

• Software services available through APIs
• Software development kits that facilitate writing applications
• A “developer zone” that provides, for example, access to tools, sample code, and technical support
• Member forums for online discussions with other developers

The AWS offerings now allows creating and hosting enterprise applications completely on the cloud.

Thinking in terms of platform and API allows great deal of innovation. In recent times with the advent of free web services, numerous mashups were created, which in turn created value for end users. Availability of Android OS and the 'apps' it can run has created a win-win situation for developers, hardware manufacturers and end-users.

A good indicator for creating a platform is when a big pieces of inter-dependent business functionalities needs to be realized and the future requirements are unknown or vague at best.

While deciding about creating a platform, it is important to have a clear vision. Some basic questions that needs to be answered are:
  • What problem in the platform solving?
  • Is the platform reusable? Can many clients benefit from it?
  • Who will use it and how?
And then more specific questions like what APIs should be provided and many more. Infact, creating and externalizing a software platform is similar to writing a business plan and then launching a startup! It is learning all the way.

Often while interacting with other software engineers/architects, I find it very difficult to explain the need to go with SOA or think in terms of platform . Software engineers are inherently opposed to the idea. Mainly because they have not seen and experienced the architecture before or 'fear' the additional complexity that comes with it. The most common excuse for not using web services I hear is that SOAP is heavy weight, that web services add complexity and even phrases like "too many moving parts" and "we don't need SOA now"!

Indeed, these are valid statements but the benefits of SOA far exceed these short comings. Companies like Amazon has clearly demonstrated it. Not only does SOA solve business problems, it lowers costs and create values for end customers. With low margins in E-Retailing, Amazon had to adopt it.

The biggest hurdle software developers have to cross is the ability to 'think' in terms of platform and to realize it.

More interesting articles on SOA:
Software design at it's best
SOA @ Amazon


Tuesday, October 25, 2011

Social validation and E-Commerce

I was browsing through the Books section of an Indian E-Commerce website, looking for a best seller or a book from a known author. Needless to say, there were many options and I was confused. So rather than looking at the story line and finding about the author, I started looking at what others wrote about the book, how many ratings it had and the average book ratings. Sounds familiar?

"When we are uncertain about what to do we will look to other people to guide us. And we do this automatically and unconsciously." says Professor Robert Chialdini.

It is a very common psychological phenomemon and it is called 'Social Validation'. More precisely, the social validation stems from the Bystander effect or Genovese syndrome, a social psychological phenomenon that refers to cases where individuals do not offer any means of help in an emergency situation to the victim when other people are present.

Why do we ignore? Put simply, it’s a matter of trying to fit in with the crowd. If everybody else is just watching and not doing anything, therefore, I should act like them to fit in with the social norm.

Only 5 to 10 percent of the population engages in behavior contrary to the social norm. Because we want to fit into these groups and maintain our membership with them, we conform our actions to the norm.

The bystander effect is a great way to show the power of social validation. We tend to do what we think is best to fit in with the crowd, regardless of the situation.
Another common place reasoning that suppors Social Validation: If everyone else is doing it, let’s just go with it too…besides, how can so many other smart, capable people be wrong?

So how does this apply to E-Commerce and online shopping? Look at the images below:








Do these images look familiar? Visual cues, textual cues and even stasticial numbers are used by online retailers to help shoppers take decision. Often times the social validation principle and the theory of scarcity (the illusion of diamond scarcity) go hand in hand. I will leave the details of which to a future blog.

In sum, social validation is a powerful phenomenon. In E Commerce, the cues works, provided each feature is implemented the right way.


You may be interested in other E-Commerce article:
Gamification
Personalization and E-Retailing

Friday, October 21, 2011

Ra.One, Shah Rukh Khan and Homeshop18

In yet another first of its kind marketing association, Ra.One the most awaited film of the year and Homeshop18, India’s largest virtual retail platform, came together for a unique partnership. Through this association Ra.One merchandise spearheaded by Shah Rukh Khan will get access to HomeShop18’s widest distribution network which will enable him to take the movie merchandise to the masses. As official shopping partner of Ra.One, HomeShop18 will sell the original movie merchandise through both its 24 hours home shopping channel and India’s fastest growing ecommerce portal, www.homeshop18.com.
Shah Rukh Khan takes movie marketing to an all new level in India, with a deep rooted movie merchandise strategy, enabling audiences from across the country to have a piece of the super hero. HomeShop18 offers unmatched value propositions on its products; free home delivery and cash on delivery, which makes it a smarter and convenient shopping platform.  

Bollywood’s biggest superhero, Shah Rukh Khan on the association with HomeShop18 said, “I would like everyone to watch the movie and bring home G.One. It gives me immense pleasure to collaborate with HomeShop18 as the association would help us reach out to the masses with our super-movie. Through this association, we are trying a new form of movie marketing in India, much like the practice in Hollywood. We want our audiences to not only get entertained by Ra.One but also get a chance to own a piece of the movie as they shop in the comfort of their homes”

Shah Rukh Khan further added “HomeShop18’s innovating retail setup is a significant boost to the movie industry and we look forward to partnering with them for our future projects”


Saturday, October 15, 2011

Software design at it's best!

Once upon a time, in one of the software projects, team A was collaborating with team B to develop a workflow system. The main workflow would invoke other smaller workflow - an orchestration system was the need of the day. A meeting was held and the agenda was to integrate the two workflows.

Day 1:
Team B: We have a web service and the only API 'pushMessage' should be called from your code.
Team A: Why remoting? We need a jar.
Team B: That is a very good question. The workflow is a system and not a library. We want to avoid code inter dependency. Code deployment and maintainability is easier. Service is reusable, and in this case, it encapsulates a complete workflow.
Team A: Generating web service client is pain ...(sorry but I don't remember the following words as I was rotfl; in my mind of course). We need a jar.
Team B: Generating web service client is trivial. It  takes about 15 mins. We can do it for you.
Team A: Ok, let us discuss this again tomorrow.

Day 2:
Team A: We have a great idea. Let us merge the code of these 2 separate projects. We will combine all the APIs and create 1 WSDL. It will run very fast.
Team B: (After recovering from the shock) These are 2 distinct functional components. The APIs have no relation to each other. There is a clear boundary between the 2 sub systems.
Team A: It is complex.
Team B: What is complex?
Team A: Making a service call from our code is complex.
Team B: Making a service call is complex? How?
Team A: Yes, we have to make this service call inside a transaction and it is complex.
Team B: What!! When did you decide to make a service call inside a database transaction?
Team A: Today. And we have a problem. It is complex.
Team B: What is complex?
Team A:  Making a service call from our code is complex.
Team B: Write 0 lines of code and you get code that has zero complexity .
Team A: Does writing 0 lines of code makes the code fast?
Team B: Yes, the code runs at infinite speed!
Team A: And can I apply encapsulation to the design?
Team B: All team members faint.

You make also like this humorous post on NoSQL vs RDBMS.

Saturday, October 8, 2011

He's asking if God exists

Atheist: You seem to believe that God exists. Then you should be able to show me God.

Theist: Sorry, I do not 'posses' God as if God were an object; so I can not 'show' God to you.

Atheist: Do you know what God is?

Theist: For me, this world is God in manifest form. When we see the world, we don't see God and when we don't see the world, we will see God. It is like water - when it is liquid, it is not ice and when it is ice, it is not liquid!

Atheist: But what exactly are you pursuing?

Theist: My belief, what sages have been saying for eons, is that the phenomenal world is not real; it is an illusion. One must not get entrapped by it; one must make the effort to get away from it's allure. So, I am not not 'pursuing' anything that I can show you. Rather, I am trying to get away from what is unreal in order to experience the real.

It is like seeing the reflection of the sun in water and believing it to be the sun. Only when you turn your eyes away from the water and towards the sky will you realize that the sun is up there. Or like believing that the image in the mirror is you and then turning away from the mirror to discover who you truly are.

 Atheist: But isn't that naive? What if you are chasing a mirage in your search for an imagined God?

Theist: A dear chases a mirage because it 'sees' water when it is thirsty. I am not 'seeing' anything in the distance. I am simply disillusioned by what I am experiencing with my body, mind and ego in the physical world. It is so evanescent and transitory, yet so engaging. My mission is to get away from it in the faith that it will lead me to something better, something real and absolutely true. It is when you are immersed in worldly ways, that you are chasing a mirage!

Atheist: I can't take your word for it. Give me some proof of God's existence.

Theist: When you ask for proof, you mean the 'triputi' of seer (you), the seen (God) and the process of seeing. But God does not fit into this kind of 'triputi'. The only way to get proof of God is by merging with God, by becoming God. It is like a salt doll wanting to experience the ocean. The moment it goes for a feel of the ocean, it is no more it's erstwhile limited existence. It becomes the ocean itself.

Atheist: Perhaps, but surely, there must be some other way of convincing me of God's existence.

Theist: I am afraid not. Faith is the only instrument that can give you a measure of God. Nothing else can work. Just like a micrometer and not a foot rule is the right instrument to measure the thickness of a sheet of paper.

Atheist: Okay. Suppose I grant all of what you are saying. But, just consider what happens in case you do not experience God. Would you not then repent for having missed out on what the world has to offer?

Theist: Absolutely not because this world does not charm me anymore. I have nothing to seek in the world of objects. So, there can be no question of 'missing out' on anything. I am distancing myself from it of my own accord.

Taken from "Speaking Tree" section of TOI.

Sunday, September 18, 2011

From a programmer with love: Collection of programming and software quotes

Being a programmer is not easy. It is a continous strife for perfection.
Programming is like an optimization problem - with the end goals of elegance and simplicity, reliability, user-friendly, reusability, workability, performant etc. You can't have the best of all.

Some easy ways to save your neck when QAs, Managers and client shouts "bugs"!

“My software never has bugs. It just develops random features.”
Any sufficiently advanced bug is indistinguishable from a feature. - Rich Kulawiec
It's not a bug - it's an undocumented feature. - Author Unknown
It's easy to cry "bug" when the truth is that you've got a complex system and sometimes it takes a while to get all the components to co-exist peacefully.  - Doug Vargas

Some classic programming and software quotable quotes:

Walking on water and developing software from a specification are easy if both are frozen.

Any good software engineer will tell you that a compiler and an interpreter are interchangeable. - Tim Berners-Lee

As a rule, software systems do not work well until they have been used, and have failed repeatedly, in real applications. - Dave Parnas

However, writing software without defects is not sufficient. In my experience, it is at least as difficult to write software that is safe - that is, software that behaves reasonably under adverse conditions. - Wietse Venema

I'm not saying we purposely introduced bugs or anything, but this is kind of a natural result of any complexities of software... that you can't fully test it. - Will Wright

Indeed, the woes of Software Engineering are not due to lack of tools, or proper management, but largely due to lack of sufficient technical competence. - Niklaus Wirth

Most of the effort in the software business goes into the maintenance of code that already exists. - Wietse Venema

People who are really serious about software should make their own hardware. - Alan Kay

Software is like entropy. It is difficult to grasp, weighs nothing, and obeys the second law of thermodynamics; i.e. it always increases. - Norman Ralph Augustine

Why shouldn't we give our teachers a license to obtain software, all software, any software, for nothing? Does anyone demand a licensing fee, each time a child is taught the alphabet? - William Gibson

Writing software that's safe even in the presence of bugs makes the challenge even more interesting. - Wietse Venema

Every good work of software starts by scratching a developer's personal itch.

Any fool can write code that a computer can understand. Good programmers write code that humans can understand. --Martin Fowler

Good code is its own best documentation. As you’re about to add a comment, ask yourself, ‘How can I improve the code so that this comment isn’t needed?’ Improve the code and then document it to make it even clearer. -Steve McConnell

There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies. The first method is far more difficult. -C.A.R. Hoare

Premature optimization is the root of all evil in programming. -C.A.R. Hoare

Inside every large program, there is a small program trying to get out. -C.A.R. Hoare

The key to performance is elegance, not battalions of special cases. -Jon Bentley and Doug McIlroy

Simplicity is prerequisite for reliability -Edsger W.Dijkstra

Correctness is clearly the prime quality. If a system does not do what it is supposed to do, then everything else about it matters little. -Bertrand Meyer

Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it. -Brian W. Kernighan

Always code as if the person who ends up maintaining your code will be a violent psychopath who knows where you live.

... programming requires more concentration than other activities. It's the reason programmers get upset about 'quick interruptions' - such interruptions are tantamount to asking a juggler to keep three balls in the air and hold your groceries at the same time."  -Steve McConnell

“Software is a great combination between artistry and engineering. When you finally get done and get to appreciate what you have done it is like a part of yourself that you've put together. I think a lot of the people here feel that way.” - Bill Gates

Saturday, September 10, 2011

Why Har ek friend jaroori hota hai - The new Airtel Advertisement

It is impossible to believe that any Indian would have missed watching this ad:


Catchy it is, pleasant to the ears, but soon due to its excessivity,  like the flood gates have been opened, it starts hurting your ears. I fear watching television these days, for I do not know when this megaton bomb would be dropped. My hand automatically reaches for the mute button on the remote to spare my brain of this incessant pain.

It looks like Airtel has made it the national anthem of India and it won't spare any Indian till each one of us has learned the lyrics by heart.

Chai Ke Liye Jaise Toast Hota Hai
Vaise Har Ek Friend Zaroori Hota Hai
Yeh sun sun ke bheja roast hota hai
Har Ek Friend Zaroori Hota Hai

With the success of Facebook, I think the song will be next picked up by them to promote the social networking website. "Facebook...Kyonki har ek friend zaroori hota hai!"

No wonder Airtel is soon going to increase the mobile phone tariff by 20% soon. I guess it is time for them to pay the broadcasters.

Till this ad goes off air, I have decided to watch my TV in mute :)

You may also like this article on my blog: The new Sony Vaio Ad.

Friday, September 9, 2011

Messaging using JMS, ActiveMQ, Apache Camel

Event drive architecture (EDA) is a powerful architecture and quite useful in some contexts. It is very useful in cases where applications and systems transmit events among loosely coupled software components and services. Event-driven architecture can complement service-oriented architecture (SOA) because services can be activated by triggers fired on incoming events.

As an example, in an E-Commerce system, if a shopper places an order, an order confirmation email will be sent out. "Order" is an event produced by the Ordering system, and multiple processing engine can react to this event. One such processor can send out Order confirmation email to the buyer.

So as you can see, EDA puts events in the middle of the design. It is about recognizing business events and how to design them in terms of data modeling. A business event is something that happens (change of state) where your business has planned to react upon in a predefined way. A business event is represented by a message, but not all messages are representations of business events. Also, EDA-approach in an asynchronous communication style.

 The Java Message Service (JMS) API is a Java Message Oriented Middleware (MOM) API for sending messages between two or more clients. ActiveMQ is one of the popular JMS providers.

The next section discusses utilizing JMS, ActiveMQ and Apache Camel for designing a simple event based notification mechanism. I assume that you have some basic understanding of JMS.

Scenario: Say for an E-Commerce business, when an out of stock product becomes available, it is a significant event. When such an event is received, the product should be available for sale on the website. We will design a simple notification system that generates a 'Inventory Received' event when inventory is received for an out of stock items.

Event producer: Warehouse management system, Event: Inventory Received , Event processor: NotificationService

Some observations here about this use case:
  • Event producer/Caller is not aware of Event consumer/callee
  • Many events can happen per second
  • Order of events is not important
  • No state management is required 
  • Communication is asynchronous
 ActiveMQ provides JMS implementation along with many advanced features like Virtual destinations, Total ordering, selectors etc.

Apache Camel is a rule-based routing and mediation engine which provides a Java object-based implementation of the Enterprise Integration Patterns using an API to configure routing and mediation rules. Apache Camel uses URIs so that it can easily work directly with any kind of Transport or messaging model such as HTTP, ActiveMQ, JMS, JBI, SCA, MINA or CXF Bus API together with working with pluggable Data Format options.

1. Configure ActiveMQ to enable Virtual destinations. Start ActiveMQ broker.
2. Event is a simple bean called InventoryUpdate.


 public class InventoryUpdate implements Serializable {

	private static final long serialVersionUID = 1L;

	private long itemId;
	private int quantity;
	private Date updateDate;

  ... 

}

3. Event producer


public class InventoryUpdateProducer {

    private static String url = "failover:(tcp://localhost:61616)?maxReconnectDelay=5000";

    // Name of the queue we will be sending messages to
    private static String subject = "InventoryUpdate";

    public static void main(String[] args) throws JMSException {

        // Getting JMS connection from the server and starting it
        ConnectionFactory connectionFactory =
            new ActiveMQConnectionFactory(url);
        Connection connection = connectionFactory.createConnection();
        connection.start();

        // JMS messages are sent and received using a Session. We will
        // create here a non-transactional session object. If you want
        // to use transactions you should set the first parameter to 'true'
        Session session = connection.createSession(false,
            Session.AUTO_ACKNOWLEDGE);

        // Destination represents here our topic 'InventoryUpdate' on the
        // JMS server. You don't have to do anything special on the
        // server to create it, it will be created automatically.
        Destination destination = session.createTopic(subject);

        // MessageProducer is used for sending messages (as opposed
        // to MessageConsumer which is used for receiving them)
        MessageProducer producer = session.createProducer(destination);
        producer.setDeliveryMode(DeliveryMode.PERSISTENT);

        // We will send a InventoryUpdate message.
        Random randomGenerator = new Random();
        InventoryUpdate iUpdate = new InventoryUpdate(randomGenerator.nextInt(10), 100, new Date());
        ObjectMessage objMessage = session.createObjectMessage(iUpdate);

        // Here we are sending the message!
        producer.send(objMessage);
        System.out.println("Sent message '" + objMessage + "'" + iUpdate);

        connection.close();
    }
}

4. Now let us look at the consumer side. A consumer can be a simple JMS client consuming from a Queue - "VirtualTopicConsumers.ConsumerB.InventoryUpdate" or we can use the power of Apache Camel to help us integrate by hiding the JMS details from the client. Remember, we are using Virtual Topic here.


public class InventoryUpdateConsumer {

    // URL of the JMS server
    private static String url = "failover:(tcp://localhost:61616)?maxReconnectDelay=5000";    // Name of the queue we will receive messages from
    private static String subject = "VirtualTopicConsumers.ConsumerB.InventoryUpdate";

    public static void main(String[] args) throws JMSException {
        
        // Getting JMS connection from the server
        ConnectionFactory connectionFactory
            = new ActiveMQConnectionFactory(url);
        Connection connection = connectionFactory.createConnection();
        connection.start();

        // Creating session for seding messages
        Session session = connection.createSession(false,
            Session.AUTO_ACKNOWLEDGE);

        // Getting the queue 'InventoryUpdate'
        Destination destination = session.createQueue(subject);

        // MessageConsumer is used for receiving (consuming) messages
        MessageConsumer consumer = session.createConsumer(destination);

        // Here we receive the message.
        // By default this call is blocking, which means it will wait
        // for a message to arrive on the queue.
        Message message = consumer.receive();

        // There are many types of Message and TextMessage
        // is just one of them. Producer sent us a TextMessage
        // so we must cast to it to get access to its .getText()
        // method.
        if (message instanceof ObjectMessage){
        	InventoryUpdate update = (InventoryUpdate) ((ObjectMessage) message).getObject();
        	System.out.println(update);
        }

        System.out.println("Done");
        connection.close();
    }
}

Ofcourse, we can also use camel template for publishing InventoryUpdate events to the topic instead of using JMS client directly as described in step 3 above.

Now, let us start by configuring Apache Camel to receive the message and process it.

i. Configure camelContext.xml


 <beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:camel="http://camel.apache.org/schema/spring"
	xmlns:broker="http://activemq.apache.org/schema/core"
	xsi:schemaLocation="
       http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
       http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd
       http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core-5.5.0.xsd">

	<!-- If a bean is defined in Spring XML or scanned using the Spring 2.5
		component scanning mechanism and a <camelContext> is used or a CamelBeanPostProcessor
		then we process a number of Camel annotations to do various things such as
		injecting resources or producing, consuming or routing messages. -->
	<!-- lets configure the Camel ActiveMQ to use the ActiveMQ broker declared
		above -->
	<bean id="main-broker" class="org.apache.activemq.camel.component.ActiveMQComponent">
		<property name="brokerURL" value="tcp://localhost:61616" />
	</bean>

	<bean id="jmsToFileRoute" class="com.JmsToEmailRoute" />

	<camel:camelContext id="camelContext">
		<camel:routeBuilder ref="jmsToFileRoute" />
	</camel:camelContext>
</beans>

ii. Define the route

import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.processor.idempotent.jdbc.JdbcMessageIdRepository;

import com.mchange.v2.c3p0.ComboPooledDataSource;

public class JmsToEmailRoute extends RouteBuilder {

	@Override
	public void configure() throws Exception {

		ComboPooledDataSource dataSource = new ComboPooledDataSource();
		dataSource.setDriverClass("com.mysql.jdbc.Driver");
		dataSource.setJdbcUrl( "jdbc:mysql://localhost:3306/test" );
		dataSource.setUser("root");
		dataSource.setPassword("");

		from("activemq:queue:VirtualTopicConsumers.ConsumerA.InventoryUpdate")
				.idempotentConsumer(
						bean(MessageProcessor.class, "getId"),
						JdbcMessageIdRepository.jpaMessageIdRepository(
								dataSource, "notificationProcessor"))
				.bean(MessageProcessor.class,
						"process(InventoryUpdate)");
	}

}

iii. Define message processor.


import InventoryUpdate;

public class MessageProcessor {
	public void process(InventoryUpdate message){
		System.out.println("Got message: " + message);
	}

	public long getId(InventoryUpdate message){
		return message.getItemId();
	}
}


Explanation: If you look at  JmsToEmailRoute, it is defining a route from Queue "VirtualTopicConsumers.ConsumerA.InventoryUpdate" to the processor bean "MessageProcessor". As such, Camel will invoke "process(InventoryUpdate message)" method for every InventoryUpdate event published to the topic "InventoryUpdate".

One important consideration here is that of Idempotence i.e. how will your application react to duplicate messages? Duplicate messages can be delivered for reasons such as buggy producer, broker crashing just before the acknowledgement etc.

There are couple of ways to solve this problem. One way is to use JMS or JTA/XA transactions. This is relatively slow as 2 phase commits and multiple flush to disks are involved.

The other way is duplicate detection and caching of messages. Here is the pseudocode



[WebMethod]

ResponseMessage DoRequest(RequestMessage)
{
	Extract the request identifier from the request.

	Check if the request has already been processed by querying the persistent log.

	If (request processed) then 
		return response
	Else
	{
                ITransaction trans = CreateTransaction()

                ProcessMessage(RequestMessage, trans);
 
                //Store the request identifier the persistent log or database
                UpdateLog(request identifier, trans)

                trans.Commit();

                Return responsemessage;
	}
}


Camel provides support for Idempotent Consumer. Look at the route definition again:


//Use C3P0 connection pooling 
ComboPooledDataSource dataSource = new ComboPooledDataSource();
dataSource.setDriverClass("com.mysql.jdbc.Driver");
dataSource.setJdbcUrl( "jdbc:mysql://localhost:3306/test" );
dataSource.setUser("root");
dataSource.setPassword("");

from("activemq:queue:VirtualTopicConsumers.ConsumerA.InventoryUpdate")
	.idempotentConsumer(
			bean(MessageProcessor.class, "getId"),
			JdbcMessageIdRepository.jpaMessageIdRepository(
				dataSource, "notificationProcessor"))
				.bean(MessageProcessor.class,
					"process(InventoryUpdate)");


Here, the bean method  MessageProcessor.getId() is invoked to get the request identifier. ' JdbcMessageIdRepository.jpaMessageIdRepository ' checks this identifier against the database. If no existing identifier is found, it stores the identifier in the database and invokes  'process(InventoryUpdate)' method.

iv. Finally, start the consumer.
 
public class TestMessageProducer {

	public static void main(String[] args) {

		//Talks to InventoryUpdateProducer
		ConfigurableApplicationContext context = new ClassPathXmlApplicationContext("camelContext.xml");
        }
}

Happy messaging!

Sunday, August 21, 2011

BBA Entrance Interview

My cousin recently appeared for her BBA interview. The interviewer asked her "Do you read the newspaper?". "Yes", my cousin replied. The interviewer asked an expected follow up question "Which sections of the newspaper do you read?" to which my cousin candidly replied "Actually my father once told me that if the Sensex reaches 20,000 points, he will buy me a laptop. So I check the business Sensex section daily!"

No prizes for guessing, she was selected!

You may also like: Thirst

Friday, August 19, 2011

Bangalore meri jaan!

Traffic humor @ Bangalore






You may also like: Humor: Software

SOA Technology @Amazon

Amazon and Service Oriented Architecture (SOA)

The excerpt below comes from the 2010 Shareholder Letter by Amazon CEO Jeff Bezos. You can read more about other technologies at Amazon here.

"Our technologies are almost exclusively implemented as services: bits of logic that encapsulate the data they operate on and provide hardened interfaces as the only way to access their functionality. This approach reduces side effects and allows services to evolve at their own pace without impacting the other components of the overall system. Service-oriented architecture -- or SOA -- is the fundamental building abstraction for Amazon technologies. Thanks to a thoughtful and far-sighted team of engineers and architects, this approach was applied at Amazon long before SOA became a buzzword in the industry. Our e-commerce platform is composed of a federation of hundreds of software services that work in concert to deliver functionality ranging from recommendations to order fulfillment to inventory tracking. For example, to construct a product detail page for a customer visiting Amazon.com, our software calls on between 200 and 300 services to present a highly personalized experience for that customer."


Continuing from the excerpt above, of course there are challenges with the SOA approach. If 200+ services are called to render a single page, dependencies and handling failures becomes a critical aspect. A critical service failure can directly impact customer experience and revenue loss. Hence the need for software to monitor hardware and software in real time and  generate alerts. Processes need to be set to respond to such events.

Another important aspect is meeting the SLA (service level agreements) for a page. The slowest service will definitely impact the overall load time of the web page. But it also depends on the overall rendering framework. To keep the overall latency low, the services have to be categories based on customer impact, availability and bunch of other factors. Each service will have to meet the SLA defined for it's band.

Meeting SLA for critical services requires sound engineering. Right from understand the use cases, to designing a clean API, designing caching mechanism, using the right hardware and software for the job, handling peak traffic (throttling), capacity planning and monitoring are some of the things to consider even before starting on code.

Another point to consider is the impact of service upgrades on clients. Changing the service interface is not trivial and often requires coordination among multiple teams. More than 1 version of the service needs to be deployed to avoid breaking current systems.

With many services around, service discoverability is difficult. A centralized service registry can help here. To access sensitive data, authentication and authorization needs to be in place. Most of it should be transparent to a service developer.

In short, SOA has some great benefits but it comes with it's own set of baggages. SOA needs an ecosystem of software and processes to reap benefits from it.

You may also be interested in this article: SOA learnings from Amazon

Friday, July 29, 2011

Social Commerce: Part 3 : The case for and against Facebook

This is the third article in the series 'The case for and against Facebook'

While the previous strategies can be labeled as "Direct Sales", using 'Indirect sales', a retailer can create more value. And all of it can be achieved right from your fans and the fan page.

i. Using Facebook, it is easy to extract trends, identify influencers, monitor competition and glean customer insights. Here is an example of a survey conducted by an e-retailer:








Simple questions like these can provide insights around what strikes a chord with your target audience.

ii. What more? If you boast of your website's fan base, build a detailed profile of your fans using their interests, posts and other information and feed it to your CRM system. Track customer's interaction history on fan page, learn from their kudos and complaints and they will love your customer service. The important point to understand here is to move away from boasting about the number of ‘fans’ and ‘likes’ and instead explore actual consumer behavior.

iii. Another important 'scorecard' that can be derived from Facebook (and social media in general) is customer satisfaction data. It is a fast and inexpensive way to measure customer satisfaction.

With technologies like sentiment analysis and NLP (Natural language processing), you can capture:
  • Buzz - How much people are talking about your brand?
  • Net Sentiment - How positively they perceive your brand?
  • Passion Intensity - How emotionally charged their feelings are?

Hope you enjoyed this series. Feel free to share your ideas.

Previous posts in this series: Part 1 Part 2


Wednesday, July 20, 2011

Metrics and more


In my quest for quality open source software, I came across a library for "Capturing JVM and application-level metrics. So you know what's going on."

Meet Metrics @ https://github.com/codahale/metrics

One great feature about this library is it can be easily integrated with Graphite. Graphite is a a highly scalable real-time graphing system.

A presentation about the library can be viewed here. I must say it is a great presentation. A memorable line from the presentation:

"Business value is anything which makes people more likely to give us money. We want to generate more business value." 

The library is actually very simple to use. Some code snippets to help you get started:



public class EServiceImpl implements EService {

//Measure the # of times the API is invoked
private final MeterMetric eMetric = Metrics.newMeter(
 getClass(), "Send API", "reqs", TimeUnit.SECONDS);

//Measure the rate of event i.e. req/ sec.
private final CounterMetric eCounter = Metrics.newCounter(getClass(),
 "ECounter");

ConsoleReporter.enable(10, TimeUnit.SECONDS) //Print metrics to STDERR every 10s

public void service1(){
        eCounter.inc();//Increase counter by 1
        .........
        eMetric.mark();//Mark the occurrence of an event.
}


Output on the console:

7/20/11 10:41:06 AM =============================================================
com.impl.EServiceImpl:

  ECounter:
    count = 10

  Send API:
             count = 10
         mean rate = 5.00 reqs/s
     1-minute rate = 2.50 reqs/s
     5-minute rate = 0.50 reqs/s
    15-minute rate = 0.16 reqs/s

If you're writing a Servlet-based web service, you can add MetricsServlet from the metrics-servlet subproject to an internally-accessible context. Add the following lines in web.xml.



<servlet>
 <servlet-name>CodahaleMetrics</servlet-name>
 <servlet-class>com.yammer.metrics.reporting.MetricsServlet</servlet-class>
 <init-param>
  <param-name>metrics-uri</param-name>
  <param-value>/metrics</param-value>
 </init-param>
 <init-param>
  <param-name>ping-uri</param-name>
  <param-value>/ping</param-value>
 </init-param>
 <init-param>
  <param-name>healthcheck-uri</param-name>
  <param-value>/health</param-value>
 </init-param>
 <init-param>
  <param-name>threads-uri</param-name>
  <param-value>/threads</param-value>
 </init-param>
</servlet>

....

<servlet-mapping>
 <servlet-name>CodahaleMetrics</servlet-name>
 <url-pattern>/CodahaleMetrics/*</url-pattern>
</servlet-mapping>

Navigate to: http://hostname:port/WebAppName/CodahaleMetrics

The output metrics is dumped in JSON format. You can parse it and pass it to your favorite graphing library.

Note: All metrics are reported via JMX, which you can view using VisualVM or JConsole.

Documentation needs some work but overall, I would rate the project as Very Good.

Friday, July 15, 2011

The New Sony Vaio Advertisement

Have you seen the new Sony Vaio advertisement featuring Kareena Kapoor in a myriad of colors? Indeed, the background setup and the costumes in the ad are bright and vibrant and the music pleasing. It goes well with the tag line "More colors, more style".

But what caught my attention was the concept of the commercial. The concept is simple - wear a matching dress to go with your laptop's exterior or the other way around. It's like the Kohler commercial that was on air few years back. The words struck me - "So.. what can I do for you?..... Design a house around this."

The Kohler advertisement was simply amazing but Sony Vaio commercial made me feel pity about it. In the competitive laptop market, each company is trying to beat it's competitor. With no real USP, companies are now competing on trivials. It seems as if these companies are saying - Take 1 GB more RAM, or the new 15.555" wide screen laptop, or take more covers for your laptop. As if the women of our times don't have burden of choosing the right dress for the occassion, they have one more to make now.

It is not difficult to vision what is coming next, for tomorrow, they may come up with washable laptop covers, and God forbid, tie up with a detergent selling companies and offer 1 kg pack free with every purchase of laptop!

My point is innovation is important and there is no substitute for it. Cheap marketing gimmicks and creativity shown at the wrong place will not lead to happy customers and not drive sales. There is no value creation in trifles.

The Sony Vaio Commercial:



The Kohler commercial:

Tuesday, July 5, 2011

Trance music is bliss!

Perhaps the most ambiguous genre of dance music, trance could be described as a melodic, more or less free-form style of music, partially derived from house. While there is no strict definition for trance, songs of this genre are usually characterized as being accessible and having anthemic and epic and rifting and psychedelic qualities.

It is my favorite genre. Be it vocal or purely digital, I am crazy about it.

So what is trance? How is it different from Techno music?

1. Techno focuses on rhythm and beat.
2. Trance focuses on melody.
3. Techno is more repetitive, and can be blended with other tracks.
4. Trance is less repetitive, and is a stand-alone piece
5. Techno is pure energy
6. Trance has a building up of energy, much like a story with a climax
7. Techno was developed in Detroit in the 1980’s
8. Trance was developed in Europe (Germany) in the 1990’s
9. Techno is inspired dance music
10. Trance is inspired music dedicated to the listening mind!

The most popular DJs in the world often play Trance music. Popular among them is Paul Oakenfold (1998–1999), Sasha (2000), John Digweed (2001), Tiësto (2002–2004), Paul van Dyk (2005–2006), and Armin van Buuren (2007–2011). Nadia Ali was recently made the queen of trance because of her amazing voice!

Armin van Buuren is one of the most terrific DJ of all times. In 2007, 2008, 2009 and 2010 he was voted number one in DJ Magazine's annual top 100 list of the most popular DJs. You can experience his music in the video below:





And "For an Angel":


Impressed?