planet.foss.in

VaibhaV Sharma (Dalfry): Barbeque this afternoon

2010-03-22T06:30:28+00:00

Barbeque this afternoon Originally uploaded by Dalfry Our first barbeque at home. Lots of yummy food, some friends and excellent weather. Recipe for a good time.

VaibhaV Sharma (Dalfry): Barbeque this afternoon

2010-03-22T06:30:28+00:00

Barbeque this afternoon Originally uploaded by Dalfry Our first barbeque at home. Lots of yummy food, some friends and excellent weather. Recipe for a good time.

VaibhaV Sharma (Dalfry): Barbeque at home

2010-03-22T00:30:21+00:00

Barbeque at home Originally uploaded by Dalfry Our first barbeque ever at home. Lots of yummy food ready on the skewer.

VaibhaV Sharma (Dalfry): Photo Shoot for shaadi.com

2010-03-21T19:30:24+00:00

Photo Shoot for shaadi.com Originally uploaded by Dalfry Matrimony photography. :p Ankit is my cousin.

Swaroop C H: An experiment to be Google-Free

2010-03-21T16:22:49+00:00

A series of incidents and thoughts led me to try an experiment – to be “100% Google Free”. This turned out to be a lot harder than I thought, and ended up admiring Google a lot, and at the same time, worried and curious about what they do with all that data they have.

First things first, since I no longer use Google’s Feedburner, please kindly update your RSS readers to use http://www.swaroopch.com/feed/ instead of the earlier Feedburner link. For those 140+ people who are subscribed via email, I have migrated to MailChimp (emails were also being sent by Feedburner earlier), so emails will continue to be delivered to you from this post onwards. You can subscribe or unsubscribe for email delivery on this page.

Back to the main topic… there were a few reasons that led me to this experiment:

Security : I have been bitten in the past due to flaws in certain intentional security designs by Google products, and this was a put-off for me.
Privacy : Google seems to be blind to the importance of privacy – the recent Google Buzz fiasco was just a confirmation of my fears. Sure, they did do a quick turnaround immediately because it was a publicly released product. What about the data they have and are using privately? For example:
- Did you realize that if you “log out” of Google, it doesn’t matter – they still associate your activities with your “log in” ID!
- In order to make “improvements” to their ads matching, they will use the searches you just made “for a few hours” to show ads if Google cannot find good ads for a random web page that you are visiting which is displaying Google ads.
- And what about the long-term damage they might be inflicting? Especially the kids? Are they getting a bit… arrogant? Is it just “good business”?
- The worst part is that user’s don’t care about their own privacy. They just won’t do anything about it.
- Before you stereotype me as a luddite, let me be clear – I do not think that online == unsafe (I trust LastPass with notes on my bank details), I am not against data collection (that’s the only way online businesses can improve their products), I am not against data mining (shopping recommendations FTW), and I am not against ad-supported business models (otherwise, freemium products will go away). It’s just that I am worried about Google’s intentions, so I’m moving on. Simple.
Profiling : I’m okay with them mining the data that I choose to make public, not my private data – that’s why I’m weary of Gmail. That’s why I’m weary of very-cool but data-mining note-taking tools such as Springpad.
Lock-in : In spite of good steps they have taken, I have still not been able to take a full backup of my email from Gmail, why is that, Google? I have tried Thunderbird, Evolution and Outlook to do a full IMAP sync, but the sync always fails. Their IMAP server just won’t let me download more than a hundred emails, and will completely error out after a few tries. If you know how to export a Gmail mailbox containing around 15 folders and 14,210 emails, please let me know. Even Backupify.com hasn’t been able to do it!
Longevity : I think I’ll prefer a for-fee business for my critical data – because I know the company will be around and it is in their interest to continuously improve the actual product vs. the profiling algorithms to show better ads. For example, I have shifted to Zoho Mail because they show NO ADS even in the free version! Their business model is simple – the basic product is forever free to use, you can pay for more storage and users.
Overaching : It’s handling of Arunachal Pradesh in Google Maps, it’s deal with the NSA, it’s handling of deletion of music blogs, etc. is just plain bad.
Boring : If you ask me, there is nothing interesting coming out of Google these days. They’re just expanding ways (Chrome, Android) to make people search more and buying up companies that can be potentially threatening. Even Buzz seems like a love child of Twitter and Facebook. Sure it might be a game changer because of Gmail’s user base, but seriously, what’s interesting or new about it?
Monoculture : A huge problem I have observed is that people are not willing to adopt any new tool unless it is by Google. And there is a current startup hype in India. I find this direct conflict amusing because if even an average techie in India (e.g. the common person in the service industry, for example) is not willing to give non-Google products a try, how exactly will people be open to use tools made by startups? Pop Quiz: “How many of you have started using a new tool or new service (esp. by a startup) in the past one year and have used it for more than 2 months?” I wanted to switch to give other companies and startups a try.

Phew. I think those were enough reasons to move away from Google, at least for a while.

And, boy, it has been tough. Let’s face it, it’s hard for companies to beat Google when Google makes slick products and gives it away for free.

Here is what my transition looks like:

Search – The funny thing is I used Google Search only in 2004-2005, started using Yahoo! Search since 2006, and have moved to Bing exclusively since the past 6 months. (free)
Analytics – Moved to Mint ($30) + Piwik (open source)
Reader – Moved to Tiny Tiny RSS (open source)
Feedburner – Moved to the default WordPress feed link + MailChimp for emails (freemium)
Google Apps – Moved to Zoho for Business ($5 per month)
Docs – Moved to Zoho Docs which turned out to be way more powerful (free)
GTalk – Stopped using IM, it was a distraction anyway. (zero)
Contacts – Exported from Google, stored only on iPhone (free)
Calendar – Zoho Calendar (free)
Google Groups – subscribe to RSS feeds of the group (free)
Maps – Since the map application on iPhone uses Google Maps, no alternative
Google Alerts – no alternative
Google Adsense – This is still a todo item, haven’t looked into it yet. I have heard about Komli, Chitika, etc. but yet to investigate.
Phone – My next phone is probably going to be an Android phone, looks like there is no alternative (I’m tired of having to use Windows just for iTunes, only because I have an iPhone)

As I’m sure you have deciphered, this took some installation of server-side software and some money to make this transition. These were the best alternatives that I came across that suited me.

So far I’ve been very happy about this experiment, because I got to discover and try out new tools and realized that there is so much more cool functionality available out there that I would have never discovered otherwise!

And at the same time, I admire Google even more now (from a startupper’s perspective) because they discovered a business model because of which they are able to give away so much functionality for free, and hence brought more people online.

Vinayak Hegde (vinayak): Into the Baltics

2010-03-21T10:05:21+00:00

After doing a longish stopover in Helsinki for three and a half days, I took the Linda Line Cruise into Tallin, Estonia – A distance of just 80 kms across the Baltic Sea. While planning the Europe trip, the Baltics were never part of the plan. Estonia, Latvia and Lithuania were part of the USSR before it disintegrated in 1991. As a young kid I saw this on TV, as the USSR broke into independent states. At that point, I was saddened (Remember Misha – the Russian Kids Magazine ? ) as it probably meant I would not get any of the cheap Russian books that I had grown accustomed to. Over the years, I had been following the news about new-born states as some fell into chaos and dictatorship and some prospered as they broke the shackles of communism and the USSR and adopted free-market policies. This was a great chance to visit them and see for myself. As I started reading about them, I got more and more interested and decided to visit them all for a brief while.

I traveled through the Baltics from the north to south before flying to the North of Norway (Tromso – north of the Arctic Circle). I traveled through the cities of Tallinn and Parnu (Estonia), Riga, Salaspils, Liepaja and Cesis (Latvia) and Vilnius and Trakai (Lithuania). Though the Baltic countries have shared history and close ties they are different in several ways from each other.

Estonia is a confident country that has wholeheartedly adopted new technologies and free market policies. I met several people in Estonia who were planning to start their own businesses or already running them. The adoption of mobile, wireless technologies and Internet was high. Random factoid – The founders of Skype were from Estonia. It also helps that Finland is just across the Baltic seas so there is quite a bit of expertise and investment coming from across the border. Economic activity was buzzing and people were cheerful, friendly and optimistic about the future. They were well informed about the environment – An example was an internet mobilized cleanup of the forests – one of the largest in the world. Also most people were atheists (A survey suggested that Estonia had the highest percentage of atheists). Estonia has just 1.3 million people but for a small country it has achieved a lot in the last 2 decades since independence.

Latvia has the most fabulous (and most populous) capital – Riga – amongst the Baltic states. When I was there, it’s economy was going through a severe recession as the real estate bubble (fueled somewhat by money from the UK) had burst and the GDP had fallen a staggering 20+ percentage points. About half of Latvia’s population lives in Riga – which is also considered as one of the unofficial Capital of the Baltics. Riga is relatively modern. Riga is a photographer’s delight with wonderful architecture – right from the medieval ages to the bland Soviet style buildings to the swanky steel and glass towers of the 21st Century. Some of the local economy is fueled by the tourism from the UK (Notoriously young Britons who come to the Riga for sex and booze). Undeniably, Riga has a dark edgy underside which it tries to hide. However the people are friendly (almost everyone speaks English) though the older generation seems to be somewhat nostalgic (and conflicted) about the Soviet Era.

Lithuania seems more influenced by Eastern Europe (It shares a border with Poland in the South) than the Scandinavian countries. It’s beautiful capital Vilnius is rich in culture but seems more provincial and backward when compared to rest of the Baltics. Vilnius had a sizable population of Jews before World War II and was called Jerusalem of the North. Few people speak English and getting around is not very easy. Like Latvia, Lithuania was in the midst of a severe recession and it was apparent even in the city. I passed several factories that had closed down and a car factory that had it’s huge temporary parking space full of cars as people could no longer afford to buy them. There were dilapidated buildings around the outskirts (Where tourists typically never visit) and people still lived in cramped soviet style apartment blocks. Lithuania is still dependent on Russia for large amount of trade and there is a sizable Russian population in the country which is also represented in Parliament.

Tarique Sani: Twitter Weekly Updates for 2010-03-21

2010-03-21T07:51:39+00:00

@diptesh14 I don't own a video camera so don't know much about them. in reply to diptesh14 #
SANIsoft: blog : Geolocating with Maxmind Geoip PHP API http://bit.ly/aibvyg #
Shapes I http://is.gd/aGEXO An old photo, nearly 6 years old… #
#nowplaying The Eagle And The Hawk by John Denver #
Another amusing search on my blog "maybe god wants us to meet a few bad people" #
"expressive practical physical giver" No prizes for guessing where I got that from

Read the rest of this entry »
Originally published at http://tariquesani.net/blog/. Please leave any comments there.

Swati Sani: Twitter Weekly Updates for 2010-03-21

2010-03-21T04:33:01+00:00

Watching Love aaj kal again. #
@anilchop LOL another tweetdeck user Tweetdeck does not show devnagri. See it here http://bit.ly/9bjpmM in reply to anilchop #
@azmishabana Since Javedsaab is not active on twiiter, I would be obliged if you shared this link with him. http://bit.ly/8s28R #
मैं होश में हूँ तो तेरा हूँ, दीवाना हूँ तो तेरा हूँ; हूँ राज़ अगर तो तेरा हूँ, अफसाना हूँ तो तेरा हूँ #खुसरो #

Read the rest of this entry »
Originally published at Swati Sani. Please leave any comments there.

VaibhaV Sharma (Dalfry): Curled up on a swing

2010-03-20T22:30:18+00:00

Curled up on a swing Originally uploaded by Dalfry Simi figured out that she can curl up and fit comfortably on the swing.

VaibhaV Sharma (Dalfry): Wheeee!

2010-03-20T22:30:18+00:00

Wheeee! Originally uploaded by Dalfry Simi's first time on a swing and she loves it. She is 7.5 months old already!

Philip Tellis (bluesmoon): WebDU

2010-03-18T23:57:39+00:00

Wow, it's been another couple of weeks that I haven't updated this blog. Most of my time's gone in travelling. I was at ConFoo last week, and have a pending blog post about that, though it will be on the YDN blog. Next month I'll spend some time in India and then in May I head over to Bondi beach in Australia for WebDU.

I'll be speaking about web performance there and will probably do a pub event as well. Stay posted for more details. In any event, if Sydney is where you're at, then go register for WebDU. Geoff and crew put on a great show last year and it only looks to be getting better.

Vinayak Hegde (vinayak): UNESCO World Heritage Site – Soumenlinna

2010-03-18T18:49:30+00:00

Soumenlinna (literally Finland’s Castle in Finnish) or Sveborg (Fortress of Svea in Sweden) is a UNESCO World Heritage site built on an a group of islands south-east of Helsinki in the Baltic sea. It is accessible by the water ferries that ply across the sea from Helsinki. The ramparts of the forts are star-shaped and consists of many fortifications made of large mounds of sand. (called Skansen in Swedish – literally meanings fortifications or fort walls).

The fortress itself is inhabited and there are a number of cafes and restaurants to cater to the gaggle of picnicking Helsinkians (Not Helsinkers !! however tempted you are to use that word !!). The fortress also has Finland’s smallest official beach just a few metres across (see below). Suomenlinna was used during the Second World War as one of Helsinki’s air surveillance centres and served as a garrison until 1972. Today it houses the Finnish Naval War Academy. It also houses the Helsinki open prison. A substantial part of the repairs to the walls, ramparts and buildings are carried out by convicts.

Interestingly, Soumenlinna was built by a Swedish King (when Finland was still a part of Sweden 250 years ago) to guard against Russia whose increasing maritime power in the Gulf of Finland made the Swedes uneasy. Soumenlinna contains a shipbuilding yard that is now defunct. The Soumenlinna Church was built originally in Greek Orthodox Style with five onion domes. Later it was converted into Lutheran Church (Most Finnish are Lutheran Christians). Today, the main dome of the church doubles up as a lighthouse. The light in the Lighthouse signals the morse code for the letter ‘H’ (for Helsinki) and is one of the first landmark for vessels approaching Helsinki.

Runa Bhattacharjee: runa

2010-03-18T18:13:05+00:00

The other day I was playing around my Fedora 12 box to check some widget alignments and came across this interesting ‘bug’ related to the updation of the default directories in the user’s home. The directories ‘Desktop’, ‘Downloads’, ‘Templates’, ‘Public’, ‘Documents’, ‘Music’ and ‘Pictures’ are automatically present in the user’s home directory and these names can be translated in the xdg-user-dirs module. If the xdg-user-dirs-gtk module is installed, everytime a user logs into a new language interface from the gdm a dialog is presented prompting the user if she would like to rename these directories to the translated version. If she chooses to rename, then after logging in she would get these folders in the language she chose for the current session. Next time, when the same or another user chooses a different language while logging into another session, the prompt reappears and the user can again choose to rename the folders into their choice of language for the session. Rinse repeat.

The catch here is that the translation of these folders have to be present for this dialog prompt to be displayed. In the earlier example, if there were no translations of these folders in user2’s choice of language, the dialog prompt would not have been displayed. This would result in user2 being stuck with (in all probability) incomprehensible folder names from user1’s session. The solution here is to revert back to the more conventionally accepted standard English names. The process of reverting involves, logging out from the session and logging into the English session, choose to rename the folders into English from the displayed dialog prompt and then logging back again into a session with the preferred choice of language.

A probable solution to avoid this situation, is perhaps to display the dialog prompt for languages that do not have translations, with an option to rename them back to English. The other probable solution can be, to automatically rename them to English if there are no translations. The latter is the standard procedure for untranslated portions of UI messages.

This is particularly important for languages that are written in non-latin scripts like the CJKI languages. Since the folders are actually moved, writing their names would become difficult from the console. On the other hand, if they choose to not translate then renaming them back to English would require an user to go through the hoops mentioned earlier.

Since blog is not a bug, so one exists here (would have helped around if I had the skills). I hope I am not missing any existing solutions that are already present for this issue. Thoughts?

Vinayak Hegde (vinayak): The Sibelius Monument and the Olympics Stadium

2010-03-17T16:51:17+00:00

Sibelius Monument

The Sibelius Monument (more info) is one of the must-see places in Helsinki. It is an interesting Abstract Art sculpture made by Eila Hiltunen to commemorate Jean Sibelius – A celebrated Finnish composer whose work was instrumental in the formation of the Finnish Identity. The Finlandia Concert Hall built by Alvar Alto is named after a one of his most famous compositions. The sculpture itself is made of acid-proof stainless steel pipes that are welded together individually to form the distinctive abstract shape.

The Helsinki Harbour

The Helsinki harbour is beautiful by night. The photo below (of a bridge and the power station) was taken while going to a Madonna concert that night.

Finland and Russia have a troubled past though relations are very cordial right now. Finland was a part of Russia for more than 100 years before it was annexed by Swedish Empire. Finland was attacked by Russia during World War II and some symbols of Russia in Helsinki (see photo below – the two-headed eagle in the Helsinki Harbour) still remind of this uneasy past. Russian Ships were docked in the harbour and heavily guarded when I visited it.

The Olympic Stadium

The Olympic Stadium in Helsinki (and the tall tower beside the stadium) are iconic. The stadium was built for hosting the 1940 Olympics which was canceled due to World War II but hosted the 1952 Olympics. The tall tower beside the stadium offers great views of the entire city. The Finnish football team was practicing when I visited it. On the board inside the stadium,there are quite a few recognizable names amongst those who have set records here – Marion Jones, Justin Gatlin (both Sprinters), Moses Kiptanui and Haile Gebrselassie (both have held several long distance running records). Outside the stadium, there is a statue of the “Flying Finn” – Paavo Nurmi – who has won 12 Olympic medals and is considered one of the greatest track and field athletes of all time.

Helsinki Cathedral

The Helsinki cathedral (or St. Nicholas’ Church) is the distinctive landmark in Helsinki with it’s green domes and and bright white exterior. It looks stunning standing on a top of a hill with stairs leading to it and is considered as the unofficial symbol of the Helsinki city.

Sankarshan Mukhopadhyay: GNUnify’10 etc

2010-03-17T09:22:37+00:00

I wasn’t able to spend too much time at GNUnify10 – a weekday came in and, then there wasn’t enough time to do anything. A couple of things did strike me though.

The profile of the attendees was different from last time. If the organizers talk about the demographics, this observation might be validated but I got the feeling that the crowd was more of professionals than students
The event has introduced diversity and, that is a good thing to have
There have been new speakers as opposed to the “same old faces” talking
There were elements of avid interest in things Fedora where the speakers were enthusiastic and participated in rapid Q/A

I liked what I saw. Including Shreyank’s enthusiasm to not get off-stage till the “download link came up”

One of the things that I’d hoped would happen at the event is that the group of Fedora folks who met would sit down and discuss their goals for this year. By goals I meant the stuff they would be focussing on and, more importantly, how they would be measuring their achievements. I don’t know how much was discussed along these lines but it is a good time to start doing it. Keeping the focus on a few important things and then creating easy-to-visualize ways of looking at the achievements allow the contributors to assess themselves. Self-assessment goes a long way in removing any perceptions of anonymity that might be lingering on. And, it also creates a sense of involvement – of belonging.

The other important bit this would achieve is that it would make the developers more visible and approach-able. For too long I have seen developers have an aloof or, stand-offish approach to their projects. And, it isn’t because they are arrogant but perhaps it is their trait. Unfortunately, “they will contribute if they figure out that the project is good” isn’t a nice approach. Going upfront and talking about goals, plans and in general doing advocacy allows potential contributors the confidence to tinker with the code and, start contributing. Building up the confidence to tinker not because it is “good for the nation” but because it is “good for oneself” and, is profitable is a concept that needs to be repeated over and over again. The students aren’t rolling up their sleeves enough and, it is an urgent need to exhort them to do it. The world is moving forward at a fairly fast clip and they cannot take comfort in the “learn-by-rote-to-join-TWITCH” way of life in the various colleges across the country. In money terms as well as in time and effort an enormous quantity is invested in students, that shouldn’t go to waste.

The others have already blogged about the event, I’m waiting for Hiemanshu’s writeup.

Posted from GScribble.

Sankarshan Mukhopadhyay: Rabindra Rachanabali and Bangla fonts

2010-03-17T07:41:58+00:00

The fonts that can be obtained from the site here display the following information. Now, how is one supposed to package (there isn’t a defined upstream as much as I could fathom) and redistribute (especially Bangla Akademi.ttf) them ? The fonts by themselves are fairly nice and, that’s a sad aspect as well.

And, an evaluation version of the BitRock Installbuilder seems to be used for creating the font installer.

$ otfinfo -i Vidya.ttf

Family: Vidya
Subfamily: Normal
Full name: Vidya
PostScript name: Vidya
Version: Version 0.6
Unique ID: PfaEdit : BanglaTemplate : 30-3-2003
Designer: NLTR
Manufacturer: NLTR
Copyright: Copyright NLTR License: GPL
version 2 (or later, at your option).

and,



$ otfinfo -i Bangla\ Akademi.ttf

Family:              Bangla Akademi

Subfamily:           Regular

Full name:           Bangla Akademi

PostScript name:     BanglaAkademi

Version:             1.0  2008 initial release

Unique ID:

SocietyforNaturalLanguageTechnologyResearch(SNLTR),Kolkata,India.DesignedaccordingtoPaschimBangaBanglaAkademiStandardbyBiswarupBhowmik:

Aangla Akademi: 2008

Description:         Society for Natural Language Technology Research

(SNLTR),Kolkata,India. Designed according to Paschim Banga Bangla

Akademi Standard by Biswarup Bhowmik, 24B Lake Road, Kolkata 700029

Designer:            Biswarup Bhowmik, 24B Lake Road, Kolkata 700029

Manufacturer:        Society for Natural Language Technology Research

(SNLTR),Kolkata,India.  Designed according to Paschim Banga Bangla

Akademi Standard by Biswarup Bhowmik

Trademark:           Bangla Akademi is a trademark of Society for

Natural Language Technology Research (SNLTR),Kolkata,India.

Designed according to Paschim Banga Bangla Akademi Standard by Biswarup Bhowmik.

Copyright: Copyright (c) 2008 by Society for Natural Language

Technology Research (SNLTR),Kolkata,India.  Designed according to

Paschim Banga Bangla Akademi Standard by Biswarup Bhowmik. All rights

reserved.

Tarique Sani: Life Updates!

2010-03-17T05:30:51+00:00

Was going through older posts searching where to people land up with the weird search terms that I so often keep seeing in the analytics screen I felt the urge to do an LJ style update post (Take that Twitter!! You just don’t make the cut).

Things which are dominating my free time? TV serials: Started watching LOST completed the 5 seasons and now up to sync with season 6, season 1 and 2 really held my attention but after that it was mostly watching something while I read thing. The serial which has really caught me up is Caprica. If you are a sci-fi buff or more specifically BSG then you should not miss it! Tells the story of 58 years before the War. Oh yes! You probably guessed it right I have picked up a WDTV.

The most recent books that I finished were “Raiders from the North” by Alex Rutherford, “Goddess of the Market” by Jennifer Burns and “Spectacular India” by Shobha De. The selection doesn’t really represent my likings but since I have gotten better reading glasses I am enjoying reading anything I can get my hands on

The bird photography has slackened a bit but last Sunday we drove around in the non-protected areas of the Bor wildlife Sanctuary, an almost 230km trip. No we did not see any big cats, though it would not have been unlikely we did see a good number of birds. A photo lifer for me was Green Heron. Meanwhile I have been updating my Flickr account fairly regularly. Have a conceptual photo-shoot planned with a Painter for tomorrow, let see how that goes.

Have been coding on a few itches – basically Wordpress plugins for better Flickr integration. Wordpress code is a nightmare but it gets the job done very well and that is what ultimately counts – Interested? Have written about them here

Originally published at http://tariquesani.net/blog/. Please leave any comments there.

Vinayak Hegde (vinayak): Cloudcamp Bangalore 2010 and Hadoop Summit

2010-03-16T15:27:38+00:00

The 2nd CloudCamp Bangalore was held at Dayanand sagar College of Engineering. It was co-located with the First Hadoop summit in India. The Hadoop summit was interesting and more relevant to me as I am using a Hadoop cluster for Analytics at Inmobi. Dave kicked off Cloudcamp with signature “unPanel”. I was on the Unpanel this time and answered some questions on mobiles, netbooks and smartphones as access devices for the cloud and the on impact of Google patent on MapReduce.

The corridor discussions with a bunch of Hadoop committers were insightful. I also found out more about Mahout. Mahout is a Apache project to build scalable machine learning libraries. It is not restricted to Hadoop implementations, but much of the current activity seems to be around Hadoop.

Notes and embedded slides from the sessions I attended follow:

Hadoop summit Keynote

Hadoop Summit 2010 Keynote

Data Management on Grid

Hadoop Summit 2010 Data Management On Grid

Notes:

Y! uses a HDFS replication factor of 3 (the hadoop default) in most cases. Exceptions are big clusters with large number of applications running simultaneously.
Y! does not use Avro yet due to large amount of legacy data. Twitter uses Avro.
Data ingestion layer uses MapReduce for heavy lifting and format conversion for storage.
LZO is used for compression. gzip (not ideal due to non-block-level indexing) and bzip2 is also used. There are problems with slowness of bzip2 decompression but bzip2 delivers better compression ratios.
Data ingestion layer also oversees policy for data retention and purging.
Underlying filesystems is rarely a bottleneck for Hadoop. Mostly the synchronization semantics of HDFS is a bottleneck. A file operation is not successful until all the replicas are in sync.

Machine Learning using Hadoop

Hadoop Summit 2010 Machine Learning Using Hadoop

Notes:

There are clear differences between data mining and machine learning.
ML is harder to implement efficiently on Hadoop. Improving efficiency is still a research problem.
Hadoop creates one map job / block creating too many empty files and also many reducers.

Optimizing and Benchmarking Hadoop

Hadoop Summit 2010 Benchmarking And Optimizing Hadoop

Notes:

As a thumb rule, adding as much memory as money can buy is a a good idea for Hadoop
Consider Network connections as shuffle stage does heavy network I/O
Solid state disks might make sense at certain price/performance ratios. They are also more power efficient.

Tuning Hadoop To Deliver Performance To Your Application

Hadoop Summit 2010 Tuning Hadoop To Deliver Performance To Your Application

Notes:

Several parameters to tune Hadoop but must be used in conjunction with each other.
Set number of map jobs slightly more than number of cores to ensure better utilization. Makes sure that data is processed in waves. Also better network utilization (as shuffle phase happens parallely with Map phase) along with CPU scheduling
Choosing a good HDFS block size is important. Number of HDFS blocks is directly proportional to number of Map tasks generated

planet.foss.in

VaibhaV Sharma (Dalfry): Barbeque this afternoon

VaibhaV Sharma (Dalfry): Barbeque this afternoon

VaibhaV Sharma (Dalfry): Barbeque at home

VaibhaV Sharma (Dalfry): Photo Shoot for shaadi.com

Swaroop C H: An experiment to be Google-Free

Vinayak Hegde (vinayak): Into the Baltics

Tarique Sani: Twitter Weekly Updates for 2010-03-21

Swati Sani: Twitter Weekly Updates for 2010-03-21

VaibhaV Sharma (Dalfry): Curled up on a swing

VaibhaV Sharma (Dalfry): Wheeee!

Philip Tellis (bluesmoon): WebDU

Vinayak Hegde (vinayak): UNESCO World Heritage Site – Soumenlinna

Runa Bhattacharjee: runa

Vinayak Hegde (vinayak): The Sibelius Monument and the Olympics Stadium

Sibelius Monument

The Helsinki Harbour

The Olympic Stadium

Helsinki Cathedral

Sankarshan Mukhopadhyay: GNUnify’10 etc

Sankarshan Mukhopadhyay: Rabindra Rachanabali and Bangla fonts

Tarique Sani: Life Updates!

Vinayak Hegde (vinayak): Cloudcamp Bangalore 2010 and Hadoop Summit

Hadoop summit Keynote

Data Management on Grid

Machine Learning using Hadoop

Optimizing and Benchmarking Hadoop

Tuning Hadoop To Deliver Performance To Your Application

Links to all presentations