CareerCup

At high level, sort file data then print unique lines.
Sorting is the real problem here.Since it is a file which can be too big and memory overflow may occur if we put complete file in main memory and try to sort it.
I believe we need to use external R way merge sorting here. The way unix sort works.
Look like "sort -u" unix implementation would be the right solution here.

or u could use a hash table?

Use the getLine() method to retrieve the lines from the file and put them in a Hash table .... it will only contain the unique lines... Print the hash table contents

^v and Ajay Singh.. Yes, hashing looks much better option here (instead of sorting and then printing unique)

@ v and Ajay...can u explain more how would u do that?like what are the key and value pairs here?and wat is de hash function?

If file size is too big, do multiple scans of file. In first scan consider only lines starting with 'a', in 2nd scan with 'b'... and so on. or a variation of scanning method to ensure scanning is uniform.

If the lines are too big, one idea is to hash each line and store it in memory and then in second pass, very that the lines are actually equal.

Or maybe use a prefix tree? When a new line comes in traverse the prefix tree and see if you have the corresponding path in the tree, if not then it's a new line otherwise add the path in the tree.

Compared to hash it will save you some space.

Use hashing. If file size is too big, Use bloom filter.