对每一行比较以前行是否出现过即可. 是则删除, 否则保留.

If (you are femal) && (under 30 ) Then
company me to bar in this saturday night;
While ( talking, touching, ....);
you can get your answer
else
you have to do it by yourself
end

while (not eof (sourcefile))
set flag=0
readline(sourcefile) into tmpline
while( not eof(destinefile))
readline(destinefile) into tmp1line
if tmpline=tmp1line then
set flag=1
exit while
end if
move the cursor to the next line
end while
if flag=0 then
writeline(tmpline) into destinefile
end if
move the cursor to the next line
end while

#!/usr/bin/perl
open FILE, "theTestFile.txt" or die "can't open file";
while (<FILE>) { $a{$_}=1; }
for (keys %a) { print; }
close FILE;


PERL can be run on windows, MAC, unix or linux
if you have to use Java, check java's hash table class.
if you have to use C, check oreilly.com
it has a famous 'Algorithm in C'. Chapter 8 can be free downloaded
The hash table code is included.

You don't need to do any sorting. waste time.
Sorry for the English. I don't have Chinese in my pc
I have tried the code on our solaris server, it works
Cheers.

本文发表在 rolia.net 枫下论坛import Java.io.*;
class check_different {
public static void main (String args[]) {
System.out.println("Please enter the directory which your text file located in:");
StringBuffer stfDir = new StringBuffer();
While((char ch=(char)System.in.read())!='\n') { strDir.appendChar(ch); }
File dir = new File(strDir.toString());
File newFile = new File(dir, "Result.txt");
System.out.println("Please input your text file name:");
StringBuffer strFilename = new StringBuffer();
char ch = (char) System.in.read();
While ( ch != '\n' ) { strFilename.appendChar(ch); }
File readFrom = new File(dir,stfFilename.toString());
if(readFrom.isFile()&&readFrom.canRead()) {
RandomAccessFile rafFile = new RandomAccessFile(readFrom, "r");
RandomAccessFile resultFile = new RandomAccessFile(newFile, "rw");
String[] lineString = new String[]{};
StringBuffer strTemp = new StringBuffer();
while(rafFile.getFilePointer()<rafFile.length()) {
if (lineString.length()<2) {
lineString[0] = rafFile.readLine(); }
else {
boolean found = false;
for (int i=0; i<lineString.length(); i++) {
strTemp = rafFile.readLine();
if (lineString[i] == strTemp.toString()) {found = true;}
else { found=false; }
}
if (!found) {lineString[i+1] = strTemp.toString();
resultFile.writeBytes(strTemp.toString());}
}
}
rafFile.close();
resultFile.close();
}
else
System.out.println("File cann't be read!");
}
}更多精彩文章及讨论，请光临枫下论坛 rolia.net

open (IN, "input.txt");
open (OUT, ">output.txt");
my $line = "";
while (<IN>)
{
s/^\s+//g;

if ("$line" ne $_)
{
$line = $_;
print OUT "$line";
}
}
close (OUT);
close (IN);