Forum Discussion

Ddib_147504's avatar
Ddib_147504
Icon for Nimbostratus rankNimbostratus
Mar 18, 2014

How to remove duplicate lines in file with TCL

Hi Team,

 

I want to remove duplicate files in a txt file with TCL, i opened the file in r and w in the same time and i did two foreach, but with no result.

 

thank you

 

3 Replies

  • i tested this code, but i dont know where is the probleme:

     

                                set in [open file.txt "r"]
                array set seen {}
                while {[gets $in line] != -1} {
                     blank lines should be printed as-is
                    if {[string length [string trim $line]] == 0} {
                        puts $line
                        continue
                    }
    
                     create the "key" for this line
                    regsub {\mPoint \d+} $line {} key
                    regsub {\mcolor=\w+} $key {} key
    
                     print the line only if the key is unique
                    if { ! [info exists seen($key)]} {
                        puts $line
                        set seen($key) true
                    }
                }
                close $in
  • Your code actually seems to work with a minor change:

     

    set in [open file.txt r]
    array set seen {}
    while {[gets $in line] != -1} {
         blank lines should be printed as-is
        if {[string length [string trim $line]] == 0} {
            puts $line
            continue
        }
    
         create the "key" for this line
        regsub {\mPoint \d+} $line {} key
        regsub {\mcolor=\w+} $key {} key
    
         print the line only if the key is unique
        if { ! [info exists seen($key)]} {
            puts $line
            set seen($key) true
        }
    }
    close $in

    If you didn't care about the order presented or the blank lines, you could simply use a list function:

     

    set file [open file.txt r]
    set read [lsort -unique [read $file]]
    foreach x $read { puts $x }
  • thank you for your answer Kevin, i tested your code, but the problem is not resolved, should i split the file into list?