remove-duplicates

[I X Code X 1]

Hello all,

I have a question about the remove-duplicates function.

I understand how it works, just want to ask a few things about it:

remove-duplicates sequence &key :from-end :test :test-not
:start :end :key


1. What exactly is the :test-not argument used for?

2. I wanted to use the remove-duplicates function to get rid of duplicate characters in a string. So I typed up something like this:

Code: Select all

 (remove-duplicates "heyyyy cooooool aaaaaa" :TEST #'CHAR-EQUAL :from-end t)

Now my intention was to get hey cool a, but of course this did not happen. I instead got hey coola. #\space is also being seen as a duplicate. My question is, is there anyway to say "remove-duplicates EXCEPT ... "? I assumed maybe this is what :test-not does, but I tried it out and got this error: Error: Invalid :TEST-NOT argument: #\Space



Thanks!

[ramarren]

1. What exactly is the :test-not argument used for?

The specification explains it here. In general the :test argument provides a two-argument predicate function which can be used to compare things. The :test-not argument has the same meaning, but the result of the predicate is reversed. The effect of providing both is undefined and likely an error.

My question is, is there anyway to say "remove-duplicates EXCEPT ... "

You can provide a predicate which always returns false when encountering a space:

Code: Select all

(remove-duplicates "heyyyy cooooool aaaaaa"
                   :test #'(lambda (x y)
                             (unless (eql x #\space)
                               (char-equal x y)))
                   :from-end t)

Which results in "hey col a", since obviously all #\o are duplicates.

The thing about REMOVE-DUPLICATES is that it works on fully general objects, which places limitations on its performance. If your objects are sortable or hashable it is generally better to use a custom function.

[edgar-rft]

[Ramarren was faster than me, he answered while I was still typing ... ]

The :test and :test-not keyword parameters are intended to be used as either :test or :test-not but not both at the same time. Altough it sometimes works [it's unspecified and therefore implementation dependent if an error will be signalled or not you give both] the result will usually be very confusing.

I think the main reason why there are both :test and :test-not is that Lisp originally was designed by mathematicians and testing numerical results often requires heavy complicated :test functions and often it's easier to write a :test-not math-test to exclude the opposite of the desired numbers.

The typical Lisp solution for your problem is to use an anonymous lambda-function as :test or :test-not argument, so both solutions shown below will work:

Code: Select all

(remove-duplicates "heyyyy cooooool aaaaaa"
                   :test #'(lambda (x y)
                             (and (char-equal x y)
                                  (char/= x #\space)
                                  (char/= y #\space))))

(remove-duplicates "heyyyy cooooool aaaaaa"
                   :test-not #'(lambda (x y)
                                 (or (char-not-equal x y)
                                     (and (char= x #\space)
                                          (char= y #\space)))))

The :from-end keyword parameter is not necessarily needed here and can be omitted.

More code examples to manipulate strings [subtype of sequence] can be found in:

• Cliki: SPLIT-SEQUENCE - download link for the code at the bottom of the page
• common-lisp.net: Alexandria, i.e. Section Sequences

- edgar

[I X Code X 1]

Okay, that all makes sense now. Thanks!

[findinglisp]

I think the main reason why there are both :test and :test-not is that Lisp originally was designed by mathematicians and testing numerical results often requires heavy complicated :test functions and often it's easier to write a :test-not math-test to exclude the opposite of the desired numbers.

If you had a function FOO, but not NOT-FOO, then if all you had was :TEST and wanted NOT-FOO, you'd have to define it specifically for this purpose or use a LAMBDA form to create it. Both of these work, but are a bit clumsy. :TEST-NOT makes this simple and clean.

Note that if you use :TEST-NOT a lot, then you might be better off defining NOT-FOO and using it with :TEST, for readability.