More Complex Predicates: allOddQ, allIdenticalQ, subsetQ

"Mr Wizard Todd", Manfred Plagmann, and Sophia Scheibe have done Mathematica users a nice favor by getting permission from McGraw-Hill to distribute licensed copies of David Wagner' s superb book, Power Programming in Mathematica. (Dropbox link: https://www.dropbox.com/s/j2dsyvptnxjd369/Wagner%20All%20Parts-RC.pdf)

A related post is How to See the Equivalence of Select and Cases.

Here are some nice examples of predicates from Wagner (re - written in my Prefix/Postfix dialect).Test a list to see if all of its entries are odd integers.

allOdd@aList_List:=Length@Select[aList,OddQ]==Length@aList

allOdd@{2,3,5,6}

False

allOdd@{3,5,7,9}

True

This predicate tests a List to see if its parts are identical. Count returns all Parts that are Equal to the First Part.

identicalListPartsQ@aList_List:=Count[aList, First@aList] == Length@aList

identicalListPartsQ@{a,b,c,4}

False

identicalListPartsQ@{a,a,a,a}

True

SameQ vs. Equal  and Testing for Lists

SubsetQ determines if its first argument is a subset of its second argument and returns True or False. Wagner's function did not include a List test and he notes on Equal vs.SameQ : "The use of === rather than == makes SubsetQ return False if either set1 or set2 is not a list. Try it with Equal."

I use the Head test: set1_List, which rejects a non-List argument before evaluation. And more importantly, SameQ should always be used to test non-numerical equivalence. In fact using Equal here can give screwy results in part because of its usage as the mathematical "equals" (=) in Mathematica's syntax for equations.

When using abbreviated operators, we should first determine their Precedence:

Precedence/@{Union,Equal,SameQ}

{300.,290.,290.}

Since Union is slightly stickier than SameQ, the Union will be performed before the SameQ test.

subsetQ[set1_,set2_]:=set1∪set2===set2

subsetQ[{a,b,c},{a,b,d}]

False

subsetQ[{a,b},{a,b,2}]

False

Union Sorts Its Result

Whoops! Wagner's function didn't work. Let's find out why:

subsetQ[{a,b},{a,b,2}]//Trace

{subsetQ[{a,b},{a,b,2}],{a,b}∪{a,b,2}==={a,b,2},{{a,b}∪{a,b,2},{2,a,b}},{2,a,b}==={a,b,2},False}

The fact that Union returns a sorted List fouls up the comparison. Let's try a fix with Sort:

Clear@subsetQ;subsetQ[set1_,set2_]:=set1∪set2===Sort@set2

And it works - you see in the last step Sort fixes the order.

subsetQ[{a,b},{a,b,2}]//Trace

{subsetQ[{a,b},{a,b,2}],{a,b}∪{a,b,2}===Sort[{a,b,2}],{{a,b}∪{a,b,2},{2,a,b}},{Sort[{a,b,2}],{2,a,b}},{2,a,b}==={2,a,b},True}

Greater (>) is a "non-Q" predicate so the last statement (lacking a return-suppressing semi-colon), returns True or False.

More on predicates: http://mathematica-guide.blogspot.com/2015/07/how-to-see-equivalence-of-select-and.html

Create Predicates to Test Strings vs Numerics: Test for File Extensions

Elsewhere I tell why predicates are important to use, show how to find all predicates in Mathematica including ones not suffixed with "Q", and create ones of your own to test numerical expressions. Here I cover predicates for testing Strings, and create some tests for file types, as judged by their extensions. Note that FileExtension does not capture the period before the extension.

Equal (==) and SameQ (===) are valid predicates for Strings as well as numerics, and work fine for simple applications.

In[206]:= uncusFileQ@fileName_String := FileExtension@fileName == "unc"

In[207]:= uncusFileQ@"afile.unc"

Out[207]= True

In[208]:= uncusFileQ@"afile.txt"

Out[208]= False

In[209]:= textFileQ@fileName_String := FileExtension@fileName == "txt"

In[215]:= textFileQ@"sampleFile.txt"

Out[215]= True

To test for two alternative file extensions is a little trickier. Since we are testing for String equivalence between String patterns, we use the general String predicate function StringMatchQ and Alternatives (|), not logical Or (||). An example is testing for HTML files, since their extensions come in .htm and .html flavors.

In[210]:= Clear@htmlFileQ;
htmlFileQ@fileName_String := StringMatchQ[FileExtension@fileName, "html" | "htm"]

In[212]:= htmlFileQ@"www.jognog.com/index.html"

Out[212]= True

In[213]:= htmlFileQ@"www.jognog.com/index.htm"

Out[213]= True

In[214]:= htmlFileQ@"www.jognog.com/sitemap.xml"

Out[214]= False

How It Works: DeleteDuplicates

Delete Duplicates

While Union is commonly used to select all unique elements from a List, including a set of Lists, DeleteDuplicates is commonly used to select unique elements from a single List, which Union can do, too. Union sorts the result, while DeleteDuplicates leaves the result in its original order. Consequently, DeleteDuplicates is a faster function if you do not need the Sort. Both functions include an optional second argument to specify the function used to remove duplicates, which greatly increases their power and versatility. First, here is DeleteDuplicates' basic functionality.

DeleteDuplicates@{c,a,b,d,a,c,a,e,e,a,a,e}

{c,a,b,d,e}

Note that if you do feed DeleteDuplicates a set of Lists, you do need to enclose the Lists in curly brackets.

DeleteDuplicates[{c,a,b},{d,a,c},{a,e},{e,a},{a,e}]

DeleteDuplicates::argb: DeleteDuplicates called with 5 arguments; between 1 and 3 arguments are expected. >>

DeleteDuplicates[{{c,a,b},{d,a,c},{a,e},{e,a},{a,e}}]

{{c,a,b},{d,a,c},{a,e},{e,a}}


You can use DeleteDuplicates' second argument to increase its breadth by specifying how it will detect the duplicates. So in the example above, by default neither Union nor DeleteDuplicates treats Lists with the same elements as equivalent, as would be the case in set theory, while this can be done with their sameness test.

It is relatively straightforward to construct the second argument if you keep in mind that the default is DeleteDuplicates[expr, SameQ] and therefore extensions of the function can take the form DeleteDuplicates[expr, f@#~SameQ~f#2&], where the comparison function f can be as complex as you wish. Here we need Sort because:

{a,b}~SameQ~{b,a}

False

DeleteDuplicates[{{c,a,b},{d,a,c},{a,e},{e,a},{a,e}},Sort@#~SameQ~Sort@#2&]

{{c,a,b},{d,a,c},{a,e}}

Here is a second, neat example from the Doc Center. Extending the power of DeleteDuplicates, this function uses Equal instead of SameQ, possibly since Equal will yield True for Reals and non-Reals. I've modified the example to show that.

5==5.

True

5===5.

False

list1 = {{0,0,0,1,0},{1,0,1,0,1},{1.,1.,1.,0.,0.},{0,0,0,0,1},{1,1,1,0,1}};

DeleteDuplicates[list1,Total@#==Total@#2&]

{{0,0,0,1,0},{1,0,1,0,1},{1,1,1,0,1}}

Here is a potential issue that limits the power of DeleteDuplicates. Trace tells us that the second 4 gets removed before it can be compared to the 16.

squaresList=Table[{x,x^2},{x,2,4}]//Flatten

{2,4,3,9,4,16}

DeleteDuplicates[squaresList,#2==#^2&]//Trace
{{squaresList,{2,4,3,9,4,16}},DeleteDuplicates[{2,4,3,9,4,16},#2==#1^2&],{(#2==#1^2&)[2,4],4==2^2,{2^2,4},4==4,True},{(#2==#1^2&)[2,3],3==2^2,{2^2,4},3==4,False},{(#2==#1^2&)[2,9],9==2^2,{2^2,4},9==4,False},{(#2==#1^2&)[2,4],4==2^2,{2^2,4},4==4,True},{(#2==#1^2&)[2,16],16==2^2,{2^2,4},16==4,False},{(#2==#1^2&)[3,9],9==3^2,{3^2,9},9==9,True},{(#2==#1^2&)[3,4],4==3^2,{3^2,9},4==9,False},{(#2==#1^2&)[3,16],16==3^2,{3^2,9},16==9,False},{2,3,16}}

We're asking DeleteDuplicates to do something beyond deleting duplicates. We should use DeleteCases to do this job.

DeleteCases[squaresList,x_/;(Sqrt@x//IntegerQ)]

{2,3}

How It Works

Somewhere Roman Maeder gives a solution to deleting duplicates and implies that his solution is efficient. From memory here is the solution (with my more efficient syntax). We create a simple list of duplicate integers.

dupeList = Table[{i, i}, {i, 10}] // Flatten

{1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6, 7, 7, 8, 8, 9, 9, 10, 10}

We partition them into sets of two with offset 1 (meaning overlap of one on the original List).

dupeListPartitioned2Offset1 = Partition[dupeList, 2, 1]

{{1, 1}, {1, 2}, {2, 2}, {2, 3}, {3, 3}, {3, 4}, {4, 4}, {4, 5}, {5, 5}, {5, 6}, {6, 6}, {6, 7}, {7, 7}, {7, 8}, {8, 8}, {8, 9}, {9, 9}, {9, 10}, {10, 10}}

Now it's a simple matter to Select the sets where Part 1 is not the same as Part 2. Select always takes a predicate, sometimes of the form testQ, but here with an abbreviated operator, UnsameQ. Unequal would work for numerical entries, but UnsameQ will also work for symbols and Strings.

dupeListdupeSetsDeleted = Select[dupeListPartitioned2Offset1, #[[1]] =!= #[[2]] &]

{{1, 2}, {2, 3}, {3, 4}, {4, 5}, {5, 6}, {6, 7}, {7, 8}, {8, 9}, {9, 10}}

We're still left with duplicates. So we take the First entry of each set, and Append the Last entry of the Last set, which would have been left out. I remember thinking at this point that I would have spent time trying to not 'hack' this last part--somehow capture that last entry without another operation--but if it's good enough for Maeder, it's good enough for me. The lesson is to do what is expedient and move on to the next task.

Append[First /@ dupeListdupeSetsDeleted, Last@Last@dupeListdupeSetsDeleted]

{1, 2, 3, 4, 5, 6, 7, 8, 9, 10}

DeleteDuplicates

Predicates, Tests, and Test Patterns: How to Build Your Own

Build Your Own: User-defined Examples

Xiangdong Wen of Tech Support noted that the hard part of constructing Predicates is considering all the possible inputs and exceptions. To do a thorough job, we might need to do a Piecewise definition of a Predicate, or define a Predicate with Which, and if desired leave an error message as the final condition at the bottom to catch any exceptions that escaped us.

Limit a Built-in Predicate: lessThanOneQ

lessThanOneQ@x_ := If[ x1 < 1, True, False]

testSet = Range[0, 1.5, .3]

{0., 0.3, 0.6, 0.9, 1.2, 1.5}

lessThanOneQ /@ testSet

{True, True, True, True, False, False}

Test for Specified Length: lengthTwoQ

Clear@lengthTwoQ; lengthTwoQ@x_ := If[Length@x == 2, True, False]

lengthTwoQ /@ {{x, y}, {y, x}, {x, y, z}}

{True, True, False}

Test for Meeting Boolean Combination of Conditions: BinaryQ

By setting the Attributes of our Predicate to Listable, we shortcut the need to use Map over a List.

ClearAll@BinaryQ; SetAttributes[BinaryQ, Listable];
BinaryQ@x_ := If[x === 1 || x === 0, True, False ]

BinaryQ@{0, 3, 1, 1., 2}

{True, False, True, False, False}

One can also use Boolean And (&&) similarly, or any combination of And and Or in a compound Expression.

Test for Membership in a Set: integerLessThan10Q

Here we make a Predicate from a test for an element's membership in a defined set.

integerLessThan10 = Range@10

{1, 2, 3, 4, 5, 6, 7, 8, 9, 10}

integerLessThan10Q@x_ := If[MemberQ[integerLessThan10, x], True, False]

integerLessThan10Q /@ {4, 4/5, Pi, 3, 11, 251/25}

{True, False, False, True, False, False}

More Examples: maxEqualQ, maxNearQ, valueAtQ, positionNearQ

Different Predicates can test for a variety of related conditions. These are functions I used to test properties of book bestseller lists but here I use random lists for illustration. The first one tests to see if the maximum value of two lists of numbers are the same.

maxEqualQ[x_List, y_List] := If[Max@x == Max@y, True, False]

list1 = RandomInteger[{0, 25}, 20] // Sort

{0, 2, 2, 4, 5, 6, 8, 8, 9, 9, 9, 11, 15, 16, 16, 17, 17, 19, 19, 19}

list2 = RandomInteger[{0, 25}, 20] // Sort

{1, 2, 5, 6, 6, 6, 9, 12, 13, 14, 17, 17, 18, 18, 23, 23, 24, 24, 25, 25}

maxEqualQ[list1, list2]

False

This tests to see if difference between the maximum of two lists is less than a specified magnitude.

maxNearQ[x_List, y_List, nearness_Integer] :=
 If[Abs[Max@x - Max@y] <= nearness, True, False]

maxNearQ[list1, list2, 10]

True

maxNearQ[list1, list2, 1]

False

This tests to see if the Position of the maxima of two lists is the same.

maxPositionEqualQ[x_List, y_List] :=
 If[Position[list1, Max@x] == Position[list2, Max@y], True, False]

maxPositionEqualQ[list1, list2]

False

Using Predicates in Select, Cases, and Related Filter Functions

As you look at these simple examples realize they are but a glimpse of the power of this type of usage that I mentioned at the start of this section. The Predicates, Conditions and Boolean conjunctions used in these filter functions can be arbitrarily complex, and as Mangone notes we are only restricted to anything that Mathematica can compute.

First we define a Predicate and use pure Function since those are idioms that Select uses, then we just use Condition in Cases, since that is the syntax that Cases uses. Consequently I recommend using the Q suffix when naming a Predicate used in Select. Note in the Cases example the additional restriction to Reals using a Head test.

Consider using Predicates in the related filter functions Count, Position, Tally, Gather, Sort, SortBy, etc.

Clear@lessThanOneQ; lessThanOneQ@x_ := If[ x < 1, True, False]

Select[Range[-0.2, 1.1, 0.2], lessThanOneQ]

{-0.2, 0., 0.2, 0.4, 0.6, 0.8}

Select[Range[-0.2, 1.1, 0.2], # < 1 &]

{-0.2, 0., 0.2, 0.4, 0.6, 0.8}

Function[x, x < 1] /@ Range[-0.2, 1.1, 0.2]

{True, True, True, True, True, True, False}

Cases[Range[-0.2, 1.1, 0.2], x_Real /; x < 1]

{-0.2, 0., 0.2, 0.4, 0.6, 0.8}

lessThanOnePositiveQ@x_ :=
 If[0 < x < 1 , True, False](* version restricted to positive numbers *)

lessThanOnePositiveQv2@x_ :=
 If[x < 1 && Positive@x, True, False] (* alternate version *)

Select[Range[-0.2, 1.1, 0.2], lessThanOnePositiveQ]

{0.2, 0.4, 0.6, 0.8}

Select[Range[-0.2, 1.1, 0.2], lessThanOnePositiveQv2]

{0.2, 0.4, 0.6, 0.8}

Predicate Using a Piecewise Function

Note the usage in the Named Pattern variable of a NumberQ predicate to allow both Reals and Integers. This example is intended to illustrate that a Piecewise-defined Predicate can include an unlimited number of orthogonal conditions.

Clear@pieceWiseElementQ; pieceWiseElementQ@number_?NumberQ :=
 Piecewise[{{False, number < -3}, { True, -3 < number < 0}, {False,
    0 <= number <= 6}, {True, number > 6}, {True, number === 4.5}}]

pieceWiseElementQ /@ {-3.00001, -0.0000000001, -4/5, 0, 0.000000001, 4/5, 4.5,
   6, 6.01, 9}

{False, True, True, False, False, False, False, False, True, True}

Predicates, Tests and Test Patterns: NumberQ, NumericQ, Equal, SameQ

NumberQ and NumericQ

NumberQ is more restrictive than NumericQ and seems to be True only if the evaluated expression is explicitly a number, while NumericQ tests for a "numeric quantity," which seems to mean that Mathematica tests each component to see if its fully evaluated form is a number. NumberQ probably also contains exceptions that should be considered "numeric," for instance all of the built-in numerical constants. More Predicates are implicit in built-in defined sets, such as Rationals and Reals (see below).

NumberQ@3.14159

True

NumberQ@Pi

False

NumericQ@Pi

True

NumberQ@Sqrt@2

False

Both Predicates will test a compound expression, but only NumericQ will test the fully evaluated form of each component. If any component evaluates False by NumberQ, the whole expression evaluates as False.

NumberQ[Sqrt@2 + 5]

False

NumericQ[Sqrt@2 + 5]

True

NumberQ@Sin@Sqrt@2

False

Since in the end the sine of the square root of 2 evaluates to a Real, NumericQ tests it as True; in effect it's testing 0.987766, while NumberQ cannot see the evaluated form.

Sin@Sqrt@2 // N

0.987766

NumericQ@Sin@Sqrt@2

True

Equal (==) and SameQ (===)

The distinction between these two Predicates can be confusing, but the basic idea is Equal tests for numeric equality while SameQ tests for syntactical equality. There are subtleties; here Mathematica compares Pi using its internal storage form.

N[Pi, 4] == N[Pi, 6]

True

However if we compare explicitly different numbers Equal will return False.

3.141 == 3.14159

False

SameQ will pick up syntactic differences that Equal misses.

Pi === N[Pi, 5]

False

Here is a nice example from a good introduction to Mathematica by Nancy Blachman and Colin Williams of using SameQ to test syntax in a paradigm for testing user input for a correctness (Mathematica, A Practical Introduction, 2nd ed., p 433, syntax modifications mine).

testUser1[question_, correctAnswer_] :=
 Module[{userAnswer = Input@question},
  If[userAnswer === correctAnswer, Print@"Correct!",
   Print["You said ", userAnswer, " but the correct answer is ",
    correctAnswer] ]
  ]


testUser1["What is Newton's second law of motion?", f = ma]

Correct!

It is easy to see how using Equal or StringMatchQ would give different tests for equality ("What is the square root of 2?") or an exact string match ("What is the largest continent?") in user input.

Part 3 will show how to build and use your own user-defined Predicates.