Bug#37285902: Illegal mix of collations NONE vs. COERCIBLE

The problem is that an unexpected error message is reported from a
SELECT statement that involves multiple functions and columns and
strings with multiple collations. For details, see the analysis in
the bug report.

In essence, an IFNULL function returns a binary collation and collation
derivation NONE. The result of IFNULL is used in a LIKE clause.
Derivation NONE is disallowed in this context, thus an error is
reported.

This is behaviour according to documentation.

However, we can change the semantics slightly and thus accept the
problematic query.

The change is to lower the strength of collation derivation NONE to be
less than the strength of any other collation derivation. This means
that when an expression with derivation NONE is used e.g. in a LIKE
expression, then the other operand will determine the comparison
collation to be used by LIKE. The change can be used with minimal impact
on existing functionality, in fact it causes no changes in existing
test suite.

In addition, we simplify the collation determination slightly and
provide more comprehensive documentation.

We also change collation aggregation as implemented in
agg_item_collations() to only consider collations with the same (and
highest) strength when determining the result collation. This might
seem dangerous since it means that operands with possibly incompatible
character sets but lesser strength are now allowed. However we have
already implemented code that converts operands with different
character sets into the character set determined for the operation,
and report errors if strings are actually invalid, thus pushing some
error rejection from the resolver stage into the execution stage.

We also rename the previous IGNORABLE derivation to NULL, since it is
only used for NULL values, and since it is no longer ignorable given
that NONE now has lesser strength.

We leave a hole in the "strength" values for the former NONE strength,
in order that as few COERCIBILITY() calls as possible return values
that are incompatible with the previous implementation.

A special problem was hit with resolving of aggregated types for the
result columns of set operations. The new way of determining character
set and collation works fine when all aggregated items are processed
in one operation, but an anomaly was discovered for set operations,
since type aggregation is handled for pairs of items. We have therefore
implemented a new function Item_aggregate_type::unify_types() that
unifies the types from a set of items.

Query_term_set_op::prepare_query_term() has been refactored so that it
may call the new unify_types() function. It has also been changed so
that the type holder objects are created and finalized in one place,
which is just after all query term children have been resolved in a
non-recursive set operation, and just before resolving the first
recursive query term in a recursive set operation.

The actual preparation of type holder objects is performed in a new
function Query_term_set_op::prepare_type_holders().

Finally, an earlier mistake was discovered and corrected. The call to
report_conversion_error() inside Item::convert_charset() had
collation arguments reversed. Fixing this caused a few minor changes
in result test files.

Change-Id: Ie30906758078559fcba8ae881125e039bab9298a

Author: roylyseng

mysql-test/r/ctype_recoding.result

@@ -200,7 +200,7 @@ select * from t1 where a=concat(_koi8r'
 a
 ����
 select * from t1 where a=_latin1'����';
-ERROR HY000: Cannot convert string '\xD7\xC1\xD3\xD1' from cp1251 to latin1
+ERROR HY000: Cannot convert string '\xD7\xC1\xD3\xD1' from latin1 to cp1251
 drop table t1;
 set names latin1;
 set names ascii;
@@ -284,5 +284,5 @@ t1	CREATE TABLE `t1` (
 ) ENGINE=ENGINE DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci
 drop table t1;
 create table t1(a char character set latin1 default _cp1251 0xFF);
-ERROR HY000: Cannot convert string '\xFF' from latin1 to cp1251
+ERROR HY000: Cannot convert string '\xFF' from cp1251 to latin1
 End of 4.1 tests

mysql-test/r/ctype_utf8mb4.result

@@ -2341,9 +2341,9 @@ DROP TABLE t2;
 CREATE TABLE t1 (utf8mb4 VARCHAR(10) CHARACTER SET utf8mb4);
 INSERT INTO t1 VALUES (x'f48fbfbf');
 SELECT CONCAT(utf8mb4, _utf8mb3 '�') FROM t1;
-ERROR HY000: Cannot convert string '\xBF' from utf8mb4 to utf8mb3
+ERROR HY000: Cannot convert string '\xBF' from utf8mb3 to utf8mb4
 SELECT CONCAT('a', _utf8mb3 '�') FROM t1;
-ERROR HY000: Cannot convert string '\xBF' from utf8mb4 to utf8mb3
+ERROR HY000: Cannot convert string '\xBF' from utf8mb3 to utf8mb4
 DROP TABLE t1;
 #
 # Bug#51675 Server crashes on inserting 4 byte char.

mysql-test/r/ctype_utf8mb4_heap.result

@@ -2371,9 +2371,9 @@ DROP TABLE t2;
 CREATE TABLE t1 (utf8mb4 VARCHAR(10) CHARACTER SET utf8mb4) ENGINE heap;
 INSERT INTO t1 VALUES (x'f48fbfbf');
 SELECT CONCAT(utf8mb4, _utf8mb3 '�') FROM t1;
-ERROR HY000: Cannot convert string '\xBF' from utf8mb4 to utf8mb3
+ERROR HY000: Cannot convert string '\xBF' from utf8mb3 to utf8mb4
 SELECT CONCAT('a', _utf8mb3 '�') FROM t1;
-ERROR HY000: Cannot convert string '\xBF' from utf8mb4 to utf8mb3
+ERROR HY000: Cannot convert string '\xBF' from utf8mb3 to utf8mb4
 DROP TABLE t1;
 #
 # End of 5.5 tests

mysql-test/r/ctype_utf8mb4_innodb.result

@@ -2523,9 +2523,9 @@ DROP TABLE t2;
 CREATE TABLE t1 (utf8mb4 VARCHAR(10) CHARACTER SET utf8mb4) ENGINE InnoDB;
 INSERT INTO t1 VALUES (x'f48fbfbf');
 SELECT CONCAT(utf8mb4, _utf8mb3 '�') FROM t1;
-ERROR HY000: Cannot convert string '\xBF' from utf8mb4 to utf8mb3
+ERROR HY000: Cannot convert string '\xBF' from utf8mb3 to utf8mb4
 SELECT CONCAT('a', _utf8mb3 '�') FROM t1;
-ERROR HY000: Cannot convert string '\xBF' from utf8mb4 to utf8mb3
+ERROR HY000: Cannot convert string '\xBF' from utf8mb3 to utf8mb4
 DROP TABLE t1;
 #
 # End of 5.5 tests

mysql-test/r/ctype_utf8mb4_myisam.result

@@ -2527,9 +2527,9 @@ DROP TABLE t2;
 CREATE TABLE t1 (utf8mb4 VARCHAR(10) CHARACTER SET utf8mb4) ENGINE MyISAM;
 INSERT INTO t1 VALUES (x'f48fbfbf');
 SELECT CONCAT(utf8mb4, _utf8mb3 '�') FROM t1;
-ERROR HY000: Cannot convert string '\xBF' from utf8mb4 to utf8mb3
+ERROR HY000: Cannot convert string '\xBF' from utf8mb3 to utf8mb4
 SELECT CONCAT('a', _utf8mb3 '�') FROM t1;
-ERROR HY000: Cannot convert string '\xBF' from utf8mb4 to utf8mb3
+ERROR HY000: Cannot convert string '\xBF' from utf8mb3 to utf8mb4
 DROP TABLE t1;
 #
 # End of 5.5 tests

mysql-test/r/derived.result

@@ -4791,7 +4791,7 @@ SELECT * FROM (SELECT _utf8mb4 'a' AS a) AS dt UNION SELECT c FROM t;
 a
 a
 SELECT _utf8mb4 '±' AS a UNION SELECT c FROM t;
-ERROR HY000: Cannot convert string '\xC2\xB1' from utf8mb4 to latin2
+ERROR HY000: Cannot convert string '\xC3\x82\xC2\xB1' from utf8mb4 to latin2
 SELECT * FROM (SELECT _utf8mb4 '±' AS a) AS dt UNION SELECT c FROM t;
 ERROR HY000: Cannot convert string '\xC2\xB1' from utf8mb4 to latin2
 SELECT _utf8mb4 'a' AS a UNION SELECT d FROM t;

mysql-test/r/func_gconcat.result

@@ -499,7 +499,7 @@ select collation(group_concat(a,_koi8r'test')) from t1;
 collation(group_concat(a,_koi8r'test'))
 cp1250_general_ci
 select collation(group_concat(a,_koi8r 0xC1C2)) from t1;
-ERROR HY000: Cannot convert string '\xC1\xC2' from cp1250 to koi8r
+ERROR HY000: Cannot convert string '\xC1\xC2' from koi8r to cp1250
 select collation(group_concat(a,b)) from t1;
 ERROR HY000: Illegal mix of collations (cp1250_general_ci,IMPLICIT) and (koi8r_general_ci,IMPLICIT) for operation 'group_concat'
 drop table t1;

mysql-test/r/func_str.result

@@ -443,6 +443,107 @@ QUOTE('A')
 'A'
 'A'
 DROP TABLE t1;
+SELECT '1' COLLATE latin1_swedish_ci = '1' COLLATE latin1_swedish_ci;
+'1' COLLATE latin1_swedish_ci = '1' COLLATE latin1_swedish_ci
+1
+SELECT CONCAT('1' COLLATE latin1_swedish_ci, '2' COLLATE latin1_swedish_ci);
+CONCAT('1' COLLATE latin1_swedish_ci, '2' COLLATE latin1_swedish_ci)
+12
+SELECT '1' COLLATE latin1_swedish_ci = '1' COLLATE latin1_general_ci;
+ERROR HY000: Illegal mix of collations (latin1_swedish_ci,EXPLICIT) and (latin1_general_ci,EXPLICIT) for operation '='
+SELECT CONCAT('1' COLLATE latin1_swedish_ci, '2' COLLATE latin1_general_ci);
+ERROR HY000: Illegal mix of collations (latin1_swedish_ci,EXPLICIT) and (latin1_general_ci,EXPLICIT) for operation 'concat'
+SELECT '1' COLLATE latin1_swedish_ci = '1';
+'1' COLLATE latin1_swedish_ci = '1'
+1
+SELECT CONCAT('1' COLLATE latin1_swedish_ci, '2');
+CONCAT('1' COLLATE latin1_swedish_ci, '2')
+12
+CREATE TABLE t1(
+b0 VARBINARY(10),
+c0 VARCHAR(10) COLLATE utf8mb4_bin,
+c1 VARCHAR(10) COLLATE utf8mb4_0900_ai_ci,
+c2 VARCHAR(10) COLLATE utf8mb4_0900_as_cs,
+c3 VARCHAR(10) COLLATE utf8mb3_general_ci,
+c4 VARCHAR(10) COLLATE utf16_general_ci,
+c5 VARCHAR(10) COLLATE utf32_unicode_ci,
+c6 VARCHAR(10) COLLATE ascii_general_ci,
+c7 VARCHAR(10) COLLATE latin1_swedish_ci,
+c8 VARCHAR(10) COLLATE latin2_general_ci
+);
+Warnings:
+Warning	3778	'utf8mb3_general_ci' is a collation of the deprecated character set UTF8MB3. Please consider using UTF8MB4 with an appropriate collation instead.
+INSERT INTO t1 VALUES
+(x'e6f8e5', 'abc', 'abc', 'ABC', 'æøå', 'abc', 'abc', 'abc', 'æøå', 'abc');
+SELECT c1 = b0 FROM t1;
+c1 = b0
+0
+SELECT CONCAT(c1, b0) FROM t1;
+CONCAT(c1, b0)
+abc���
+SELECT c1 = c0 FROM t1;
+c1 = c0
+1
+SELECT CONCAT(c1, c0) FROM t1;
+CONCAT(c1, c0)
+abcabc
+SELECT c1 = c2 FROM t1;
+ERROR HY000: Illegal mix of collations (utf8mb4_0900_ai_ci,IMPLICIT) and (utf8mb4_0900_as_cs,IMPLICIT) for operation '='
+SELECT CONCAT(c1, c2) FROM t1;
+CONCAT(c1, c2)
+abcABC
+SELECT c1 = c3 FROM t1;
+c1 = c3
+0
+SELECT CONCAT(c1, c3) FROM t1;
+CONCAT(c1, c3)
+abcæøå
+SELECT c1 = c4 FROM t1;
+c1 = c4
+1
+SELECT CONCAT(c1, c4) FROM t1;
+CONCAT(c1, c4)
+abcabc
+SELECT c1 = c5 FROM t1;
+c1 = c5
+1
+SELECT CONCAT(c1, c5) FROM t1;
+CONCAT(c1, c5)
+abcabc
+SELECT c1 = c6 FROM t1;
+c1 = c6
+1
+SELECT CONCAT(c1, c6) FROM t1;
+CONCAT(c1, c6)
+abcabc
+SELECT c1 = c7 FROM t1;
+c1 = c7
+0
+SELECT CONCAT(c1, c7) FROM t1;
+CONCAT(c1, c7)
+abcæøå
+SELECT c1 = c8 FROM t1;
+c1 = c8
+1
+SELECT CONCAT(c1, c8) FROM t1;
+CONCAT(c1, c8)
+abcabc
+SELECT c7 = c8 FROM t1;
+ERROR HY000: Illegal mix of collations (latin1_swedish_ci,IMPLICIT) and (latin2_general_ci,IMPLICIT) for operation '='
+SELECT CONCAT(c7, c8) FROM t1;
+ERROR HY000: Illegal mix of collations (latin1_swedish_ci,IMPLICIT) and (latin2_general_ci,IMPLICIT) for operation 'concat'
+SELECT DATABASE() = _utf8mb4'æøå';
+DATABASE() = _utf8mb4'æøå'
+0
+SELECT CONCAT(DATABASE(), _utf8mb4'æøå');
+CONCAT(DATABASE(), _utf8mb4'æøå')
+test���
+SELECT DATABASE() = _utf8mb4'āđē';
+DATABASE() = _utf8mb4'āđē'
+0
+SELECT CONCAT(DATABASE(), _utf8mb4'āđē');
+CONCAT(DATABASE(), _utf8mb4'āđē')
+test???
 select 1=_latin1'1';
 1=_latin1'1'
 1
@@ -479,6 +580,33 @@ select concat(_latin1'a',_latin2'a',_latin5'a',_latin7'a');
 ERROR HY000: Illegal mix of collations for operation 'concat'
 select concat_ws(_latin1'a',_latin2'a');
 ERROR HY000: Illegal mix of collations (latin1_swedish_ci,COERCIBLE) and (latin2_general_ci,COERCIBLE) for operation 'concat_ws'
+SELECT CONCAT(_utf8mb4'æøå', 1);
+CONCAT(_utf8mb4'æøå', 1)
+���1
+SELECT CONCAT(_utf8mb4'āđē', 1);
+CONCAT(_utf8mb4'āđē', 1)
+???1
+SELECT _utf8mb4'æøå' = NULL;
+_utf8mb4'æøå' = NULL
+NULL
+SELECT CONCAT(_utf8mb4'æøå', NULL);
+CONCAT(_utf8mb4'æøå', NULL)
+NULL
+SELECT c3 = CONCAT(c1, c2) FROM t1;
+c3 = CONCAT(c1, c2)
+0
+SELECT GREATEST(c3, CONCAT(c1, c2)) FROM t1;
+GREATEST(c3, CONCAT(c1, c2))
+æøå
+SELECT COERCIBILITY(CONCAT(c1, c2)) FROM t1;
+COERCIBILITY(CONCAT(c1, c2))
+7
+SELECT CONCAT(c1, c2) = CONCAT(c1, c2) FROM t1;
+ERROR HY000: Illegal mix of collations (utf8mb4_bin,NONE) and (utf8mb4_bin,NONE) for operation '='
+SELECT CONCAT(CONCAT(c1, c2), CONCAT(c1, c2)) FROM t1;
+CONCAT(CONCAT(c1, c2), CONCAT(c1, c2))
+abcABCabcABC
+DROP TABLE t1;
 select FIELD('b','A','B');
 FIELD('b','A','B')
 2
@@ -6021,3 +6149,31 @@ BETWEEN '2037' AND ( -71 * ( -1552207807 * 440370369 ) )
 COLLATE utf8mb4_et_0900_as_cs;
 ERROR 22003: BIGINT value is out of range in '(-(71) * (-(1552207807) * 440370369))'
 DROP TABLE customer_demographics;
+#
+# Bug#37285902: Illegal mix of collations NONE vs. COERCIBLE
+#
+CREATE TABLE t1(
+id DECIMAL(38,0)
+);
+CREATE TABLE t2(
+name VARCHAR(150) COLLATE utf8mb4_0900_as_cs
+);
+WITH cte0 AS
+(SELECT CAST(t1.id AS CHAR(38)) AS c2
+FROM t1
+),
+cte1 AS
+(SELECT t2.name AS c1
+FROM t2
+),
+cte2 AS
+(SELECT d3.c1 AS c1, d1.c2 AS c2
+FROM cte0 AS d1 LEFT JOIN cte1 AS d3 ON TRUE
+WHERE IFNULL(d3.c1, d1.c2) LIKE 'Vision Operations'
+ )
+SELECT * FROM cte2;
+c1	c2
+SELECT IFNULL(name, CAST(t1.id AS CHAR(38))) LIKE 'Vision Operations'
+FROM t1 LEFT JOIN t2 ON TRUE;
+IFNULL(name, CAST(t1.id AS CHAR(38))) LIKE 'Vision Operations'
+DROP TABLE t1, t2;

mysql-test/r/func_test.result

@@ -948,7 +948,7 @@ SELECT GREATEST(dt.a, t.a) FROM t, (SELECT _utf8mb4 'a') AS dt(a);
 GREATEST(dt.a, t.a)
 a
 SELECT GREATEST(_utf8mb4 '±', a) FROM t;
-ERROR HY000: Cannot convert string '\xC3\x82\xC2\xB1' from ascii to utf8mb4
+ERROR HY000: Cannot convert string '\xC3\x82\xC2\xB1' from utf8mb4 to ascii
 SELECT GREATEST(dt.a, t.a) FROM t, (SELECT _utf8mb4 '±') AS dt(a);
 ERROR HY000: Cannot convert string '\xC3\x82\xC2\xB1' from utf8mb4 to ascii
 SELECT GREATEST(_utf8mb4 'a', b) FROM t;
@@ -970,7 +970,7 @@ SELECT GREATEST(dt.a, t.c) FROM t, (SELECT _utf8mb4 'a') AS dt(a);
 GREATEST(dt.a, t.c)
 a
 SELECT GREATEST(_utf8mb4 '±', c) FROM t;
-ERROR HY000: Cannot convert string '\xC3\x82\xC2\xB1' from latin2 to utf8mb4
+ERROR HY000: Cannot convert string '\xC3\x82\xC2\xB1' from utf8mb4 to latin2
 SELECT GREATEST(dt.a, t.c) FROM t, (SELECT _utf8mb4 '±') AS dt(a);
 ERROR HY000: Cannot convert string '\xC3\x82\xC2\xB1' from utf8mb4 to latin2
 SELECT GREATEST(_utf8mb4 'a', d) FROM t;
@@ -986,7 +986,7 @@ SELECT GREATEST(dt.a, t.a) FROM t, (SELECT _latin1 'a') AS dt(a);
 GREATEST(dt.a, t.a)
 a
 SELECT GREATEST(_latin1 'Ã¥', a) FROM t;
-ERROR HY000: Cannot convert string '\xC3\x83\xC2\xA5' from ascii to latin1
+ERROR HY000: Cannot convert string '\xC3\x83\xC2\xA5' from latin1 to ascii
 SELECT GREATEST(dt.a, t.a) FROM t, (SELECT _latin1 'Ã¥') AS dt(a);
 ERROR HY000: Cannot convert string '\xC3\x83\xC2\xA5' from latin1 to ascii
 SELECT GREATEST(_latin1 'a', c) FROM t;
@@ -996,7 +996,7 @@ SELECT GREATEST(dt.a, t.c) FROM t, (SELECT _latin1 'a') AS dt(a);
 GREATEST(dt.a, t.c)
 a
 SELECT GREATEST(_latin1 'Ã¥', c) FROM t;
-ERROR HY000: Cannot convert string '\xC3\x83\xC2\xA5' from latin2 to latin1
+ERROR HY000: Cannot convert string '\xC3\x83\xC2\xA5' from latin1 to latin2
 SELECT GREATEST(dt.a, t.c) FROM t, (SELECT _latin1 'Ã¥') AS dt(a);
 ERROR HY000: Cannot convert string '\xC3\x83\xC2\xA5' from latin1 to latin2
 DROP TABLE t;

mysql-test/suite/collations/r/binary.result

@@ -61,28 +61,73 @@ Abc
 abc
 abc 
 SELECT col_bin FROM t UNION SELECT col_0900_bin FROM t;
-ERROR HY000: Illegal mix of collations for operation 'UNION'
+ERROR HY000: Illegal mix of collations (utf8mb4_bin,IMPLICIT) and (utf8mb4_0900_bin,IMPLICIT) for operation 'UNION'
 SELECT col_0900_bin FROM t UNION SELECT col_bin FROM t;
-ERROR HY000: Illegal mix of collations for operation 'UNION'
+ERROR HY000: Illegal mix of collations (utf8mb4_0900_bin,IMPLICIT) and (utf8mb4_bin,IMPLICIT) for operation 'UNION'
 SELECT col_bin FROM t UNION SELECT col_0900_bin FROM t
 UNION SELECT col_aici FROM t;
-ERROR HY000: Illegal mix of collations for operation 'UNION'
-SELECT col_bin FROM t UNION SELECT col_0900_bin FROM t
-UNION SELECT col_aici COLLATE utf8mb4_0900_ai_ci FROM t;
-col_bin
+ERROR HY000: Illegal mix of collations (utf8mb4_bin,IMPLICIT), (utf8mb4_0900_bin,IMPLICIT), (utf8mb4_0900_ai_ci,IMPLICIT) for operation 'UNION'
+# Add an explicit collation in the first query term.
+# The explicit collation is used.
+SELECT col_aici COLLATE utf8mb4_0900_ai_ci FROM t
+UNION
+SELECT col_bin FROM t
+UNION
+SELECT col_0900_bin FROM t;
+col_aici COLLATE utf8mb4_0900_ai_ci
 ABC  
 Abc   
 abc
 abc 
-SELECT col_bin FROM t UNION SELECT col_0900_bin FROM t
-UNION SELECT col_aici COLLATE utf8mb4_bin FROM t;
-col_bin
+SELECT col_aici COLLATE utf8mb4_bin FROM t
+UNION
+SELECT col_bin FROM t
+UNION
+SELECT col_0900_bin FROM t;
+col_aici COLLATE utf8mb4_bin
 ABC
 Abc
 abc
-SELECT col_bin FROM t UNION SELECT col_0900_bin FROM t
-UNION SELECT col_aici COLLATE utf8mb4_0900_bin FROM t;
-col_bin
+SELECT col_aici COLLATE utf8mb4_0900_bin FROM t
+UNION
+SELECT col_bin FROM t
+UNION
+SELECT col_0900_bin FROM t;
+col_aici COLLATE utf8mb4_0900_bin
+ABC
+ABC  
+Abc
+Abc   
+abc
+abc 
+# Add an explicit collation in the last query term.
+# The explicit collation is used, so same result as
+# above.
+SELECT col_aici FROM t
+UNION
+SELECT col_bin FROM t
+UNION
+SELECT col_0900_bin COLLATE utf8mb4_0900_ai_ci FROM t;
+col_aici
+ABC  
+Abc   
+abc
+abc 
+SELECT col_aici FROM t
+UNION
+SELECT col_bin FROM t
+UNION
+SELECT col_0900_bin COLLATE utf8mb4_bin FROM t;
+col_aici
+ABC
+Abc
+abc
+SELECT col_aici FROM t
+UNION
+SELECT col_bin FROM t
+UNION
+SELECT col_0900_bin COLLATE utf8mb4_0900_bin FROM t;
+col_aici
 ABC
 ABC  
 Abc