Rewrite our internal type system

(Sorry for the really large diff but this touches nearly all parts of
diesel)

This commit does in two things:
* Refactor our handling of nullable values. Instead of having a marker
trait for non nullable sql types we now indicate if a sql type is
nullable by using a associated type on a new fundamental trait named
SqlType. This allows us to reason if an type is nullable or not in a
much precise way without running in conflicting implementation
issues. This allows us to address #104 in a much more fundamental
way. (The correct way as mentioned there by sgrif).
* Refactor our handling of typed and untyped sql fragments. Instead of
having two separate code paths for `Queryable` (sql types known) and
`QueryableByName` (sql types not known) we now have only one code path
and indicate if a query is typed or untyped as part of the expression
sql type. This is required to address #2150. `Queryable` and
`QueryableByName` now simple . Additionally we separate the static size component of
`FromSqlRow` to allow dynamically sized rows there (but only at the
last position for tuple impls.)

I should probably have implement those changes in different commits
but as both changes basically requiring touching most of our code base
this would have required much more work...

Both changes are theoretically big major breaking changes. For
application code I expect the actual breakage to be minimal, see the
required changes in `diesel_tests` and `diesel_cli` for examples. For
highly generic code I would expect quite a few required changes.

Additionally there are a few small drive by fixes.

Fixes #104
Fixes #2274
Fixes #2161

CHANGELOG.md

@@ -39,6 +39,9 @@ for Rust libraries in [RFC #1105](https://github.com/rust-lang/rfcs/blob/master/
   functionality of `NonAggregate`. See [the upgrade
   notes](#2-0-0-upgrade-non-aggregate) for details.
 
+* It is now possible to inspect the type of values returned from the database
+  in such a way to support constructing a dynamic value depending on this type.
+
 
 ### Removed
 
@@ -47,6 +50,7 @@ for Rust libraries in [RFC #1105](https://github.com/rust-lang/rfcs/blob/master/
 * Support for `bigdecimal` < 0.0.13 has been removed.
 * Support for `pq-sys` < 0.4.0 has been removed.
 * Support for `mysqlclient-sys` < 0.2.0 has been removed.
+* The `NonNull` for sql types has been removed in favour of the new `SqlType` trait.
 
 ### Changed
 
@@ -68,8 +72,6 @@ for Rust libraries in [RFC #1105](https://github.com/rust-lang/rfcs/blob/master/
 * The `RawValue` types for the `Mysql` and `Postgresql` backend where changed
   from `[u8]` to distinct opaque types. If you used the concrete `RawValue` type
   somewhere you need to change it to `mysql::MysqlValue` or `pg::PgValue`.
-  For the postgres backend additionally type information where added to the `RawValue`
-  type. This allows to dynamically deserialize `RawValues` in container types.
 
 * The uuidv07 feature was renamed to uuid, due to the removal of support for older uuid versions
 
@@ -93,6 +95,26 @@ for Rust libraries in [RFC #1105](https://github.com/rust-lang/rfcs/blob/master/
   `#[non_exhaustive]`. If you matched on one of those variants explicitly you need to
   introduce a wild card match instead.
 
+* `FromSql::from_sql` is changed to construct value from non nullable database values.
+   To construct a rust value for nullable values use the new `FromSql::from_nullable_sql`
+   method instead.
+
+* Custom sql types are now required to implement the new `SqlType` trait. Diesel will
+  automatically create implementations of that trait for all types having a `#[derive(SqlType)]`
+
+* The workflow for manually implementing support custom types has changed. Implementing
+  `FromSqlRow<ST, DB>` is not required anymore, as this is now implied by implementing
+  `FromSql<ST, DB>`. The requirement of implementing `Queryable<ST, DB>` remains
+  unchanged. For types using `#[derive(FromSqlRow)]` no changes are required as the
+  derive automatically generates the correct code
+
+* The structure of our deserialization trait has changed. Loading values from the database
+  requires now that the result type implements `FromSqlRow<ST, DB>`. Diesel provides wild
+  card implementations for types implementing `Queryable<ST, DB>` or `QueryableByName<DB>`
+  so non generic code does not require any change. For generic code you likely need to
+  replace a trait bound on `Queryable<ST, DB>` with a trait bound on `FromSqlRow<ST, DB>`
+  and a bound to `QueryableByName<DB>` with `FromSqlRow<Untyped, DB>`.
+
 
 ### Fixed
 
@@ -129,6 +151,10 @@ for Rust libraries in [RFC #1105](https://github.com/rust-lang/rfcs/blob/master/
 
 * We've refactored our type translation layer for Mysql to handle more types now.
 
+* We've refactored our type level representation of nullable values. This allowed us to
+  fix multiple long standing bugs regarding the correct handling of nullable values in some
+  corner cases (#104, #2274)
+
 ### Deprecated
 
 * `diesel_(prefix|postfix|infix)_operator!` have been deprecated. These macros
@@ -138,7 +164,6 @@ for Rust libraries in [RFC #1105](https://github.com/rust-lang/rfcs/blob/master/
 * `diesel::pg::upsert` has been deprecated to support upsert queries on more than one backend.
   Please use `diesel::upsert` instead.
 
-
 ### Upgrade Notes
 
 #### Replacement of `NonAggregate` with `ValidGrouping`

diesel/Cargo.toml

@@ -32,6 +32,7 @@ num-integer = { version = "0.1.39", optional = true }
 bigdecimal = { version = ">= 0.0.13, < 0.2.0", optional = true }
 bitflags = { version = "1.2.0", optional = true }
 r2d2 = { version = ">= 0.8, < 0.9", optional = true }
+itoa = "0.4"
 
 [dependencies.diesel_derives]
 version = "~2.0.0"

diesel/src/associations/belongs_to.rs

@@ -4,6 +4,7 @@ use crate::expression::array_comparison::AsInExpression;
 use crate::expression::AsExpression;
 use crate::prelude::*;
 use crate::query_dsl::methods::FilterDsl;
+use crate::sql_types::SqlType;
 
 use std::borrow::Borrow;
 use std::hash::Hash;
@@ -139,6 +140,7 @@ where
     Id<&'a Parent>: AsExpression<<Child::ForeignKeyColumn as Expression>::SqlType>,
     Child::Table: FilterDsl<Eq<Child::ForeignKeyColumn, Id<&'a Parent>>>,
     Child::ForeignKeyColumn: ExpressionMethods,
+    <Child::ForeignKeyColumn as Expression>::SqlType: SqlType,
 {
     type Output = FindBy<Child::Table, Child::ForeignKeyColumn, Id<&'a Parent>>;
 
@@ -154,6 +156,7 @@ where
     Vec<Id<&'a Parent>>: AsInExpression<<Child::ForeignKeyColumn as Expression>::SqlType>,
     <Child as HasTable>::Table: FilterDsl<EqAny<Child::ForeignKeyColumn, Vec<Id<&'a Parent>>>>,
     Child::ForeignKeyColumn: ExpressionMethods,
+    <Child::ForeignKeyColumn as Expression>::SqlType: SqlType,
 {
     type Output = Filter<Child::Table, EqAny<Child::ForeignKeyColumn, Vec<Id<&'a Parent>>>>;
 

diesel/src/connection/mod.rs

@@ -6,10 +6,10 @@ mod transaction_manager;
 use std::fmt::Debug;
 
 use crate::backend::Backend;
-use crate::deserialize::{Queryable, QueryableByName};
+use crate::deserialize::FromSqlRow;
+use crate::expression::QueryMetadata;
 use crate::query_builder::{AsQuery, QueryFragment, QueryId};
 use crate::result::*;
-use crate::sql_types::HasSqlType;
 
 #[doc(hidden)]
 pub use self::statement_cache::{MaybeCached, StatementCache, StatementCacheKey};
@@ -169,18 +169,12 @@ pub trait Connection: SimpleConnection + Sized + Send {
     fn execute(&self, query: &str) -> QueryResult<usize>;
 
     #[doc(hidden)]
-    fn query_by_index<T, U>(&self, source: T) -> QueryResult<Vec<U>>
+    fn load<T, U>(&self, source: T) -> QueryResult<Vec<U>>
     where
         T: AsQuery,
         T::Query: QueryFragment<Self::Backend> + QueryId,
-        Self::Backend: HasSqlType<T::SqlType>,
-        U: Queryable<T::SqlType, Self::Backend>;
-
-    #[doc(hidden)]
-    fn query_by_name<T, U>(&self, source: &T) -> QueryResult<Vec<U>>
-    where
-        T: QueryFragment<Self::Backend> + QueryId,
-        U: QueryableByName<Self::Backend>;
+        U: FromSqlRow<T::SqlType, Self::Backend>,
+        Self::Backend: QueryMetadata<T::SqlType>;
 
     #[doc(hidden)]
     fn execute_returning_count<T>(&self, source: &T) -> QueryResult<usize>