Kickstarting a database-wg

[weiznich]

Does the current “sql_query” interface not support prepared statements at all? That doesn’t seem suitable for a production-level API since parsing SQL on every query is suboptimal.

It uses prepared statements internally, just like every other part of diesels api.

Again, several layers:

1. An API that just passes the SQL to the database,
2. A simple string-based SQL builder
3. Something that can tokenize SQL and do basic transforms between databases like changing placeholders
4. An untyped SQL AST with a parser, renderer and transforms
5. A dynamically typed SQL AST
6. A statically typed SQL AST like what Diesel has currently

Using this layered approach will involve a significant performance overhead because you need to go back and forth between a typed representation and several string representations. (As seen from the 6. layer). Diesel does currently something differently in some cases: It skips building the actual sql query, because we have already build that query before and it’s now in the prepared statement cache. The corresponding prepared statement could be received using a statically known type id. Such optimizations cannot work if you have to pass a sql statement though several layers that expect some sort of sql string.

Probably have a “ResultReader” interface with a method for every database type that allows to read one value of that type from the database connection and returns an Error (or garbage if the types are not self-described) if the type is incorrect, and also something that returns a non-exhaustive enum of all possible types across all databases.

Returning a “non-exhaustive enum of all possible types across all databases” is not an option here because some database systems allow user defined types.

On top of that, provide higher level interfaces like custom derive deserializers and serde deserializers. Same for serializing.

How would that be to the currently provided derives for Queryable, Insertable and AsChangeset?

[DanielFath]

Using this layered approach will involve a significant performance overhead because you need to go back and forth between a typed representation and several string representations. (As seen from the 6. layer). Diesel does currently something differently in some cases: It skips building the actual sql query, because we have already build that query before and it’s now in the prepared statement cache. The corresponding prepared statement could be received using a statically known type id. Such optimizations cannot work if you have to pass a sql statement though several layers that expect some sort of sql string.

Hm, then how would the API for Diesel drivers look like? Or are you suggesting that for diesel the Driver is tightly coupled to the implementation?

From what I know about Diesel (and it’s not much) it turns queries like select * from users into a type like Selectable<Star, From<users>>, and uses its TypeId to cache that query.

[weiznich]

Hm, then how would the API for Diesel drivers look like? Or are you suggesting that for diesel the Driver is tightly coupled to the implementation?

At the lowest level there is the Connection trait. Everything inside of that is implementation specific, though there are several shared components like the StatementCache or the TransactionManager.

From what I know about Diesel (and it’s not much) it turns queries like select * from users into a type like Selectable<Star, From<users>> , and uses its TypeId to cache that query.

Basically yes (The types are a bit different). That only works in specific cases, otherwise the sql is used as key.

[DanielFath]

Basically yes (The types are a bit different). That only works in specific cases, otherwise the sql is used as key.

Ok, so hypothetically speaking, if some future ConnectionDriver returned PrepareStatement or at least some kind of PreparedStatementId would that still work for Diesel, or does Diesel require some specific scenario for it to cache its SQL statements?