Plinth.Database.PgSql 1.7.0

README

Plinth.Database.PgSql

Stored Procedure based mini-framework for PostgreSQL

Provides Transaction management, stored procedure execution, result set handling, and transient error detection

1. Register the transaction factory and provider with DI in Setup

  // IConfiguration configuration;

   var txnFactory = new SqlTransactionFactory(
       configuration,
       "MyDB",
       config.GetConnectionString("MyDB"));

   services.AddSingleton(txnFac);              // for injecting the SqlTransactionFactory
   services.AddSingleton(txnFac.GetDefault()); // for injecting the ISqlTransactionProvider

2. Settings in appsettings.json

Example appsettings.json 👉 All settings in PlinthPgSqlSettings are optional. The defaults are shown below.

{
  "ConnectionStrings": {
    "MyDB": "Host=...."
  },
  "PlinthPgSqlSettings": {
    "SqlCommandTimeout": "00:00:50",
    "SqlRetryCount": 3,
    "SqlRetryInterval": "00:00:00.200",
    "SqlRetryFastFirst": true,
    "DisableTransientRetry": false
  }
}

• SqlCommandTimeout: A TimeSpan formatted time for the default time for each SQL operation. Default is 50 seconds.
• SqlRetryCount: If a transient error is detected, maximum number of retries after the initial failure. Default is 3. This allows up to 4 attempts.
• SqlRetryInterval: If a transient error is detected, this is how long between retry attempts. Default is 200 milliseconds.
• SqlRetryFastFirst: If true, upon the first transient error, the first retry will happen immediately. Subsequent transient errors will wait the SqlRetryInterval. Default is true.
• DisableTransientRetry: If true, transient errors will not trigger retries. Default is false.

3. Transient Errors

It is very common on cloud hosted databases to have the database return transient errors that will work perfectly if retried. These errors can be things like deadlocks, timeouts, throttling, and transport errors.

The framework accepts a function to execute the whole transaction. When a transient error occurs, the entire transaction is rolled back and the function is executed again.

⚠️ Your code inside a transaction should be re-entrant. Anything that is performed that cannot be rolled back (such as sending an email), should be performed outside the transaction or be checked to confirm that it won't execute more than once. 👉 When running an operation without a transaction, the function may still retry, depending on whether a call to ExecuteProc has occurred. Nothing will be rolled back. See section on executing without a transaction for more details.

4. Creating a Transaction

Below is an example controller that creates a transaction, executes a stored procedure, and returns the result.

[Route("api/[controller]")]
[ApiController]
public class MyThingController : Controller
{
    private readonly ISqlTransactionProvider _txnProvider;

    public MyThingController(ISqlTransactionProvider _txnProvider)
    {
        _txnProvider = txnProvider;
    }

    [HttpGet]
    [Route("{thingId}")]
    [ProducesResponseType(200)]
    public async Task<ActionResult<MyThing>> GetMyThing(Guid thingId, CancellationToken ct)
    {
        var myThing = await _txnProvider.ExecuteTxnAsync(connection =>
        {
            return await connection.ExecuteQueryProcOneAsync(
                "fn_get_mything_by_id",
                row => Task.FromResult(new MyThing
                {
                    Field1 = row.GetInt("i_field1"),
                    Filed2 = row.GetDateTimeNull("dt_field2")
                    ... etc
                }),
                new NpgNpgsqlParameter("@i_thing_id", thingId)).Value;
        }, ct);

        if (myThing is null)
            throw new LogicalNotFoundException($"MyThing {thingId} was not found");

        return Ok(myThing);        
    }
}

5. Executing Stored Procedures with no Result Set

To execute a stored procedure that does not return a result set, use one of these three options. Typically used with DML procedures that insert/update/delete. 👉 All forms also have an overload that accepts a CancellationToken

1. ExecuteProcAsync(string procName, params NpgsqlParameter[] parameters)
   • This will execute the procedure, 👉 and fail if no rows were modified
2. ExecuteProcAsync(string procName, int expectedRows, params NpgsqlParameter[] parameters)
   • This will execute the procedure, and fail if the rows modified does not match expectedRows
3. ExecuteProcUncheckedAsync(string procName, params NpgsqlParameter[] parameters)
   • This will execute the procedure, and return the number of rows modified

6. Executing Stored Procedures that return a Result Set

To execute a stored procedure returns a result set, use one of these three options. Typically used with SELECT queries. 👉 All forms also have an overload that accepts a CancellationToken

1. ExecuteQueryProcAsync(string procName, params NpgsqlParameter[] parameters)
   • Returns an IAsyncEnumerable<IResult> which can be enumerated to extract objects from rows.
2. ExecuteQueryProcListAsync<T>(string procName, Func<IResult, Task<T>> readerFunc, params NpgsqlParameter[] parameters)
   • Returns a List<T> of objects returned from the Func called on each row returned.
   • 👉 Always returns a non-null List<T> that may be empty.
3. ExecuteQueryProcOneAsync(string procName, Func<IResult, Task> readerFunc, params NpgsqlParameter[] parameters)
   • Calls the Func with a single row result (if one found), returns true/false if row was found.
4. ExecuteQueryProcOneAsync<T>(string procName, Func<IResult, Task<T>> readerFunc, params NpgsqlParameter[] parameters)
   • Calls the Func with a single row result and returns the output inside a SqlSingleResult<T> object.
   • Use .Value to get the result and .RowReturned to determine if a row was returned.

7. Special Connection Features For Transactions

• SetRollback(): Will mark this transaction for later rollback when the transaction function is complete
• _WillBeRollingBack(): Determine if SetRollback() has been called on this transaction
• IsAsync(): Determine if this transaction supports async operations
• CommandTimeout {get; set;}: The default timeout for sql commands (in seconds)

8. Rollback and Post Commit Actions

These allow you to have code execute after a rollback or a commit occurs. Useful for cleaning up non-transaction items or taking actions after database operations are committed.

Post Rollback Actions:

• AddRollbackAction(string? desc, Action onRollback) AddAsyncRollbackAction(string? desc, Func<Task> onRollbackAsync)
  • These will execute the action/func after a rollback has completed
  • Common use case: Undoing a non-transactional thing that should only exist if the transaction succeeded

Post Commit Actions:

• AddPostCommitAction(string? desc, Action postCommit) AddAsyncPostCommitAction(string? desc, Func<Task> postCommitAsync)
  • These will execute the action/func after the transaction has been committed
  • Common use case: Performing some action that should only occur if the database operations are confirmed.

9. Running without a Transaction

Plinth as a general philosophy prefers "always correct, even if sometimes sub-optimal". Doing all database operations within a transaction ensures that if new SQL operations are added, they will always join the transaction with any other operations run within the function.
However, sometimes the overhead of starting and committing a transaction becomes an issue. If running only queries, or always just a single ExecuteProcAsync, a transaction is not technically required.

ISqlTransactionProvider contains several forms of ExecuteWithoutTxn() which run a function to perform SQL operations, without an explicit transaction.

Below is example code that executes a stored procedure without a transaction

    var myThing = await _txnProvider.ExecuteWithoutTxnAsync(connection =>
    {
        return await connection.ExecuteQueryProcOneAsync(
            "fn_get_mything_by_id",
            row => Task.FromResult(row.GetInt("my_column")),
            new NpgsqlParameter("@i_thing_id", thingId)).Value;
     }, ct);

Notes:

• The connection provided by ExecuteWithoutTxnAsync can be used to perform any number of ExecuteQueryProc type calls, but only one ExecuteProc type calls. An exception will be thrown upon the second ExecuteProc call
• After an ExecuteProc call that modifies data, an exception will not trigger a rollback. Once the call completes, the data is committed.
• The function provided will be retried upon a transient error unless an ExecuteProc type call has been made

10. Error and Post Close Actions

These allow you to have code execute after a connection is closed due to error or successful completion of the callback. Useful for cleaning up items or taking actions after database operations are completed. These are analogous to those in section 8 but are not tied to a transaction lifecycle.

Post Error Actions:

• AddErrorAction(string? desc, Action<Exception?> onError) AddAsyncErrorAction(string? desc, Func<Exception?, Task> onErrorAsync)
  • These will execute the action/func after an exception has been caught in the callback
  • Common use case: Undoing an operation that should only exist if the callback succeeded

👉 These will be executed each time the callback executes, even if there are retries. There will be not retries if an ExecuteProc type call was made.

Post Close Actions:

• AddPostCloseAction(string? desc, Action postClose) AddAsyncPostCloseAction(string? desc, Func<Task> postCloseAsync)
  • These will execute the action/func after the callback has completed successfully
  • Common use case: Performing some action that should only occur if the entire set of database operations complete successfully

11. Multiple Result Sets

Some stored procedures can actually return multiple result sets in a single call.

To execute and process each result set, use this method: ExecuteQueryProcMultiResultSetAsync(string procName, Func<IAsyncMultiResultSet, Task> readerFunc, params NpgsqlParameter[] parameters)

Example

  await c.ExecuteQueryProcMultiResultSetAsync(
      "fn_get_multiple_results", 
      async (mrs) =>
      {
          var rs = await mrs.NextResultSetAsync();
          await processSet1(rs);

          rs = await mrs.NextResultSetAsync();
          await processSet2(rs);

          rs = await mrs.NextResultSetAsync();
          await processSet3(rs);
      },
      new NpgsqlParameter("@i_int1", 10));

12. IDeferredSqlConnection

This allows for recording a sequence of stored procedure calls (without actually executing them) and then executing them all at one at a later time.

Example:

    var deferred = _txnProvider.GetDeferred();

    // no sql actions occur
    deferred.ExecuteProc("fn_insert_thing". new NpgsqlParameter("@i_id", 5));
    deferred.ExecuteProc("fn_insert_thing". new NpgsqlParameter("@i_id", 10));    

    await _txnProvider.ExecuteTxnAsync(connection =>
    {
        // now the sql actions are executed
        await connection.ExecuteDeferredAsync(deferred);
    });
    

13. Raw SQL Transactions

Normal transactions as shown above only allow for executing stored procedures. There are times and cases where executing a raw SQL statement is required. To do so, use ExecuteRawTxnAsync as shown in the below example:

        var myThing = await _txnProvider.ExecuteRawTxnAsync(connection =>
        {
            return await connection.ExecuteRawQueryOneAsync(
                "SELECT i_field1, dt_field2 FROM my_things WHERE i_thing_id = @i_thing_id",
                row => Task.FromResult(new MyThing
                {
                    Field1 = row.GetInt("i_field1"),
                    Filed2 = row.GetDateTimeNull("dt_field2")
                    ... etc
                }),
                new NpgsqlParameter("@i_thing_id", thingId)).Value;
        }, ct);

The methods are analogues of the methods in sections 5, 6 and 11.

• ExecuteRawAsync for DML
• ExecuteRawQueryListAsync for queries that return a list of results
• ExecuteRawQueryOneAsync for queries that return a single result
• ExecuteRawQueryMultiResultSetAsync for queries that return multiple result sets