How To Carry out Azure Cross Container Transaction Operation In Cosmos DB

Introduction

This info was gathered whereas Implementing ingesting information to Cosmos DB.

The requirement was to insert roughly one thousand paperwork in cosmos DB containers in minimal time with atomicity. We now have to ingest information into two collections utilizing Transactions

Limitations

By default cosmos DB operations don’t assist transactions for this primary we used TransactionBatch class.

At the moment, there are two identified limits:

• The Azure Cosmos DB request dimension restrict constrains the dimensions of the TransactionalBatch payload to not exceed 2 MB, and the utmost execution time is 5 seconds.
• There’s a present restrict of 100 operations per TransactionalBatch to make sure the efficiency is as anticipated and inside SLAs.

Subsequent was procedures.

As described in MS docs “The JavaScript-based saved procedures, triggers, UDFs, and merge procedures are wrapped inside an ambient ACID transaction with snapshot isolation throughout all gadgets inside the logical partition. Throughout its execution, if the JavaScript program throws an exception, your complete transaction is aborted and rolled again. The ensuing programming mannequin is easy but highly effective. JavaScript builders get a sturdy programming mannequin whereas nonetheless utilizing their acquainted language constructs and library primitives.”

Procedures have a restrict of a most of 5 seconds run time if you’ll want to run a process for greater than that then you’ll want to use continuation.

So this appeared a greater match for me because the max row would attain solely 1000 and within the context of cosmosDb, 5 seconds for 1000 rows is greater than sufficient.

Instance

Write Saved Process for upsert; i.e., insert/replace

Beneath is the upsert process which first checks the merchandise within the container and whether it is current within the container then updates the information in any other case inserts the merchandise within the container.

operate createToDoItems(gadgets) {
    var assortment = getContext().getCollection();
    var collectionLink = assortment.getSelfLink();
    var rely = 0;

    if (!gadgets) throw new Error("The array is undefined or null.");

    var numItems = gadgets.size;

    if (numItems == 0) {
        getContext().getResponse().setBody(0);
        return;
    }

    tryCreate(gadgets[count], callback);

    operate tryCreate(merchandise, callback, continuation) {
        var choices = { disableAutomaticIdGeneration: false };

		var requestOptions = {continuation: continuation};
		var question = "SELECT * FROM c WHERE c.id = '"+merchandise.id+"'";
		 
		var isAccepted = assortment.queryDocuments(collectionLink, question, requestOptions, operate (err, paperwork, responseOptions) {
            if (err) throw err;

            if (paperwork.size > 0) {
                tryUpdate(paperwork[0], merchandise);
            } else {
                var isCreated = assortment.createDocument(collectionLink, merchandise, choices, callback);
				if (!isAccepted) getContext().getResponse().setBody(rely);
            }
        });
        if (!isAccepted) getContext().getResponse().setBody(rely);
    }

    operate callback(err, merchandise, choices) {
        if (err) throw err;
        rely++;
        if (rely >= numItems) {
            getContext().getResponse().setBody(rely);
        } else {
            tryCreate(gadgets[count], callback);
        }
    }
	
	operate tryUpdate(doc, replace) {
		var requestOptions = {etag: doc._etag};

		var isAccepted = assortment.replaceDocument(doc._self, replace, requestOptions, operate (err, updatedDocument, responseOptions) {
			if (err) throw err;
			rely++;
			if (rely >= numItems) {
				getContext().getResponse().setBody(rely);
			}
			else{	
				tryCreate(gadgets[count], callback);
			}
			if (!isAccepted) getContext().getResponse().setBody(rely);
		});
	}
}

Write C# code to name the cosmos process.

attempt
{ 
    var outcome = await this.container.Container.Scripts.ExecuteStoredProcedureStreamAsync("<SProcName>", new PartitionKey("<PartitionKeyValue>"), new[] { Knowledge.ToArray() }); 
    outcome.EnsureSuccessStatusCode();    
    resultCode = outcome.StatusCode;
}
catch (CosmosException ex)
{
}

Through the use of this method we accomplished the operation in 27 milliseconds common time after 25 requests which is sort of spectacular.

Now in case you are performing operations on 2 containers with completely different partition keys then sproc will assist you numerous. For ACID I got here up with few guidelines of my very own,

1. The method I went forward with was to first upsert the kid information
2. Fetch all the present little one container information in an inventory. {We are going to use the later in rollback}.
3. If little one upsert faces some situation then the sproc will throw an exception and rollback the transaction and I cannot proceed with father or mother information
4. If the kid is profitable then I will proceed with father or mother container information
5. If Dad or mum container upsert in efficiently {Yay!}
6. If father or mother container upsert faces a problem then sproc will throw an exception and roll again this transaction however the little one information is already inserted and now we have to revert that manually. {Now we are going to use the information which we obtained in step 2.}
7. Delete your complete little one container merchandise and insert the information which we fetched in step 2.

public async Job RollBack(IEnumerable<Knowledge> existingData, IEnumerable<Knowledge> newData)
{
    // Test if exisitingData is current
    // if true  -> Delete new created paperwork and create previous paperwork
    // if false -> Delete the brand new created paperwork from assortment
    if (existingData?.Rely() > 0)
    {
        await this.DeleteAsync(existingData.ToList());
        await this.UpsertAsync(newData.ToList());
    }
    else
    {
        await this.DeleteAsync(newData.ToList());
    }
}

And that is it utilizing these easy steps I used to be in a position to carry out my transaction. The worst-case time length for this logic was 200 milliseconds for 1000 gadgets which is sweet for me. Nicely, you’ll be able to all the time modify and people guidelines in accordance with your wants and include a brand new method altogether.

Please attain out to me in case you discover a higher method of doing this.

I hope you discover this convenient.

Thanks.