Sync Cached Information With Cosmos DB Utilizing Ncache

Introduction
 

What’s CosmosDB?

Azure Cosmos DB is a completely managed and globally distributed multi-model cloud-based NoSQL database service. It’s absolutely managed due to its simplicity of deployment and complete service degree agreements (SLAs) from Microsoft. It helps versatile JSON format which suggests it helps unstructured and JSON information and straightforward to retailer information with very much less code.

Frequent use circumstances of CosmosDB 

1. IoT – To ingest, course of and retailer information 
2. Gaming – utilized in social media integration, high-score leaderboards and lots of different
3. Retail and advertising – For storing lookup information, occasion sourcing so as processing pipelines 
4. Internet and Cellular Software – Integrating with third-party companies, storing information and personalization 

NCache with CosmosDB

We are able to additional improve or scale up the efficiency of CosmosDB with NCache. NCache is an Open Supply in-memory distributed cache for .NET, Java, and Node.js. NCache is extraordinarily quick and linearly scalable and caches utility information to scale back costly database journeys. 

NCache sits a lot nearer to your utility and therefore is way quicker to entry. 

From this text, you’ll learn to combine distributed caching into your Azure Cosmos DB utility by utilizing NCache and information sync between Azure Cosmos DB and NCache utilizing Azure capabilities.

Main Issues in CosomsDB
 

1. Efficiency Bottleneck 

Microsoft Azure claims that Cosmos DB goes to present you a millisecond response time, alright, what in case your answer will minimize down a second to millisecond and millisecond to sub millisecond response time? It’s attainable by Incorporating the NCache with CosmosDB. 

ComosDB is a hosted service so will probably be throughout VNET in its personal subscription, which will increase the latency half, the efficiency of your cosmos DB goes to be degraded particularly underneath excessive transaction/masses. When you might have plenty of requests coming in and it is advisable cope with that information in a quick well timed method. Accessing it throughout your VNET could be very gradual and take into consideration accessing it from On-premise App when there may be excessive site visitors. That is the place a distributed cache similar to NCache comes into image. You may cache the often accessed information to enhance response occasions. The distributed nature of caching in NCache for Azure CosmosDB ensures optimum efficiency underneath excessive transaction masses by making the cache linearly scalable too.

2. Excessive Price 

Price is among the main elements in relation to Azure Cosmos DB. Every time if you attempt to entry Microsoft Azure Cosmos DB, even for the read-only information it prices as a result of the fee plan is one thing primarily based on the request unit. If that information isn’t being modified that often, it is the identical content material however you continue to have the grasp supply coming from the Cosmos DB that is the information that you are looking in. For that information, you must preserve going backwards and forwards to Microsoft Azure Cosmos DB and that is the place you pay by way of request unit and that will increase your price. So, prices would go up in case your request load will increase. The utilization price goes up and it is a matter particularly for Learn-only information.

So, it is very scalable, so far as requests load that it will probably deal with however it’s not going to be very price pleasant in that state of affairs. So, it may influence the fee issue as nicely. So, it may very well be a possible hurdle for scaling out. If you’re planning to scale out. You could have a number of partitions; you might have your information load growing however the fee issue goes up alongside that. Ncache integration with ComosDB shall be very helpful in relation to price issue as a result of you possibly can scale back variety of request unit by caching the often requested information in Ncache distributed caching service 

Let’s undergo following matters with demo 

1. App information caching with Cosmos DB
2. Sync NCache with Azure CosmosDB

App information with CosmosDB

Earlier than beginning, we must always have Azure CosmosDB to combine it with NCache. I’ve created an Azure CosmosDB utilizing Azure portal and downloaded the pattern code for .NET from the short begin part.

Set up NCache Supervisor 

Obtain NCache from NCache Obtain Middle (alachisoft.com). I extremely advocate you to put in the Enterprises version with the intention to discover all of the options. Earlier than the set up be sure .NET 6 has been put in in your machine. 

After finishing the Obtain and set up course of, you possibly can run the NChace Internet Manger within the browser from localhost 8251 port, as proven within the under determine.

We are able to create a clustered and Native Caches from the NCache Supervisor utility.

For this demo, I’ve created an area cache and named it as “myLocalCache” utilizing Ncache Supervisor as proven within the under determine.

Click on right here to test the best way to create an area Cache utilizing NCahce Internet Supervisor.

Open the Pattern cosmosDB fast begin supply code in visible studio and run it. It’s going to add a brand new merchandise to the database as proven in under determine. 

Set up Alachisoft.NCache.SDK package deal from NuGet package deal supervisor 

Earlier than operating the applying, test the statistics of your native cache utilizing NCache internet Supervisor.  The depend shall be zero as proven within the under determine,

Add a under perform, within the utility. 

non-public async Activity GetFamily(Household household) {
    strive {
        string key = $ "Household:FamilyID:{household.Id}";
        Household familyResult = _NCacheClient.Get < Household > (key);
        if (familyResult != null) {
            Console.WriteLine(familyResult.PartitionKey);
        } else {
            var sqlQueryText = "SELECT * FROM c";
            QueryDefinition queryDefinition = new QueryDefinition(sqlQueryText);
            FeedIterator < Household > queryResultSetIterator = this.container.GetItemQueryIterator < Household > (queryDefinition);
            whereas (queryResultSetIterator.HasMoreResults) {
                FeedResponse < Household > currentResultSet = await queryResultSetIterator.ReadNextAsync();
                foreach(Household familyTemp in currentResultSet) {
                    if (familyTemp.Id == household.Id) {
                        _NCacheClient.Insert(key, familyTemp.ToString());
                        Console.WriteLine("tRead {0}n", familyTemp.LastName);
                    }
                }
            }
            Console.WriteLine(familyResult.LastName);
        }
    } catch (Exception ex) {
        //TODO
    }
}

This perform will test if the the hot button is current within the cache or not. If the hot button is out there in cache it would fetch information from the cache utilizing the cache or else it would fetch the information from cosmosDB and insert it into Cache utilizing the Insert perform. 

Now test the NCahce statistics, the depend shall be elevated to 1 as proven in under determine, which suggests the file has been inserted into the native cache server.

If you run this system subsequent time, it would fetch the file from Native cache server primarily based on key.

Syncing NCache with Azure CosmosDB

Synchronization is an enormous side when your information exist in two totally different locations. Every time when there’s a change within the cache you additionally apply these adjustments within the database as nicely. However what occurs if information adjustments immediately in CosmosDB and that’s going to most pure state of affairs the place CosmosDB getting up to date by one other service. 

Regardless of the adjustments are accomplished in CosmosDB is not going to replicate within the cache, proper now these two sources are out of sync. We have to have some type of synchronization mechanism between Azure CosmosDB and NCache.

There are two methods you possibly can sync NCache with Azure CosmosDB 

1. Azure perform with CosmosDB Set off
   • Implicit change feed processing
   • Sync Insert & replace and Delete operation
2. Customized change Feed processor
   • Implement IChangeFeedObserver interface explicitly
   • Sync Insert, replace and delete operations

Azure Perform with CosmosDB Set off

Utilizing Azure perform, it is extremely straightforward to sync the adjustments in ComosDB to cache.

Create an Azure perform utilizing Visual Studio with the CosmosDB Set off

Copy your CosmosDB Connection string from azure portal to Connection string setting identify.

Exchange the code in Function1.cs file with under code.

[FunctionName("Function1")]
public static void Run([CosmosDBTrigger(databaseName: "ToDoList", collectionName: "Items", ConnectionStringSetting = "<Your CosmosDB Connection string>", LeaseCollectionName = "leases")] IReadOnlyList < Doc > enter, ILogger log) {
    strive {
        if (enter != null && enter.Rely > 0) {
            Household household = null;
            utilizing(var cache = CacheManager.GetCache("myLocalCache")) {
                foreach(var doc in enter) {
                    household = new Household {
                        LastName = doc.GetPropertyValue < string > ("LastName"),
                            Id = doc.GetPropertyValue < string > ("Id")
                    };
                    cache.InsertAsync(household.Id, household);
                }
            }
        }
    } catch (Exception ex) {
        //ToDo
    }
}

Run the azure perform, go to your cosmosDB in Azure portal and modify the gathering, in my case I’ve up to date the LastName from the household assortment, the second if you replace the gathering the CosmosDB set off the azure perform which can Insert the up to date assortment within the cache utilizing InsertAsync perform.

Go to cache statistic and test the depend. 

It’s only a pattern to indicate you the best way to sync the CosmosDB with Ncache with the assistance of Azure perform. Not solely replace you possibly can seize any assortment add or delete in CosmosDB and sync along with your cache server. Exhausting delete isn’t supported in ComosDB however there may be workaround to do a delicate delete. Will see extra about syncing delete and add assortment with cache in my subsequent article. 

Abstract

We noticed what’s NCache and the best way to combine it with CosmosDB to cache the information. Later we went via the demo of CosmosDB sync with NCache server with the assistance of Azure capabilities with CosmosDB set off. We’ll see extra about Syncing cached information with Cosmos DB utilizing NCache with Customized change Feed processor in my subsequent article.

Get full supply code on GitHub