What’s the idea of synchronization? How caching works? How data consistency is kept in concurrent environment? How to implement master-master synchronization?

Cache

Srv4pos follows w3c recommendations. It means you can use Expires, If-None-Match or If-Modified-Since headers. We recommend you to check out this excellent web site that explains it.

Seller Version is a source data for entity tags. It is an autoincrement number starts from 0 and +1 each time you do a change on the server. So every change get own unique ordinal number within the same seller. Each Seller makes own counting. For instance:

Let’s imagine there are two users: James and Matt. James did the actions described in the table above. And Matt does some reading. Here are few examples how Matt can use cache:

• James just added product FRUIT
• Matt requested list of available products and got one, then he requested this product and got FRUIT. This product was accompanied by response header ETag with value “1”. Because Seller version was “1”. Matt’s browser understood it’s tag and kept the response body in browser’s cache.
• Later on Matt opened up product FRUIT and his browser sends If-None-Match: “1” in the request. Srv4pos knows that nothing has been changed so far therefore Srv4pos responds with HTTP status 304 and empty body which forces Matt’s browser to open product FRUIT from it’s cache without any need to download the actual content.
• James made some changes, especially renamed the product to FRUIT_GREEN.
• Matt request product FRUIT again, accompanied by If-None-Match: “1”. But now server detects that the product’s ETag is “3”, not “1”. Which means it has been changed. Threfore server returns FRUIT_GREEN with ETag “3”. Matt’s browser downloads the changes and invalidates the cache with FRUIT_GREEN and ETag “3” for future use.

Similar situation with images:

• James uploaded picture MAIN.JPG Mon, 16 Mar 2015 18:59:14 GMT
• Matt opened up a picture which was returned with response tag Last-Modified: Mon, 16 Mar 2015 18:59:14 GMT. It was saved to the browser’s cache.
• Matt opened up a picture again, now the browser sends If-Modified-Since: Mon, 16 Mar 2015 18:59:14 GMT to the server. And server responds 304. Which forces browser to open up image from it’s cache.
• James updated the picture MAIN.JPG Mon, 16 Mar 2015 12:11:12 GMT
• Matt opened up a picture again, now the browser sends If-Modified-Since: Mon, 16 Mar 2015 18:59:14 GMT to the server. Server compares the date with the latest date and understands there was a change made by James. Server returns actual content back to Matt.
• Matt’s browser downloads the image, displays a new one and invalidates the cache.

Headers support:

1. JSON data (like info about taxes, products and etc)
   • ETag, If-None-Match (limited, see below)
2. Pictures
   • ETag, If-None-Match
   • Last-Modified, If-Modified-Since
3. Historical information about pictures. Srv4pos keeps all the versions of every picture uploaded. First James uploaded MAIN.JPG, then he updated it. So there are two different pictures: old one and new one. For the old picture Expires header is supported
   • Expires, default 24 hours
   • ETag, If-None-Match
   • Last-Modified, If-Modified-Since

JSON data has limited ETag support compared to Pictures. It’s better to show on example:

• James created FRUIT and Matt downloaded it. Now Matt knows ETag is “1”.
• James uploaded picture, but not yet changed FRUIT to FRUIT_GREEN. Seller version becomes “2”.
• Matt requested FRUIT with If-None-Match: “1”. Server compares “1” with current seller version “2”, they are not equal and server thinks FRUIT has been changed, but it’s wrong! Instead it was a picture uploaded. Server responds HTTP status 200 instead of 304. Though it’s reasonable to return 304 because the product is not yet changed.

But pictures do not have this limitation:

• James uploaded MAIN.JPG so seller version is “2”.
• Matt dowloads the picture accompanied with ETag “2”.
• James renames the product to FRUIT_GREEN and seller version becomes “3”.
• Matt opens the picture accompanied with If-None-Match “2”. Server works correctly here because in case of pictures, server does not look into global value of seller version, instead server discovers the specific seller’s version of the picture, which is still “2” and responds 304 correctly.

The limitation for JSON data is done in favour of performance. ETag for JSON data is very convenient for downloading big lists, such as GET /{sellerId}/products when you get big amount of data and does not really matter for single entityes such as GET /{sellerId}/products/{identifier}.

In general we recommend you to enable caching both for JSON data and Pictures.

In case if something is still unclear, here is some more samples:

For requests on JSON data:

For requests on picture:

For JSON data current version of seller is shared during the cache validation, but cache validator for pictures looking for their own version, which is technically is a “snapshot” of seller version after the last modification onto picture in the past.

If you are java developer then use parameter Integer version same as ETag, and treat returned value “null” as 304 HTTP status code.

Bird’s eye view for the Synchronization

The main idea of Synchronization is to establish a distributed environment with at least two different storages. One storage is on your side (e.g. database in your server, or database inside your POS), another storage at our server (cloud or local, doesn’t matter). And the requirement here: two databases are mostly contain equal set of data. Once it’s detected they are not equal the magic process of synchronization is running. And voila! They become equal!

Storage at your side is Client. Storage at our side is Server. The algorithm is very similar to any Version Control Software:

• To detect are the storages equal or not. Seller version is used. It’s an ordinal number and incremented each time some change happens in the data.
• Synchronization starts when: a) client polls up a server for “Server’s Seller Version” and it is different from “Client’s Seller version”. b) new data appears on Client. (We are working right now on some way to notify Client that “Server’s Seller Version” is changed.)
• First, Client must download changes from the Server and merge them into local storage somehow. It’s up to you to define your own merge strategy.
• Then, Client uploads his local changes up to the Server.

Master-slave synchronization algorithm

1. Get latest seller version number using /{sellerId}/version or getThisVersion(…) .
2. Compare with your local seller version (if it’s not yet set, set it to zero by default).
3. If they don’t match, request all the changes between your local version and server version via /{sellerId}/modifications or getMidifcations(…) . And save them to local database.

If you want to synchronize single entity, e.g. taxes, this request can be made via /{sellerId}/taxes-diff/{version-from}/{version-to} or listDiff(…) . Every entity has own “diff” method.