Sunday, June 26, 2011

JSON Compression algorithms


JSON (Java Script Object Notation) is a lightweight data-interchange format. It is easy for humans to read and write. It is easy for machines to parse and generate. It can be used as a data interchange format, just like XML. When comparing JSON to XML, it has several advantages over the last one. JSON is really simple, it has a self-documenting format, it is much shorter because there is no data configuration overhead. That is why JSON is considered a fat-free alternative to XML.

However, the purpose of this post is not to discuss the pros and cons of JSON over XML. Though it is one of the most used data interchanged format, there is still room for improvement. For instance, JSON uses excessively quotes and key names are very often repeated. This problem can be solved by JSON compression algorithms. There are more than one available. Here you'll find an analysis of two JSON compressors algorithms and a conclusion whether JSON compression is useful and when it should be used.

Compressing JSON with CJSON algorithm

CSJON compress the JSON with automatic type extraction. It tackles the most pressing problem: the need to constantly repeat key names over and over. Using this compression algorithm, the following JSON:

  { // This is a point
    "x": 100, 
    "y": 100
  }, { // This is a rectangle
    "x": 100, 
    "y": 100,
    "width": 200,
    "height": 150
  {}, // an empty object
Can be compressed as:

  "templates": [ 
    [0, "x", "y"], [1, "width", "height"] 
  "values": [ 
    { "values": [ 1,  100, 100 ] }, 
    { "values": [2, 100, 100, 200, 150 ] }, 
The more detailed description of the compression algorithm, along with the source code can be found here:

Compressing JSON with HPack algorithm

JSON.hpack is a lossless, cross language, performances focused, data set compressor. It is able to reduce up to 70% number of characters used to represent a generic homogeneous collection. This algorithms provides several level of compression (from 0 to 4). The level 0 compression performs the most basic compression by removing keys (property names) from the structure creating a header on index 0 with each property name. Next levels make it possible to reduce even more the size of the JSON by assuming that there are duplicated entries.

For the following JSON:

  name : "Andrea",
  age : 31,
  gender : "Male",
  skilled : true
}, {
  name : "Eva",
  age : 27,
  gender : "Female",
  skilled : true
}, {
  name : "Daniele",
  age : 26,
  gender : "Male",
  skilled : false
the hpack algorithm produces a compressed version which looks like this:

More details about hpack algorithm can be found at project home page.


The purpose of this analysis is to compare each of the described JSON compressor algorithms. For this purpose we will use 5 files with JSON content having different dimensions, varying from 50K to 1MB. Each JSON file will be served to a browser using a servlet container (tomcat) with the following transformations:

  • Unmodified JSON - no change on the server side
  • Minimized JSON - remove whitespaces and new lines (most basic js optimization)
  • Compressed JSON using CJSON algorithm
  • Compressed JSON using HPack algorithm
  • Gzipped JSON - no change on the server side
  • Gzipped and minimized JSON
  • Gzipped and compressed using CJSON algorithm
  • Gzipped and compressed using HPack algorithm


This table contains the results of the benchmark. Each row of the table contains one of the earlier mentioned transformation. The table has 5 columns, one for each JSON file we process.
json1 json2 json3 json4 json5
Original JSON size (bytes) 52966 104370 233012 493589 1014099
Minimized 33322 80657 180319 382396 776135
Compress CJSON 24899 48605 108983 231760 471230
Compress HPack 5727 10781 23162 49099 99575
Gzipped 2929 5374 11224 23167 43550
Gzipped and Minimized 2775 5035 10411 21319 42083
Gzipped and compressed with CJSON 2568 4605 9397 19055 37597
Gzipped and compressed with HPack 1982 3493 6981 13998 27358

Relative size of transformations(%)

The relative size of transformation graphic is useful to see if the size of the json to compress affects the efficiency of compression or minimization. You can notice the following:
  • the minimization is much more efficient for smaller files. (~60%)
  • for large and very large json files, the minimization has constant efficiency (~75%)
  • compressors algorithms has the same efficency for any size of json file
  • CJson compressing algorithm is less efficient (~45%) than hpack algorithm (~8%)
  • CJson compressing algorithm is slower than hpack algorihtm
  • Gzipped content has almost the same size as the compressed content
  • Combining compression with gzip or minimization with gzip, doesn't improve significantly efficiency (only about 1-2%)


Both JSON compression algorithms are supported by wro4j since version 1.3.8 by the following processors: CJsonProcessor & JsonHPackProcessor. Both of them provide the following methods: pack & unpack. The underlying implementation uses Rhino engine to run the javascript code on the serverside.

JSON Compression algorithms considerably reduce json file size. There a several compression algorithms. We have covered two of them: CJson and HPack. HPack seems to be much more efficient than CJson and also significantly faster. When two entities exchange JSON and the source compress it before it reach the target, the client (target) have to apply the inverse operation of compression (unpacking), otherwise the JSON cannot be used. This introduce a small overhead which must be taken into account when deciding if JSON compression should be used or not.

When gziping of content is allowed, it has a better efficiency than any other compression algorithm. In conclusion, it doesn't worth to compress a JSON on the server if the client accept the gzipped content. The compression on the server-side does make sense when the client doesn't know how to work with gzipped content and it is important to keep the traffic volue as low as possible (due to cost and time).

Another use-case for JSON compression algorithm is sending a large JSON content from client to server (which is sent ungzipped). In this case, it is important to unpack the JSON content on the server before consuming it.


  1. This site use full to all around the would, use it develop both.........
    this site use of all the would and system news and helps known every one...........
    System Optimization

    1. I really appreciate you for all the valuable information about algorithms you are providing us through your blog.

      SEO Company New York

    2. Hi, Great.. Tutorial is just awesome..It is really helpful for a newbie like me.. I am a regular follower of your blog. Really very informative post you shared here. Kindly keep blogging. If anyone wants to become a Front end developer learn from Javascript Training in Chennai . or learn thru JavaScript Online Training in India. Nowadays JavaScript has tons of job opportunities on various vertical industry. JavaScript Training in Chennai

  2. Very cool. I just heard of MesssagePack today, and decided to look around and see what else was available out there. I hadn't ever thought about optimizing my JSON because I assumed since it was so much more efficient than XML, it was good enough. Thanks again.

  3. I have been looking around for this kind of information. Will you post some more in future? I’ll be grateful if you will.
    SEO Services New York

  4. Appreicate your thoughts, Im not always in agreement, but you do cause a peron to think keep blogging!carpet cleaning adelaide

  5. Appreicate your thoughts, Im not always in agreement, but you do cause a peron to think keep blogging!carpet cleaning adelaide

  6. If truth be told informative and valuable detail is here.
    air compressor best price

  7. An unbelievable blog. This blog will indisputably be definitely recommended to my friends as well.
    Joycelyn Corria

  8. Would you please consider removing the spam comments? They're a distraction, and they encourage spammers to keep spamming the rest of our blogs.

    Otherwise, thanks for the JSON compression comparisons.

  9. Optimising JSON is also of interest to me (see for an opensource Java API I've developed) I'm experimenting with eliminating repeated keys using embedded tags and templating, but also using tagged string references so that examples such as [ "a repeated string", "a repeated string" ] is mapped to [ "@1:a repeated string", "@1" ] which can be easily mapped back to the original text.

    It appears my approach is mostly aligned with the hpack algorithm for optimising keys. I'd be interested to obtain a copy of the 5 json texts used in the results above so that I can see what figures I can produce.

    1. A quick follow up with at least one test result, using the json.hpack test data I discovered by following the above link (the file 5000.txt) I'm getting the following test result using my 'ojson' API:

      json.hpack-5000: Reduction: 76.0% from [776133] to optimised size [186598]

      I'm also mapping the 'ojson' output back to plain json and using this java package:


      to verify the original hpack json is indeed being restored (from json -> ojson -> newJson):

      JSONAssert.assertEquals(json, newJson, JSONCompareMode.STRICT);

      Hope this is of interest.

  10. Web site optimization produces highly competitive web sites that out-perform on every measure; traffic, speed, conversion rates, sell-throughs, and, most importantly, return on your investment.Thanks for your useful information.
    Web Developers Adelaide

  11. Thank you for this amazing Json tutorial. Very helpful. Check Our Service Below. Hope it will you too. Thanks
    SEO Services
    Local SEO Services
    Link Building Services
    Web Design & Development Services
    WordPress Development Services

  12. Thank you for sharing this informative and interesting article. Keep up the good work!

    Melbourne seo services

  13. I read this article. I think You put a lot of effort to create this article. I appreciate your work.
    thesis Writing Service

  14. Nice post. By reading your blog, i get inspired and this provides some useful information. Thank you for posting this exclusive post for our vision. 
    Microsoft azure training in Bangalore
    Power bi training in Chennai

  15. Good job in presenting the correct content with the clear explanation. The content looks real with valid information. Good Work

    DevOps is currently a popular model currently organizations all over the world moving towards to it. Your post gave a clear idea about knowing the DevOps model and its importance.

    Good to learn about DevOps at this time.

    devops training in chennai | devops training in chennai with placement | devops training in chennai omr | devops training in velachery | devops training in chennai tambaram | devops institutes in chennai | devops certification in chennai | trending technologies list 2018

  16. Such a wonderful blog on Machine learning . Your blog have almost full information about Machine learning .Your content covered full topics of Machine learning that it cover from basic to higher level content of Machine learning . Requesting you to please keep updating the data about Machine learning in upcoming time if there is some addition.
    Thanks and Regards,
    Machine learning tuition in chennai
    Machine learning workshops in chennai
    Machine learning training with certification in chennai

  17. you exlained very well abouut the resource.thanks for sharing this useful infromation.good work.keep going.

    Apple service center in Chennai
    Apple service center

  18. Nice post. Thanks for sharing! I want people to know just how good this information is in your article. It’s interesting content and Great work.
    Thanks & Regards,
    VRIT Professionals,
    No.1 Leading Web Designing Training Institute In Chennai.

    And also those who are looking for
    Web Designing Training Institute in Chennai
    SEO Training Institute in Chennai
    Photoshop Training Institute in Chennai
    PHP & Mysql Training Institute in Chennai
    Android Training Institute in Chennai

  19. Nice post!Everything about the future(học toán cho trẻ mẫu giáo) is uncertain, but one thing is certain: God has set tomorrow for all of us(toán mẫu giáo 5 tuổi). We must now trust him and in this regard, you must be(cách dạy bé học số) very patient.

  20. very good post!!! Thanks for sharing with us... It is more useful for us...
    Selenium Training in Chennai | SeleniumTraining Institute in Chennai

  21. Time is free but it's priceless(khóa học toán tư duy) . You cannot own it, but you can use it(cách dạy bé học số) . You can use it, but you can't keep it(toán tư duy logic là gì). Once you lose it, you will not be able to get it back.

  22. A Computer Science portal for geeks. It contains well written, well thought and well explained computer science and programming articles, quizzes and practice/competitive programming/company interview Questions.

  23. A Computer Science portal for geeks. It contains well written, well thought and well explained computer science and programming articles, quizzes and practice/competitive programming/company interview Questions.