Blog

MessagePack is more useful than XML or even JSON

category : Development

Today, I want to talk about exchanging data through the web that is related to the technology supporting Fenrir Pass Connect.

Unification of data structure > Implementation becoming unified

When data is sent between programs, devices, or via the web, it is necessary to send the data in a specified format.

For example, in the page being displayed now, the body uses HTML, the images uses a binary format such as PNG, linking with the web API is done using JSON or XML, and the stylesheets and JavaScript each use their own text format.
In general products, there are many occasions when XML or JSON is used out of the above.

Because these formats are in popular use, there is also a lot of information available, implemented processors are generally stable, so they are used in places in Sleipnir.

Increase in information used > Necessity for more efficient format

With the increase in things that can be done on the web, and improvement in network speeds, a difficult situation has started to arise for XML and JSON.

Even in Fenrir Pass Bookmark, there are many occasions when a few MB is used and just analyzing the syntax creates a tough situation for the mobile device.
If a binary format is created, the size and analyzing speed can be improved, but this would result in the need to write an implementation for all the processors which is definitely a tough road to climb.

Isn’t there anything that is common, can be easily used like JSON, and in a format that is small and processes at high speed?

This is where the magnificent MessagePack shines!

These problems can be solved in one go by using MessagePack!

Regarding the implementation of C++, serialization and deserialization can be directly performed in structures, and a template is used so that unnecessary copying is not performed resulting in high-speed that JSON can not come close to.

// Data structure

struct mystruct {

    std::string m_str;

    std::vector<int> m_vec;

    MSGPACK_DEFINE(m_str, m_vec);

};

int main(void) {

    mystruct original_data;

    mystruct clone_data;

	// Serialize

    msgpack::sbuffer sbuf;

    msgpack::pack(sbuf, original_data);

	// Deserialize

    msgpack::unpacked msg;

    msgpack::unpack(&msg, sbuf.data(), sbuf.size());

    msgpack::object obj = msg.get();

    obj.convert(&clone_data);

}

You can easily use it just like shown.

Furthermore, the data generated is also about half the size of that by JSON, but can create expressions that so flexible that they can be reciprocally changed with JSON.

{"a":null,"b":10,"c":[20],"d":"30"}  <= 35byte

       ↓

84 a2 61 c0 a2 62 0a a2 63 91 14 a2 64 a2 33 30 <= 16byte

The size is reduced by over half in this way.

Even the list of supported processors is impressive: Ruby, Python, Perl, C, C++, Java, Haskell, JavaScript, D, C#, Erlang, Scala, Go, Lua, node.js, OCaml.

Summary

Actually, Sleipnir 3 is already using MessagePack for areas that require speed, providing the application with a good feeling.
We are currently working towards further expanding the scope in which it is used to provide an ever better product. Please look forward to it.



Share This Article!
comments powered by Disqus