Data Integration with Ballerina

Ballerina is an open source, concurrent programming language which has both textual and graphical representation. It is mainly designed for seamless integration of networked applications. Ballerina is strongly and statically typed with powerful type system. [1]

The importance of data integration is apparent to anyone who’s spent time fetching information from multiple systems for a basic report. Data integration involves combining data from several disparate sources, which are stored using various technologies and provide a unified view of the data. Today data integration becomes increasingly important and integral part of any kind of business/process integration scenario. Data integration is a term covering several distinct sub-areas such as:

Ballerina language is specially designed for integration domain and it allows faster and easy data integration due to following reasons.

Ballerina Type System

In most of the traditional programming languages SQL result-sets, JSON data, XML data etc are not treated as first class types. When using or manipulating these data we have to use various external libraries or add-ons to get the work done. But ballerina has designed with a sophisticated type system with first class support for different data types and formats. So users can generate, manipulate, convert from one type to another easily and less number of code lines. Following are the basic types in Ballerina which is capable of handling different data types.

The table , json , xml , record types are highly useful when working with data integration scenarios. In Ballerina, table can be directly converted into xml or json type and table can be mapped into record types where each row of tabular data is mapped into record. Also json ,map, and record etc are inter-operable types and casting/converting allows transformation between these different types easily.

Connector Support for Various Data Sources

Ballerina client endpoints are used to connect with external entities or APIs. For data integration purposes ballerina provides several SQL and NoSQL endpoints to interact with tabular SQL and NoSQL data sources. Ballerina is equipped with following data endpoints as of now (Ballerina 0.970.0 version) Ballerina provide extension mechanisms for writing custom native/Ballerina client endpoints which can connect to any custom data sources if required.

Built-In Transaction Support

Ballerina transaction is a series of data manipulation statements that must either fully complete or fully fail, leaving the system in a consistent state. Ballerina language supports both local and distributed transactions for data and JMS connector actions. Ballerina provides syntax support for defining transaction boundaries and handling transaction failures and retries. [8]

Data Transformation Capabilities

The transformer syntax in ballerina is useful for having custom transformation between different types such as records and jsons. Together with the data casting/conversion functionality this becomes a key part in data integration scenarios.

Data Streaming Support

In Ballerina, table to json and table to xml type conversions are resulted in streamed data. With the data streaming functionality, when a service client makes a request, the result is streamed to the service client rather than building the full result in the server and returning it. This allows virtually unlimited payload sizes in the result, and the response is instantaneous to the client. There the result set corresponding to a particular query is converted to XML/JSON row by row and written to the wire as the conversion takes place upon a row.

Graphical Data Modeling With Composer

Composer is a tool to edit ballerina programs both in graphically and textually. The visual representation of ballerina is based on sequence diagram model and it helps the developer to have a clear view on the entire data integration flow.

Ability to Expose Data as Services via HTTP Service

Success of a business lies in its ability to integrate its data from across the organization and analyze it to make more informed decisions. So accessing data in a convenient way is a key requirement in any data integration scenario. APIs make this data exposure possible and REST is one of the most popular APIs to communicate with web, mobile and cloud apps. With the rich, fast and easy HTTP REST service development support in Ballerina, it allows rapid data services exposure via REST APIs.

References:

Related posts:

This makes sense. There isn’t much else to do, so we turn to our devices. Meaning that screen time has increased for many of us, as well as social media usage. Not only are we relying on it for…

You make babies; you make up stories; you make trouble. You create an atmosphere around you that either delights or repels people. Most of us are waterfalls of random, unconscious creativity…

At the USOC symposium I was scared, nervous, and self-conscious. Most conferences I had been to resulted in me being timid. I remember even being to nervous to introduce myself to Dan John. A guy who…