Part 3: Edge Integration

In parts 1 and part 2 we presented a federated security approach to authenticate users in multiple tenants using OAUTH2 and OIDC. In part 3 we will discuss how to connect on premises sensor systems to the cloud data services.

C-Insight is a multi-sensor system and our customers use on premises sensors for both safe and smart city infrastructure management. Sensors send operational data and health statuses. Operational data is stored in searchable databases, used to build BI dashboards and triggers various data analytics rules. Health statuses used to present real-time sensor health status dashboards and reports. In on-premises deployment scenario, none of these data leave data-center boundaries, however, in a hybrid deployment model, operational data and health statuses sent to cloud backend services for processing. This means that only a thin integration layer services are deployed on premise and data they generate travels through WAN to the cloud environment. This introduces a couple of challenges:

1. Security challenge since these services must be contacted from cloud datacenters and opening ports in a data-center firewall for in-bound communication is a security breach.
2. Deployment challenge since operations team have to manually configure IP addresses and ports for every on-premises location so they can be reached from cloud IP segment.
3. Communication challenge since data is sent over WAN needs protocols capable traversing firewalls.

We addressed these challenges using TCP based tunneling protocol.

From Wikipedia:

“In computer networks, a tunneling protocol is a communication protocol which allows for the movement of data from one network to another, by exploiting encapsulation. It involves allowing private network communications to be sent across a public network (such as the Internet) through a process called encapsulation.”

Each on premise deployment runs an Edge Integration Host (EIH from now on) service which capable of hosting any type of sensor integration (like LPR, Smart Light, Traffic Control, etc.). Once started, the service opens a secure bi-directional connection with a predefined address in the cloud data center where another service called Edge Integration Gateway (EIG from now on) is listening for such connections. Once secure channel is established its metadata is persisted and any service running in the cloud data-center can send queries and commands to the on premise EIH via this secure channel using any messaging protocol of choice and, since the channel is bi-directional, EIH can send sensor related events to the cloud. A single connection from every on premise data center is needed for the entire system operation and only one out-bound port must be opened in the on premise firewall.

Once a connection is established, JSON messages flow in both directions: either its events generated by sensors, or queries or commands sent by cloud deployed backend services. Each JSON message is digitally signed and encrypted using client certificate and the entire TCP communication is encrypted using server certificate.

Unlike client applications which use short lived tokens for calling APIs, in the edge integration gateway – host scenario we used session cookies for authentication. Each session is initiated by a connect request presenting client certificate and on-premises deployment metadata. Once authenticated, a session is created and stays active as long as connection is open and messages flow in either direction. If no messages are sent for a preconfigured timeout – the connection is terminated. This authentication mechanism is more suitable for server to server communication since no authorization takes place.

From cloud deployment perspective, the EIG deployed the same way API Gateway deployed (described in part 2). All connection requests reach L4 NLB which forwards the traffic to one of the healthy instance of EIG. None of the cloud based services need to know the EIH address in order to send messages to it, since the channel is identified by the customer’s deployment metadata.

This versatile and secure communication style allows to connect multiple on-premises locations seamlessly to form a massive multi-tenant hybrid cluster that serves unlimited number of on-premises customers.

That’s it for this series of articles. We would love to hear your thoughts on what you would like to hear form us next.