/
Conduit Services

Conduit Services

Owned by Chelsi Volpe (Unlicensed)
Last updated: May 17, 2023
2 min read

In order to see all services that make Conduit working, navigate to Settings -> Service Management.

 

TABLE OF CONTENTS

 

Conduit services

 

  • Conduit Storage - used for:

    • connector / datasource metadata and schemas

    • contains logs from Conduit server

    • uses Elasticsearch

  • Log Indexer 

    • logs indexer

    • uses LogStash

    • pushes logs from Conduit server to Conduit storage

    • one service per Conduit instance

  • Kibana

    • graphical interface to read / query logs from Conduit Storage

    • this is where we search for correlationIds

  • Graphite

    • service that stores metrics from Conduit server

      • e.g. how many times this method has been called

      • e.g. how much time does this method take

    • this runs on each Conduit server instance

  • Grafana

  • Zookeeper

    • contains configurations for Conduit

    • provides leader election mechanism 

      • for Conduit, for Spark

  • Conduit-Proxy

    • graphical interface for Conduit

  • Hive Metastore

    • used by Spark to validate query syntax

    • holds Spark tables, datasets schemas

  • Conduit Server

    • Conduit engine

    • here is where the connections to databases and virtualization takes place

  • Conduit Master

    • has Spark master

  • Conduit Worker

    • has Spark worker

 

Conduit ports

Conduit uses the following ports by default:

  • 80 - for http connections to Conduit Proxy

  • 443 - for https connections to Conduit Proxy

  • 10002 - for thrift connections to Conduit server

  • 22 - for ssh access to the virtual machine hosting Conduit services

 

Related pages