• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    公众号

Exchangis: Exchangis是微众银行开源的轻量级数据交换平台,用于数据在不同存储介质中 ...

原作者: [db:作者] 来自: 网络 收藏 邀请

开源软件名称:

Exchangis

开源软件地址:

https://gitee.com/WeBank/Exchangis

开源软件介绍:

License

English | 中文

Introduction

Exchangis is a lightweight,highly extensible data exchange platform that supports data transmission between structured and unstructured heterogeneous data sources. On the application layer, it has business features such as data permission management and control, high availability of node services and multi-tenant resource isolation. On the data layer, it also has architectural characteristics such as diversified transmission architecture, module plug-in and low coupling of components.

Exchnagis's transmission and exchange capabilities depend on its underlying aggregated transmission engines. It defines a unified parameter model for various data sources on the top layer. It maps and configures the parameter model for each transmission engine, and then converts it into the engine's input model. Each type of engine will add Exchangis features, and the enhancement of certain engine features will improve the Exchangis features. Exchangis's default engine aggregated and enhanced is Alibaba's DataX transmission engine.

Features

  • Data Source Management
    Share your own data source in a bound project;
    Set the external authority of the data source to control the inflow and outflow of data。

  • Muti-transport Engine Support
    Transmission engine scales horizontally;
    The current version fully aggregates the offline batch engine DataX and partially aggregates the big data batch derivative engine SQOOP

  • Near Real-time Task Control
    Quickly capture the transmission task log, transmission rate and other information, close the task in real time;
    Dynamically limit transmission rate based on bandwidth

  • Support Unstructured Transmission
    Transform the DataX framework and build a binary stream fast channel separately, suitable for pure data synchronization scenarios without data conversion。

  • Task Status Self-check
    Monitor long-running tasks and tasks with abnormal status, release occupied resources in time and issue alarms。

Comparison With Existing Systems

Comparison of some existing data exchange tools and platforms:

Function moduleDescriptionExchangisDataXSqoopDataLinkDBus
UIIntegrated the convenient management interface and monitoring windowIntegratedNoneNoneIntegratedIntegrated
Installation and deploymentEase of deployment and third-party dependenciesOne-click deployment, no dependenciesNo dependenciesRely on Hadoop environmentRely on ZookeeperRely on a large number of third-party components
Data authority managementMulti-tenant permission configuration and data source permission controlSupportNot supportNot supportNot supportSupport
Dynamic limit transmissionSupportPartially supported, unable to adjust dynamicallyPartially supported, unable to adjust dynamicallySupportSupport,with Kafka
Data transmissionUnstructured data binary transmissionSupport, fast channelNot supportNot supportNot support,only transport recordNot support,need to be converted to a unified message format
Embed processing codeSupport,dynamic compilationNot supportNot supportNot supportPartial support
Transmission breakpoint recoverySupport(Not open source)Not supportNot supportSupportSupport
High availabilityMutiple services, failure does not affect the useApplication high availability, transmission single point(Distributed architecture planning)Single point service(Open source version)Multipoint transmissionApplication、transmission high availabilityApplication、transmission high availability
System ManagementNodes、resources managementSupportNot supportNot supportSupportSupport

Overall Design

Architecture

Architecture

Documents

Quick Deploy
User Manual

Communication

If you desire immediate response, please kindly raise issues to us or scan the below QR code by WeChat and QQ to join our group:

Communication

License

Exchangis is under the Apache 2.0 License. See the License file for details.


鲜花

握手

雷人

路过

鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
热门推荐
热门话题
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服(服务时间 9:00~18:00)

在线QQ客服
地址:深圳市南山区西丽大学城创智工业园
电邮:jeky_zhao#qq.com
移动电话:139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap