CryptAnalyzer

Quick-view

>A platform designed by involving multiple crypto exchanges for the real-time analysis, in order to satisfy the bulk number of the request sent by customers from per exchange. In brief, cryptocurrency analysis will be performed in a system through crypto exchanges.  

Challenges

Our client demanded us to create a platform that can fetch all the real-time information of cryptocurrencies – order book( buy/sell), transaction (trading), and exchange price ticker values(current market price), and then buyers can purchase as well as sell the cryptocurrency from each by comparing them.

Maintaining every single microservices simultaneously was confronting tasks for our development team. Massive database synchronization was particularly an extremely challenging part, as crypto-exchange prices fluctuate within seconds. Along with the exchange prices, representing their graphs in a real-time manner, occupied our team for a long time.

Solution

We understood the requirements of our client and then concluded an intelligent solution implementing Kafka along with Kafka stream, which will assist the system to complete multiple tasks in minimum latency time. This solution has a benefit, it enables us to perform both the operations, read and write at an incredible speed which is necessarily needed. 

CryptAnalyzer holds variable functionalities, thus consist microservices of exchange clients, notification services, public data gatherer, user wallet management, logging and audit trails services, transaction service, utils, and spring batch services for schedule backup.

• For this complicated system, we required the entire system to work in real-time, as the exchange prices fluctuate in the intervening time. Thus, the execution is performed via the REST API (Using pooling client) or WEB Socket for communication between exchange to our system.
• According to the requirements, the system demands a database management system, with a property of scalability and retention policy, which can store stacks of data in limited-time series. In this case, Influx DB is the savior key. Within Influx DB, one can configure the number of days, so that the preceding data gets wiped out when the duration will expire.
• Also, we have used both InfluxDB as well as PostgreSQL databases for data handling purposes but separately. For storing the exchange database in a real-time manner we use time-series database Influx DB. The graph will be generated in order to demonstrate transaction, ticker value and order book from the Influx DB database we use Grafana to display graphs and for the permanent storage of user data, wallet info, transactions we executed Postgres DB.
• To maintain the day-to-day (every day) backup, here the Spring batch job is implemented to store the real-time data, via currency pair as well as per exchange. We manipulate data and store everyday data for each currency pair and for Each exchange, number of  CSV generated in backup and upload on s3 bucket.
• Spring Batch job generates CSV and its store in  Amazon S3. now that data will be used as raw material for price prediction. It will be used for data analysis and used for future price prediction.
• Another important purpose is to keep the logstash of all the microservices to collect, parse, and store the logs for the future, so we created a log management microservice. We used an elastic search for the storage functionality and Kibana for virtualization.
• Besides, we enabled the user wallet management system, which will provide maintenance of the user wallet and its own spring boot microservice. Also, this specific added feature will allow the developers to manage the account details such as user identity, position, and the balance in the account accordingly.
• For adding high-level security and authorization, we chose Kong as API Gateway because it contains strong prebuilt plugins. Every request is being filtered by kong gateway.

Solution Architecture

Result :

Crypt-Analyzer, a cryptocurrency analysis platform has a high-level architecture with multifunctional responsibility. In a short timeframe, this platform will analyze 44 crypto-exchanges data such as order book, transactions i.e; trading, as well as exchange volumes, and store their real-time information to create a backup for future prediction of cryptocurrency.