# 🪐 What is MooreData Engineering Platform

> 1. ABAVA means AI-Based Augmented Virtual Associate, this also represents the automated data enginneering idea of ABAKA AI data engineering platform&#x20;
> 2. Please contact `business@molardata.com` for the latest updates on the platform&#x20;
> 3. To experience our platform, you can log in [ABAKA AI](https://app.molardata.com/dashboard)（Click the button in the uper right corner to switch to the English version）

## Why were ABAVA Platform created?

When it comes to data annotation tools, you may think of open source software such as `labelme` or `labelImg`. Open source annotation software is sufficient to meet the data processing needs of 1.0 era (workload of 10,000)

### From Level 1 to Level 2

With the deepening of the neural network, the AI model needs more data to improve the model training effect (workload of 100,000) , and the data annotation has entered the 2.0 era. The original stand-alone annotation tools can no longer meet the needs of this era. So the data annotation platform came out. Through the process-based platform operation, the collaboration of the data production link is realized, and the support for the crowdsourcing model enables elastic expansion of data production capacity.

### From Level 2 to Level 3

With the in-depth implementation of AI in vertical industries, the magnitude of data processing is even larger, often on the order of 100,000 or even 1 million, and the 3.0 era has gradually begun. The data in this era is massive, and the requirements for data accuracy are also higher. It is difficult to meet the needs of this era by relying only on manual labeling and review. Therefore, the human-machine collaborative data production platform based on AI-assisted systems has meet the needs of this era.\
In the process of providing high-quality data services for enterprises and institutions in the AI field for a long time, we have **combined years of experience in data project management to integrate a series of best solutions into the management module of the platform**. At the same time, we invented a more efficient data annotation tool. In order to pursue the ultimate labeling efficiency, we have also\*\* integrated AI algorithms into the annotation tools in some scenarios\*\*, so that machines and annotators can collaborate to complete data annotation. It's also meet the needs of the data platform in the 3.0 era.\
Next, let's take a look at the useful functions of ABAVA platform ; )

<figure><img src="https://4019296664-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Ff4XGgKGqYBCSxddFtX2k%2Fuploads%2FI41IBHMm8cwyYVqGLzrA%2Fyuque_diagram%20(6).jpg?alt=media&#x26;token=b3085923-f057-443a-8370-6dc873e2602f" alt=""><figcaption></figcaption></figure>

## Functions introduction

### Data annotation module

We provide processing tools for common data types: image annotation tools, point cloud annotation tools, text annotation tools, audio annotation tools, multi-modal annotation tools.

#### Image annotation tools

Including `Image classification`、`Key-point annotation`、`2D line annotation`、`2D box annotation`、`2D semantic segmentation`、`Cuboid annotation`、`OCR annotation`、`Multi-frame annotation`, etc.<br>

<figure><img src="https://4019296664-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Ff4XGgKGqYBCSxddFtX2k%2Fuploads%2F8SYgt84pRK9sVWhvzxag%2Fimage.png?alt=media&#x26;token=af4287b0-7e64-4a6f-9731-eac2dccc0b3f" alt=""><figcaption></figcaption></figure>

#### Point cloud annotation tools

Including `3D box annotation` 、`3D semantic segmentation` 、`3D lane line annotation`、`2D/3D fusion annotation`, etc.

<figure><img src="https://4019296664-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Ff4XGgKGqYBCSxddFtX2k%2Fuploads%2FZ1HjMPxZJbDBDnZXhx2H%2Fimage.png?alt=media&#x26;token=02f11002-94b1-478e-8eae-4835f12f7663" alt=""><figcaption><p>3D point cloud box annotation</p></figcaption></figure>

![3D point cloud semantic segmentation](https://static.dingtalk.com/media/lALPM4OspXl-k4jNBE3NB68_1967_1101.png)

<figure><img src="https://4019296664-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Ff4XGgKGqYBCSxddFtX2k%2Fuploads%2FnwuybWqBJDjzUGOjmxGP%2Fimage.png?alt=media&#x26;token=665d8a4e-792d-4bd6-81a0-12f9d42deb62" alt=""><figcaption><p>3D point cloud lane line annotation</p></figcaption></figure>

<figure><img src="https://4019296664-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Ff4XGgKGqYBCSxddFtX2k%2Fuploads%2FbDQ5UgDqbTDztzEIRa8G%2F4e4801fe7d6da8b1c607b21bed435883.png?alt=media&#x26;token=1a783ec5-9da6-4626-b2b9-ac4cb9a0e487" alt=""><figcaption><p>2D/3D fusion annotation</p></figcaption></figure>

#### Text annotation tools

Including `NER named entity recognition`、`SPO text triplet annotation`, etc.

<figure><img src="https://4019296664-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Ff4XGgKGqYBCSxddFtX2k%2Fuploads%2Fp22QjEzVFT1nKbX9kQK4%2Fimage.png?alt=media&#x26;token=563002bc-c243-44da-bcdb-49b1659a3798" alt=""><figcaption><p>SPO text triplet annotation</p></figcaption></figure>

<figure><img src="https://4019296664-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Ff4XGgKGqYBCSxddFtX2k%2Fuploads%2FYGLwDZxLBTKUzp1s9Rs7%2Fimage.png?alt=media&#x26;token=e5fca90c-b209-4deb-bf3f-e41e3c6c0c62" alt=""><figcaption><p>NER named entity recognition</p></figcaption></figure>

#### Audio annotation tools

Including `ASR annotation`、`Phoneme annotation`, etc.

<figure><img src="https://4019296664-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Ff4XGgKGqYBCSxddFtX2k%2Fuploads%2FqqmPQWpWWPYIdE0vX0Pg%2Fimage.png?alt=media&#x26;token=b9d4262a-ecbf-4d46-9751-d8668b60df16" alt=""><figcaption><p>SPO text triplet annotation</p></figcaption></figure>

<figure><img src="https://4019296664-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Ff4XGgKGqYBCSxddFtX2k%2Fuploads%2FutloEsIlYAAOAixDMP1h%2Fimage.png?alt=media&#x26;token=d8cb8c45-dcd6-404d-b35f-2e1d3eed35bd" alt=""><figcaption><p>ASR annotation</p></figcaption></figure>

#### Other annotation tools

We supports `4D annotation`、`ACL multi-mode annotation`,etc.，In addition, we also supports customized development of other annotation tools. If you have any needs, you can contact `business@molardata.com`for specific consultation.&#x20;

<figure><img src="https://4019296664-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Ff4XGgKGqYBCSxddFtX2k%2Fuploads%2FvjxivSduEE1H6WIXXPGE%2Fimage.png?alt=media&#x26;token=c6edfffb-a0e1-42aa-95d4-881953ff2189" alt=""><figcaption><p>ACL multi-mode annotation tool</p></figcaption></figure>

### Platform management module

#### The best solution for data production

<figure><img src="https://4019296664-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Ff4XGgKGqYBCSxddFtX2k%2Fuploads%2FBDTCAhG6lP69MC6mcK5a%2Fimage.png?alt=media&#x26;token=9829569a-8fdc-4b60-b25e-616e88e1a712" alt=""><figcaption></figcaption></figure>

**Delivery** is the keyword of data production.\
In the optimal data production solution of ABAKA AI, we regard **the batch as the smallest unit of data delivery**. In a data task, **each data batch follows the same process for data production.**\
Taking the following data production process as an example, we have created a 4-node process. Node 1 carries out the labeling process, and nodes 2, 3, and 4 are all review processes. Only one team participates in each node. We will assign node 2 to the supplier team to complete the self-inspection, and node 3 will be assigned to our self-built quality inspection team to complete. Finally in node 4 , we will let the customer do the final quality inspection.

<figure><img src="https://4019296664-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Ff4XGgKGqYBCSxddFtX2k%2Fuploads%2F4VtO9yyTOa26x3dVcxva%2Fyuque_diagram%20(7).jpg?alt=media&#x26;token=11c72ee4-583e-4f7e-b8d2-95222eebb30b" alt=""><figcaption></figcaption></figure>

For example, this process is **like reimbursement process in the company**. The first node is for you to submit the reimbursement application. The second node is for your superior to review, if it is not passed, it will be called back to you for modification, if it is passed, it will enter the third node. The third node is for the review by the manager , if it is not passed, it will be called back to you for modification, if it is passed, it will enter the fourth node. The fourth node is the final node, if it is passed, your reimbursement will come down, if it is not passed, it will be called back and start the process again.\
**You can choose a process template for data production according to your actual needs**, such as 3 nodes (corresponding to double audits) or 5 nodes (corresponding to four audits).

<figure><img src="https://4019296664-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Ff4XGgKGqYBCSxddFtX2k%2Fuploads%2FXJbpRFg7ppukfpfo6ooi%2Fimage.png?alt=media&#x26;token=bffa579e-0e63-407e-9ef8-53f6aaa8ea63" alt=""><figcaption><p>In the batch report, the working status of each member is also statistically analyzed</p></figcaption></figure>

In the batch report, the working status of each member is also statistically analyzed\
After each node ends, **the system will automatically count and analyze the behavior data on the node**. As the manager of this data task, we can visually see the sampling accuracy of each node. In addition, we can also **know many secrets of the data batch by viewing the details**, which is of great significance for subsequent data production.\*\* \*\*For example, in the batch report, we can know which labeler has the lowest labeling accuracy, then by canceling the labeler's permission to continue annotation, the overall quality of the data can be improved.

#### Platform management submodule

The platform management function of ABAKA AI includes 3 major sub-modules: task management function module, team management function module, statistical analysis function module.

<figure><img src="https://4019296664-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Ff4XGgKGqYBCSxddFtX2k%2Fuploads%2FY2we8863SOpbG3sxMYUQ%2Fimage.png?alt=media&#x26;token=7ae4526f-d7a5-48bd-8f5d-0db96eb9e84a" alt=""><figcaption></figcaption></figure>

### AIPower function module

On ABAVA data management platform, there is a powerful system that can improve the efficiency of annotation and review, that is, **the AIPower closed-loop system**. "Closed loop" is a system that can be self-iterative and strengthened.

<figure><img src="https://4019296664-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Ff4XGgKGqYBCSxddFtX2k%2Fuploads%2F6oR3SkOP6LCeWLXqBnVG%2Fimage.png?alt=media&#x26;token=255bb638-edae-4183-8b47-415f6c2a8896" alt=""><figcaption></figcaption></figure>

The AIPower module can provide AI auxiliary functions including AI intelligent labeling and AI intelligent review:

* AI annotation：Before the labeling starts, the algorithm completes the algorithmic reasoning process on a whole batch of data, and obtains the pre-labeling results of a whole batch of data.
* AI audit：After the labeling is completed, the algorithm completes the algorithmic reasoning process for a whole batch of data, compares the actual labeling results with the algorithm review results, and feeds back data with a high possibility of errors.

The AIPower module can also support the iteration of the algorithm model with newly obtained datasts, and provide more accurate AI annotation and AI audit capabilities in the next batch of data production process.

{% embed url="<https://lark-video.oss-cn-hangzhou.aliyuncs.com/inputs/prod/yuque/2023/34849470/mp4/1681891083640-3ddee85c-feef-44aa-afd1-2bcc028b65cd.mp4?Expires=1715243590&OSSAccessKeyId=LTAI4GKnqTWmz2X8mzA1Sjbv&Signature=dAapxeEEz+Og7smKx/rsV9kFFdM=>" %}

{% embed url="<https://lark-video.oss-cn-hangzhou.aliyuncs.com/inputs/prod/yuque/2023/34849470/mp4/1681891083640-3ddee85c-feef-44aa-afd1-2bcc028b65cd.mp4?Expires=1715243590&OSSAccessKeyId=LTAI4GKnqTWmz2X8mzA1Sjbv&Signature=dAapxeEEz+Og7smKx/rsV9kFFdM=>" %}

#### AI annotation function

AI pre-labelling：Before the labeling starts, the algorithmic system completes the algorithmic reasoning process of the whole batch of data, and obtains the pre-labeling results of the whole batch of data. On this basis, the annotators only make manual revisions and fine-tuning, which can increase the efficiency by 2-10 times.\
AI co-labelling：During the labeling process, the labeler completes a small amount of labeling, and then the algorithm completes the remaining labeling work for this piece of data, and the labeling efficiency can be increased by 2-5 times.

{% embed url="<https://lark-video.oss-cn-hangzhou.aliyuncs.com/inputs/prod/yuque/2023/34849470/mp4/1682233655891-ced77019-1019-48f1-842f-75f22f90dc1d.mp4?Expires=1715243591&OSSAccessKeyId=LTAI4GKnqTWmz2X8mzA1Sjbv&Signature=4xBnFXgi+tq+xzDRUJ3IppgE3Ak=>" %}

#### AI review function

After the labeling finishes, the algorithmic system completes the algorithmic reasoning process of the whole batch of data, and compare the actual labeling results with the algorithm review results, then feedback the data with a high possibility of errors for review by reviewers, which will help reduce the error rate by more than 20%.

## How to manage data program from 0 to 1 ?

The scientific data production process design of ABAVA platform allows a user who has no experience in data project management to easily manage and achieve high-quality data production.

## How do we secure data ?

ABAVA Platform will protect your data from following aspects:

* Access control: all resource data is read and written privately to protect bucket information and prevent it from being stolen
* STS temporary permissions: fine-grained permission control through STS
* Referer anti-leech: a mechanism to set a whitelist for access sources to prevent OSS resources from being stolen by others
* Support data disaster recovery, CDN acceleration, and flow control capabilities

ABAVA platform has also passed ISO/IEC 27001:2013、information security management system certification and ISO 9001:2015 quality management system certification. In addition, ABAKA AI is also a member of the "Data Security Promotion Initiative" (DSI) of the China Academy of Information and Communications Technology, helping the construction of data security and jointly promoting the implementation of the "Data Security Law" with a number of companies.<br>

<figure><img src="https://4019296664-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Ff4XGgKGqYBCSxddFtX2k%2Fuploads%2FNffUvjPsAqsQqoVSR2lb%2Fimage.png?alt=media&#x26;token=d398d6b5-c792-4333-8fca-80ab064250b8" alt=""><figcaption><p>ISO9001 &#x26; ISO27001  certification</p></figcaption></figure>

## How to get our data services and tools ?

Our self-developed ABAVA Platform can be provided through SaaS and privatization deployment.\
Regarding privatization deployment, you can leave your contact information to`business@molardata.com` , and we will have a dedicated data expert to provide you with a free 1-to-1 consultation.\
In addition, we can also provide you with ACE services.

<figure><img src="https://4019296664-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Ff4XGgKGqYBCSxddFtX2k%2Fuploads%2FrYw1VaoCHqllFC2PZBx5%2Fimage.png?alt=media&#x26;token=78abf00f-7bc6-4d75-aff2-1023da99df47" alt=""><figcaption></figcaption></figure>

## ABAKA AI

ABAKA AI, originated from the Computer Innovation Institute of Zhejiang University, is committed to becoming a data navigator in the AI industry. ABAKA AI is also a member of China Artificial Intelligence Industry Development Alliance, ASAM Association, and Zhejiang Artificial Intelligence Industry Technology Alliance.\
ABAKA AI provides ABAVA Platform (AI-Based Augmented Virtual Associate) and ACE Service (Accurate & Efficient). It meets the data needs of dozens of application scenarios such as smart driving, AIGC, smart medical, smart security, smart city, etc. Currently, the company has cooperated with more than 200 top technology companies and scientific research institutions all over the world, has dozens of intellectual property rights, and participated in the writing of standards and white papers in the field of artificial intelligence several times. ABAKA AI has also been reported by many new media such as "CCTV Financial Channel", "Xinrui Hangshang", "Zhejiang Satellite TV", "Suzhou Satellite TV".

***

Finally, you are also welcome to join us if you are interested, feel free to leave your CV to`yjj@molardata.com`
