(0} Economic Daily - China economic network, August 3 - recently, China Unicom (zoqing 600050, diagnosis unit) officially launched its big data platform based on Huawei oceanstor storage Pacific series, marking a comprehensive breakthrough in the technological innovation of the big data computing and storage separation architecture between China Unicom and Huawei Technology Co., Ltd.

It is reported that China Unicom has built the global telecom industry technology and the surrounding double leading meta integration big data platform. Now, China Unicom is promoting the strategy of data center, realizing the global data sharing, cross source and cross domain analysis, and rapidly releasing the data value. And this also means that the construction of data medium platform has challenged the data infrastructure.

First of all, it is necessary to have flexible scheduling data infrastructure, separate pooling of computing and storage resources, the same storage base, and various computing engines, which can quickly schedule based on business demand and support rapid business innovation;

secondly, with lower storage costs, operators' data outlook will be increased by more than 5 times in 5g era, and they can now choose to reduce fees In the big environment, it is necessary to use the storage resources more efficiently and reduce the CAPEX and OPEX investment;

also shows that Da Sheng Fa chess and cards, data sharing, Da Sheng Fa chess and cards, data content is becoming more and more diversified. In order to avoid data island, it is in favor of bome four domain data integration analysis, and the same storage of unorganized and organized data has become a new standard of data infrastructure;

Finally, the data is solemn. Data analysis can support enterprises to upgrade their user experience and find new business opportunities. They are moving from edge business to central organization. Their infrastructure needs to be able to tolerate multiple nodes' failure at the same time, support sub-health detection, rapid fault repair, and effectively extend the maintenance cycle. Ensure the preparation of high availability of front pick down, reduce the maintenance complexity of massive data (travel 603138, diagnosis unit).

Five major innovation points of Huawei's big data storage and calculation separation scheme

in March 2019, the data center of the Information Department of China Unicom Group and Huawei Company established a saying consistent working group to jointly promote the intensive construction of big data compilation and the technological innovation of localization.

[0} operation approval platform group architecture

Huawei's big data storage and computing separation solution creatively realizes the original HDFS semantics in the storage layer, breaks the traditional placement architecture of computing and storage tight coupling of big data platform, not only realizes the significant reduction of big data platform construction cost, but also resolves the fully distributed storage layer architecture With the development of the ability to integrate with each other and upgrade the data analysis effect, China Unicom's vision and request for the big data platform have been fully met. According to reports, Huawei's big data storage and calculation separation scheme has five major innovations.

Firstly, computing and storage separation, expanding

on demand builds the native HDFS capability in the storage layer, splits the storage from the local disk of the server, and forms a completely separated scheme of computing and storage, which not only meets the demand of business elastic expansion, but also avoids resource spreading, laying a technical foundation for the same big data analysis platform.

Secondly, the large proportion of EC algorithm can increase the utilization rate of storage resources

. In the big data storage and calculation separation scheme, the mature EC technology of storage layer is used to replace the multiple copies of Hadoop, and the utilization rate of storage resources is increased to 91.6%. With the same amount of data, the number of cabinets is negative 45%, which greatly reduces the operation and maintenance costs.

Third, the fully distributed namenode, the same management of massive data

adopts the fully distributed namenode architecture selected by the storage layer. Each node is not only a data storage node, but also a metadata management node. A single namespace can support the storage of 10 billion files, providing a solid data storage base for the large data platform, which favors the data center of the group Strategy.

Fourth, native HDFS, the coexistence of new and old, cherish investment

built a metadata gateway of HDFS in big data storage layer, decided that HDFS had been installed in the existing network, and realized the same access to data in the exercise layer. In addition, it supported many kinds of writing strategies, such as priority writing to storage layer HDFS, writing to HDFS based on configuration, writing to HDFS based on load balancing, Real business has achieved a smooth evolution without perception.

Fifthly, enterprise level storage data treasure technology, upgrade solemnity

in the storage layer, using multiple fault domain technology, each fault domain approves 4 nodes to fail at the same time, and hundreds of storage node clusters can tolerate 12 nodes failure at the same time, which greatly lengthens the maintenance cycle. When the hard disk or node failure is shown, the data reconstruction speed of the storage layer is much higher than that of the traditional Hadoop, and the fault repair time is reduced from day level to juvenile level, which further strengthens the solemnity of the data layer.

As a result of this new technology, China Unicom said that Huawei has developed a special and comprehensive test plan. At the same time, it has built two sets of test environments: storage and computing integration and storage and computing separation. It uses the actual business data of the existing network desensitization, and decides the mirror image comparison, and demonstrates the feasibility and value of the scheme from the aspects of function, performance, solemnity and scalability. After several months of compliance testing, the test results were recognized by the data center of the press and Information Department of China Unicom Group, and finally led to China Unicom's surrounding procurement of Huawei oceanstor storage Pacific series.

now, the operation approval platform based on the separation of storage and calculation technology has been officially launched. It has carried 15pb data volume of Internet analysis, IOT, log, etc., and has realized the access analysis of 2 / 3 / 4 / 5gxdr, network signaling, Internet log, IOT and other data of Unicom provincial companies. The daily import data volume exceeds 70tb.

At the same time, China Unicom also has a subtle value from the scheme of separation of deposit and calculation. For example, it can significantly reduce the cost of procurement and operation and maintenance, and upgrade the operation effect and solemnity. It is reported that after the project is put into operation, the analysis effect will be increased by 20%, which will bring more than 10 million yuan of good harvest and growth every year. This table, the storage layer to determine the fault domain, rapid reconfiguration, sub-health detection and other technologies, solemnity will be increased more than 30 times.

In the future, Huawei will continue to inject power into the innovation of China Unicom's data infrastructure. By combining the capability of compatibility and interoperability of Huawei oceanstor storage Pacific series and the blooming data analysis ecology, Huawei will realize the data integration of BOM in many fields, break through the barriers between production preparation and analysis compilation, build the same data storage base, and reduce the data storage On the basis of stateless computing, we introduce more diverse computing engines, such as Bi, deep learning, graph computing, space-time analysis and other emerging engines, and quickly show their ingenuity and support the implementation of rapid innovation. It is determined to innovate constantly, jointly build a data infrastructure that can be stored, flowing and well managed, and release the value of massive data.


热门排行

最新文章

Powered by 快3正规平台 @2018 RSS地图 html地图