What Is JingQi Hadoop?

The JingQi Hadoop® platform was developed based on open-source Apache Hadoop and two softwares were put forward for reliable, scalable, distributed computing.

The JingQi Hadoop software library open two products: (1) Data Mining for Petroleum Exploration and Development; (2) Big Data Analysis for Co-evolving Data Streams. Both software allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single server to thousands of machines, each offering local computation and storage. Rather than rely on hardware to deliver high-availability, the two products itself is designed to detect and handle failures at the application layer, so delivering a highly-available service on top of a cluster of computers, each of which may be prone to failures.
The two productions support the following modules:
Hadoop Common: The common utilities that support the other Hadoop modules.
Hadoop Distributed File System (HDFS™): A distributed file system that provides high-throughput access to application data.
Hadoop YARN: A framework for job scheduling and cluster resource management.
Hadoop MapReduce: A YARN-based system for parallel processing of large data sets.

News
18 November, 2015: release 5.0 JingQi Hadoop DM available
JingQi Hadoop for data mining project with petroleum exploration and development contains a number of significant enhancements such as:

        · Application APIs for heterogeneous storage

        · SSD storage tier

        · Memory as a storage tier (beta)

o Support for Archival Storage

o Transparent data at rest encryption (beta)

o Operating secure DataNode without requiring root access

o Hot swap drive: support add/remove data node volumes without restarting data node (beta)

o AES support for faster wire encryption

·  Hadoop YARN

o Support for long running services in YARN

       · Service Registry for applications

o Support for rolling upgrades

      · Work-preserving restarts of ResourceManager

      · Container-preserving restart of NodeManager

o Support node labels during scheduling

o Support for time-based resource reservations in Capacity Scheduler(beta)

o Global, shared cache for application artifacts (beta)

o Support running of applications natively in Docker containers(alpha)

May 2014 - Avro and HBase were integrated into Big Data Analysis project.

Big Data Analysis project for abnormality detection with co-evolving data streams was developed based on Hadoop's Avro and HBase. It has graduated to become top-level JingQi Hadoop projects.

The video can now be found at https://youtu.be/nQ-eIFKOXGA

August 2013 - New JingQi Hadoop Subproject –Abnormality Detection for Co-evolving data streams

JingQi Hadoop is getting bigger!

·  It was develop based on Hadoop Common.

·  MapReduce and the Hadoop Distributed File System (HDFS) are now integerated into the projects.

·  Avro and Chukwa are integrated.

·  The project was implemented in Royal Brisbane and Women’s Hospital, Australia.

July 2008 – JingQi’s petroleum DM Hadoop WinsTerabyte Sort Benchmark

- JingQi Hadoop Wins Terabytes Award: One of the JingQi’s Petroleum DM Hadoop sorted 1 terabyte of seismic data in 300 seconds.

- The project was implemented by BHP Billiton, Australia.

上一篇：晶奇Hadoop®大数据平台

下一篇：晶奇 Hadoop 软件包产品 (version 2008116)说明