Apache Software Foundation introduced the distribution of the distributed olp -stromo storages apache Pinot 1.0 designed to fulfill analytical requests. Initially, the Pinot project was developed by LinkedIn and was transferred in 2015 for further joint development to the Apache Foundation. Storage oriented to work in the conditions of constant adding new data and is designed to ensure minimal and predictable delays that allow the use of a storage for processing requests in real time. The project code is written on Java and spreads under the license Apache.
Apache Pinot provides horizontal scalability and means of achieving failure tolerance and conservation of survivability in the event of software and hardware errors. The replication and backup processes are built directly into the processing cycle added to the data storage. On the one hand, this approach can significantly simplify architecture, but, on the other hand, leads to a delay between the addition of data and their availability for requests. Data in the storage can be loaded from different sources, starting Hadoop and ordinary files and ending with information from the Online source, such as kafka. To manage the Pinot Claster, apachelex is used.
The storage is being addressed through the usual SQL queries, which maintains typical sampling, aggregation, sorting, mergers (Join) and data grouping. There is support for window functions. The data is placed in the tables of the database focused on the column (column-oriented). Various compression schemes and the possibility of placing several values in one field are supported. Pinot provides a connected index system in which various indexation technologies can be used (a sorted index, a Bitmap index, an inverted index, Startreendex, Bloom Filter, Range Index, Lucence/FST), JSON-Index, George index).
Issue 1.0 summed up a large work on stabilizing the code base and taking into account the wishes of the community (more than 300 comments are taken into account). A new multi-stage request processing engine has been brought to a full state.