An open supply unified execution engine

• Meta is introducing Velox, an open supply unified execution engine aimed toward accelerating information administration programs and streamlining their growth.
• Velox is beneath energetic growth. Experimental outcomes from our paper printed on the Worldwide Convention on Very Massive Information Bases (VLDB) 2022 present how Velox improves effectivity and consistency in information administration programs.
• Velox helps consolidate and unify information administration programs in a way we consider will likely be of profit to the business. We’re hoping the bigger open supply group will be a part of us in contributing to the venture.

Meta’s infrastructure performs an necessary function in supporting our services and products. Our information infrastructure ecosystem consists of dozens of specialised information computation engines, all centered on completely different workloads for a wide range of use instances starting from SQL analytics (batch and interactive) to transactional workloads, stream processing, information ingestion, and extra. Lately, the fast development of synthetic intelligence (AI) and machine studying (ML) use instances inside Meta’s infrastructure has led to further engines and libraries focused at characteristic engineering, information preprocessing, and different workloads for ML coaching and serving pipelines. 

Nonetheless, regardless of the similarities, these engines have largely advanced independently. This fragmentation has made sustaining and enhancing them tough, particularly contemplating that as workloads evolve, the {hardware} that executes these workloads additionally modifications. In the end, this fragmentation leads to programs with completely different characteristic units and inconsistent semantics — decreasing the productiveness of knowledge customers that must work together with a number of engines to complete duties.

As a way to deal with these challenges and to create a stronger, extra environment friendly information infrastructure for our personal merchandise and the world, Meta has created and open sourced Velox. It’s a novel, state-of-the-art unified execution engine that goals to hurry up information administration programs in addition to streamline their growth. Velox unifies the widespread data-intensive elements of knowledge computation engines whereas nonetheless being extensible and adaptable to completely different computation engines. It democratizes optimizations that had been beforehand applied solely in particular person engines, offering a framework through which constant semantics may be applied. This reduces work duplication, promotes reusability, and improves total effectivity and consistency.  

Velox is beneath energetic growth, however it’s already in numerous phases of integration with greater than a dozen information programs at Meta, together with Presto, Spark, and PyTorch (the latter by a knowledge preprocessing library referred to as TorchArrow), in addition to different inner stream processing platforms, transactional engines, information ingestion programs and infrastructure, ML programs for characteristic engineering, and others. 

Because it was first uploaded to GitHub, the Velox open supply venture has attracted greater than 150 code contributors, together with key collaborators similar to Ahana, Intel, and Voltron Information, in addition to numerous educational establishments. By open-sourcing and fostering a group for Velox, we consider we are able to speed up the tempo of innovation within the information administration system’s growth business. We hope extra people and corporations will be a part of us on this effort. 

An outline of Velox

Whereas information computation engines could appear distinct at first, they’re all composed of an analogous set of logical elements: a language entrance finish, an intermediate illustration (IR), an optimizer, an execution runtime, and an execution engine. Velox gives the constructing blocks required to implement execution engines, consisting of all data-intensive operations executed inside a single host, similar to expression analysis, aggregation, sorting, becoming a member of, and extra — additionally generally known as the info aircraft. Subsequently, Velox expects an optimized plan as enter and effectively executes it utilizing the assets out there within the native host.

Velox leverages quite a few runtime optimizations, similar to filter and conjunct reordering, key normalization for array and hash-based aggregations and joins, dynamic filter pushdown, and adaptive column prefetching. These optimizations present optimum native effectivity given the out there data and statistics extracted from incoming batches of knowledge. Velox can be designed from the bottom as much as effectively help complicated information varieties resulting from their ubiquity in trendy workloads, and therefore extensively depends on dictionary encoding for cardinality-increasing and cardinality-reducing operations similar to joins and filtering, whereas nonetheless offering quick paths for primitive information varieties.

The principle elements supplied by Velox are:

• Kind: a generic kind system that permits builders to characterize scalar, complicated, and nested information varieties, together with structs, maps, arrays, capabilities (lambdas), decimals, tensors, and extra.
• Vector: an Apache Arrow–appropriate columnar reminiscence format module supporting a number of encodings, similar to flat, dictionary, fixed, sequence/RLE, and body of reference, along with a lazy materialization sample and help for out-of-order end result buffer inhabitants.
• Expression Eval: a state-of-the-art vectorized expression analysis engine constructed primarily based on vector-encoded information, leveraging strategies similar to widespread subexpression elimination, fixed folding, environment friendly null propagation, encoding-aware analysis, dictionary peeling, and memoization.
• Capabilities: APIs that can be utilized by builders to construct customized capabilities, offering a easy (row by row) and vectorized (batch by batch) interface for scalar capabilities and an API for mixture capabilities. 
  • A perform bundle appropriate with the favored PrestoSQL dialect can be supplied as a part of the library.
• Operators: implementation of widespread SQL operators similar to TableScan, Mission, Filter, Aggregation, Change/Merge, OrderBy, TopN, HashJoin, MergeJoin, Unnest, and extra.
• I/O: a set of APIs that permits Velox to be built-in within the context of different engines and runtimes, similar to:
  • Connectors: permits builders to specialize information sources and sinks for TableScan and TableWrite operators.
  • DWIO: an extensible interface offering help for encoding/decoding in style file codecs similar to Parquet, ORC, and DWRF.
  • Storage adapters: a byte-based extensible interface that permits Velox to connect with storage programs similar to Tectonic, S3, HDFS, and extra. 
  • Serializers: a serialization interface focusing on community communication the place completely different wire protocols may be applied, supporting PrestoPage and Spark’s UnsafeRow codecs.
• Useful resource administration: a set of primitives for dealing with computational assets, similar to CPU and reminiscence administration, spilling, and reminiscence and SSD caching.

Velox’s foremost integrations and experimental outcomes

Past effectivity positive factors, Velox gives worth by unifying the execution engines throughout completely different information computation engines. The three hottest integrations are Presto, Spark, and TorchArrow/PyTorch.

Presto — Prestissimo 

Velox is being built-in into Presto as a part of the Prestissimo venture, the place Presto Java staff are changed by a C++ course of primarily based on Velox. The venture was initially created by Meta in 2020 and is beneath continued growth in collaboration with Ahana, together with different open supply contributors.

Prestissimo gives a C++ implementation of Presto’s HTTP REST interface, together with worker-to-worker alternate serialization protocol, coordinator-to-worker orchestration, and standing reporting endpoints, thereby offering a drop-in C++ alternative for Presto staff. The principle question workflow consists of receiving a Presto plan fragment from a Java coordinator, translating it right into a Velox question plan, and handing it off to Velox for execution.

We carried out two completely different experiments to discover the speedup supplied by Velox in Presto. Our first experiment used the TPC-H benchmark and measured near an order of magnitude speedup in some CPU-bound queries. We noticed a extra modest speedup (averaging 3-6x) for shuffle-bound queries.

Though the TPC-H dataset is a regular benchmark, it’s not consultant of actual workloads. To discover how Velox may carry out in these situations, we created an experiment the place we executed manufacturing site visitors generated by a wide range of interactive analytical instruments discovered at Meta. On this experiment, we noticed a mean of 6-7x speedups in information querying, with some outcomes growing speedups by over an order of magnitude. You may be taught extra concerning the particulars of the experiments and their leads to our analysis paper.

Prestissimo’s codebase is accessible on GitHub.  

Spark — Gluten

Velox can be being built-in into Spark as a part of the Gluten venture created by Intel. Gluten permits C++ execution engines (similar to Velox) for use inside the Spark atmosphere whereas executing Spark SQL queries. Gluten decouples the Spark JVM and execution engine by making a JNI API primarily based on the Apache Arrow information format and Substrait question plans, thus permitting Velox for use inside Spark by merely integrating with Gluten’s JNI API.

Gluten’s codebase is accessible on GitHub.  

TorchArrow

TorchArrow is a dataframe Python library for information preprocessing in deep studying, and a part of the PyTorch venture. TorchArrow internally interprets the dataframe illustration right into a Velox plan and delegates it to Velox for execution. Along with converging the in any other case fragmented house of ML information preprocessing libraries, this integration permits Meta to consolidate execution-engine code between analytic engines and ML infrastructure. It gives a extra constant expertise for ML finish customers, who’re generally required to work together with completely different computation engines to finish a specific process, by exposing the identical set of capabilities/UDFs and making certain constant habits throughout engines.

TorchArrow was lately launched in beta mode on GitHub.

The way forward for database system growth

Velox demonstrates that it’s doable to make information computation programs extra adaptable by consolidating their execution engines right into a single unified library. As we proceed to combine Velox into our personal programs, we’re dedicated to constructing a sustainable open supply group to help the venture in addition to to hurry up library growth and business adoption. We’re additionally occupied with persevering with to blur the boundaries between ML infrastructure and conventional information administration programs by unifying perform packages and semantics between these silos.

Wanting on the future, we consider Velox’s unified and modular nature has the potential to be useful to industries that make the most of, and particularly people who develop, information administration programs. It’s going to permit us to companion with {hardware} distributors and proactively adapt our unified software program stack as {hardware} advances. Reusing unified and extremely environment friendly elements may also permit us to innovate sooner as information workloads evolve. We consider that modularity and reusability are the way forward for database system growth, and we hope that information firms, academia, and particular person database practitioners alike will be a part of us on this effort. 

In-depth documentation about Velox and these elements may be discovered on our web site and in our analysis paper “Velox: Meta’s unified execution engine.”

Acknowledgements

We want to thank all contributors to the Velox venture. A particular thank-you to Sridhar Anumandla, Philip Bell, Biswapesh Chattopadhyay, Naveen Cherukuri, Wei He, Jiju John, Jimmy Lu, Xiaoxuang Meng, Krishna Pai, Laith Sakka, Bikramjeet Vigand, Kevin Wilfong from the Meta crew, and to numerous group contributors, together with Frank Hu, Deepak Majeti, Aditi Pandit, and Ying Su.