Skip to content

v0.2: cascades optimizer, SIMD, and better storage

Latest
Latest
Compare
Choose a tag to compare
@skyzh skyzh released this 20 Dec 19:48
· 73 commits to main since this release
v0.2.0
0265c50
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

After 9 months' development, we are happy to announce a new version of RisingLight. Major changes in this version:

  • New cascades-style binder / planner / optimizer framework powered by egg.
  • New SIMD expression evaluation design with better performance.
  • More efficient storage with better encoding and predicate push-downs.
  • Better docs on the storage layer.

Thanks for your support!

What's Changed

  • docs: migrate to new Slack community by @skyzh in #606
  • docs: add architecture overview by @skyzh in #609
  • feat(executor): introduce DataChunkBuilder and split chunks in TopN by @wangqiim in #610
  • deps: upgrade prost 0.10 by @alissa-tung in #613
  • feat(storage): record first_key in block_index_builder by @adlternative in #598
  • feat(executor): introduce DataChunkBuilder and split chunks in Order … by @wangqiim in #612
  • chore(ci): install protoc by direct download by @skyzh in #614
  • feat(executor): implement SortMergeJoin executor by @D2Lark in #600
  • chore(test): add more test case by @D2Lark in #616
  • docs: add instructions to install protobuf by @skyzh in #617
  • docs: add badges of dbdb.io, crates.io and docs.rs by @skyzh in #621
  • feat(storage): add support composite sort key in MergeIterator by @BaymaxHWY in #622
  • executor: use DataChunkbuilder to refactor ValuesExecutor by @JayiceZ in #623
  • build: bump toolchain and deps, remove unused dep by @TennyZhuang in #629
  • fix: make header correct when using group by by @shmiwy in #624
  • refactor(planner): use AggInputRefResolver to handle group by expr by @Fedomn in #630
  • feat(binder): support “primary key (a, b)” syntax by @shmiwy in #631
  • feat: support multi ordered primary key by @shmiwy in #633
  • community: add xxchan as maintainer by @skyzh in #634
  • docs: add overview of storage by @skyzh in #635
  • refactor(executor): refactor CopyFromFile using DataChunkBuilder by @noneback in #637
  • fix(planner): enhance group by illegal column validation by @Fedomn in #642
  • feat: use ref-cast for BlobRef cast by @Kikkon in #643
  • feat(storage): support range-filter scan by sort key by @shmiwy in #644
  • feat(optimizer): push the filter down below the agg by @lokax in #648
  • binder: primary key should by-default be non-null by @Ted-Jiang in #649
  • chore: remove Mutex in database/schema/table catalog by @chowc in #650
  • feat: support sort agg by @Kikkon in #646
  • fix: typo and clippy warnings by @yuzi-neko in #652
  • feat(binder): support Having clause and more flexible OrderBy by @lokax in #651
  • chore(sqllogictest): use libtest-mimic as test harness by @skyzh in #656
  • chore(test): ensure single thread runtime for sqllogictest by @skyzh in #657
  • feat(test): add sqlplannertest by @skyzh in #661
  • feat(cli): support multi-line interactive mode by @Gun9niR in #662
  • chore: upgrade dependencies by @wangrunji0408 in #663
  • fix: 'unsupported command' everywhere by @cadl in #664
  • fix: typo by @gogim1 in #665
  • feat(storage): use variable size encoding to compress rle count by @xinchengxx in #666
  • fix: create table with invalid primary keys constraints by @gogim1 in #667
  • chore: bump dependencies by @skyzh in #672
  • docs: add logo for RisingLight by @skyzh in #671
  • fix: append NULLs when inserting value with incomplete columns by @gogim1 in #673
  • feat(storage): support dict encoding by @Kikkon in #668
  • feat(optimizer): support column pruning for some operators by @lokax in #653
  • fix(storage): reserve correct size by @lokax in #678
  • chore: bump dependencies, toolchain, and adapt to new sqlparser by @skyzh in #681
  • chore: add github codespace devcontainer configuration by @yeya24 in #680
  • refactor: refactor risinglight-sqllogictest by @xxchan in #682
  • refactor: remove unnecessary current dir by @xxchan in #683
  • storage: use Arc during pin and unpin by @chaixuqing in #685
  • chore(storage): select compaction strategy using statistics by @chaixuqing in #689
  • feat(planner): plan select distinct by @lokax in #686
  • feat(executor): support perfect hash aggregate by @lokax in #690
  • fix: add row_handler to schema by @yinfredyue in #692
  • chore: bump toolchain and update dependencies by @wangrunji0408 in #695
  • feat: introduce ordered float by @wangrunji0408 in #699
  • feat(storage): support dict encoding in primitive column builder by @Kikkon in #701
  • feat: make binary operations auto-vectorized by @lokax in #700
  • feat: introduce next-gen planner & optimizer by @wangrunji0408 in #702
  • feat(binder): introduce egg binder for queries by @wangrunji0408 in #705
  • fix(storage): checksum BlockType in block header by @unconsolable in #704
  • feat(planner): support distinct on by @wangrunji0408 in #706
  • refactor(type): unify DataTypeKind by @wangrunji0408 in #708
  • feat(storage): generic nullable block encoding by @unconsolable in #709
  • feat(binder): egg binder for drop statement by @eliasyaoyc in #711
  • feat(planner_v2): add type checker by @wangrunji0408 in #717
  • refactor(storage): replace primitive nullable encoding with generic way by @unconsolable in #713
  • feat(binder): new binder for copy, delete and create_table by @eliasyaoyc in #719
  • fix(type): add NULL type and fix type analysis by @wangrunji0408 in #721
  • feat(planner_v2): support explain by @wangrunji0408 in #722
  • refactor(executor): simply drop the futures to cancel a query by @wangrunji0408 in #720
  • feat(planner_v2): cost function and multi-stage optimization by @wangrunji0408 in #723
  • feat: introduce executor v2 for the new planner by @wangrunji0408 in #725
  • feat(planner_v2): support standalone VALUES clause by @wangrunji0408 in #726
  • feat(storage): support NULL on blob and char columns by @unconsolable in #724
  • refactor(fetch size): add fast return and compare ROWSET MAX OUTPUT by @eliasyaoyc in #716
  • feat(executor_v2): migrate executors to run TPCH by @wangrunji0408 in #727
  • feat(planner_v2): pass all sqllogictests by @wangrunji0408 in #728
  • chore: enable query engine v2 by default by @wangrunji0408 in #729
  • feat(rt): support tokio-console by @wangrunji0408 in #730
  • fix(perf): fix performance regression in HashJoin & HashAgg by @wangrunji0408 in #731
  • chore: bump toolchain and apply async-fn-in-trait by @wangrunji0408 in #732
  • chore(ci): remove bench workflow by @skyzh in #734
  • build: bump deps by @TennyZhuang in #735
  • doc: Storage basics documentation by @yinfredyue in #738
  • fix(storage): compaction type error by @yinfredyue in #737
  • chore: bump sqllogictest to 0.9.0 by @xxchan in #736
  • feat(storage): Dict encoding for compaction by @yinfredyue in #740
  • perf(expr): apply auto-vectorization and remove explicit SIMDs by @wangrunji0408 in #741
  • release: v0.2 by @skyzh in #742
  • perf(expr): further optimize performance by @wangrunji0408 in #744

New Contributors

Full Changelog: v0.1.3...v0.2.0

Contributors

cadl, skyzh, and 25 other contributors
Assets 2
Loading