Skip to content

Commit

Permalink
Merge branch 'master' into vertx5
Browse files Browse the repository at this point in the history
  • Loading branch information
michel-kraemer committed Jul 17, 2024
2 parents bbc90ed + 0cc6b20 commit 26d76b1
Show file tree
Hide file tree
Showing 66 changed files with 3,658 additions and 1,242 deletions.
8 changes: 6 additions & 2 deletions .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,10 +53,10 @@ jobs:
# run: |
# .github/workflows/free_disk_space.sh

- name: Setup Node.js 16.x
- name: Setup Node.js 20.x
uses: actions/setup-node@v1
with:
node-version: 16.x
node-version: 20.x

- uses: actions/cache@v2
with:
Expand All @@ -79,6 +79,10 @@ jobs:
- run: npm ci -f
working-directory: vertx5

# Necessary to generate OpenGraph images
- name: Install Chromium for Playwright
run: npx playwright install --with-deps chromium

# Download, extract, and compile docs
- run: npm run update-docs
- run: |
Expand Down
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
.cache/
.generated/
*.iml
.idea/
.next/
node_modules/
Expand All @@ -11,3 +13,6 @@ public/docs/
# Do not put anything in the `public/feed` directory! Contents will be added
# automatically by the '/pages/blog/[[...slug]].jsx' script
public/feed/

# OpenGraph images will be generated during build
public/images/previews/
2 changes: 1 addition & 1 deletion .tool-versions
Original file line number Diff line number Diff line change
@@ -1 +1 @@
nodejs 16.13.0
nodejs 20
7 changes: 5 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
FROM node:16-slim AS build
FROM node:20-slim AS build

RUN mkdir -p /vertx/docs
COPY package.json package-lock.json /vertx/
COPY docs/package.json docs/package-lock.json /vertx/docs

WORKDIR /vertx
RUN npm i
RUN npm ci

# Chromium is required to generate OpenGraph images
RUN npx playwright install --with-deps chromium

COPY docs /vertx/docs/
RUN npm run update-docs
Expand Down
12 changes: 9 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ This repository contains the source of the Vert.x website (https://vertx.io).
Initialize local copy:

npm i
npx playwright install --with-deps chromium

Download, extract and compile AsciiDoc source files of the Vert.x documentation:

Expand Down Expand Up @@ -34,13 +35,13 @@ If you don't want to install Node.js, use Docker instead:
```
# Initialize local copy
rm -rf node_modules docs/node_modules
docker run -it -v $(pwd):/vertx node:16-slim sh -c "cd /vertx && npm i"
docker run -it -v $(pwd):/vertx node:20-slim sh -c "cd /vertx && npm i"
# Download, extract, and compile docs
docker run -it -v $(pwd):/vertx node:16-slim sh -c "cd /vertx && npm run update-docs"
docker run -it -v $(pwd):/vertx node:20-slim sh -c "cd /vertx && npm run update-docs"
# Start website in development mode
docker run -it -v $(pwd):/vertx -p 3000:3000 node:16-slim \
docker run -it -v $(pwd):/vertx -p 3000:3000 node:20-slim \
sh -c "cd /vertx && npm run dev"
```

Expand All @@ -50,6 +51,11 @@ Alternatively, build a Docker image that runs the static website inside NGINX
docker build -t vertx-web-site .
docker run -it -p 80:80 vertx-web-site

Note: if you're on a Mac with Apple Silicon and configured Docker to use arm64,
you might have to add the `--platform` parameter during build:

docker build -t vertx-web-site . --platform=linux/amd64

## Automatic generation of the list of contributors

The community page contains a list of people who have contributed to any of the
Expand Down
10 changes: 10 additions & 0 deletions assets/logos/restate.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
325 changes: 325 additions & 0 deletions assets/ogimage-background.ai

Large diffs are not rendered by default.

Binary file added assets/ogimage-background.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion blog/2022-05-12-whats-new-in-vert-x-4-3.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,7 @@ GrpcServer grpcServer = GrpcServer.server(vertx);
GrpcServiceBridge serverStub = GrpcServiceBridge.bridge(service);
serverStub.bind(grpcServer);

router.consumes("application/grpc").handler(grpcServer);
router.consumes("application/grpc").handler(rc -> grpcServer.handle(rc.request()));
```

### Extra HTTP compression algorithms
Expand Down
1 change: 0 additions & 1 deletion blog/2023-03-02-whats-new-in-vert-x-4-4.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ authors:
github_id: vietj
summary: >-
See an overview of all new and exciting features in Vert.x 4.4, including a new OpenAPI implementation and io_uring support
pinned: true
---

Vert.x 4.4 comes with few new exciting features.
Expand Down
136 changes: 136 additions & 0 deletions blog/2023-08-25-soc-vertx-pinot-client.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
---
title: Eclipse Vert.x client for Apache Pinot
category: news
authors:
- name: Lucifer Morningstar
github_id: lucifer4j
summary: >-
Apache Pinot is a realtime distributed datastore for analytics workloads. To make
it easier for Eclipse Vert.x users to use Apache Pinot in their applications, we introduce
the Reactiverse Vert.x Client for Pinot.
---

<Alert warning>

The blog post is informational and a teaser of upcoming changes to `pinot-client` which is under active
development and thus, the finally released versions may differ. Currently, the client is not available
for download on Maven Central.
</Alert>

## Introduction

Eclipse Vert.x is a toolkit to build reactive applications on the Java virtual machine. It provides asynchronous and
non-blocking clients for different types of databases. Apache Pinot is a realtime distributed datastore for analytics
workloads. The [Apache Pinot Java Client](https://docs.pinot.apache.org/users/clients/java) uses
[AsyncHttpClient](https://github.com/AsyncHttpClient/async-http-client) as the default transport. The `Apache Pinot Java Client`
is unpractical to use in Eclipse Vert.x applications because every blocking call needs to be wrapped with `executeBlocking`.

The [`pinot-client`](https://github.com/reactiverse/pinot-client) Reactiverse project is a lightweight wrapper around the existing Pinot Java client. It also provides an alternative transport using `vertx-web-client`.
This will make the Apache Pinot more accessible to Vert.x users.

## Advantages

1. `vertx-pinot-client` offers an async and non-blocking API, that is line with the general Vert.x ecosystem's philosophy.
2. For little effort we get `RxJava` and `Mutiny` bindings for the client as well, thanks to Vert.x codegen.
3. The `vertx-pinot-client` should offer better performance as it uses the Vert.x Web Client for the underlying transport.
This allows the application to stay on the same event loop when interacting with Pinot.

## Usage

Add the `pinot-client` dependency to your project.

```xml
<dependency>
<groupId>io.reactiverse</groupId>
<artifactId>pinot-client</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
```

You can create an instance of the client to interact with the Pinot cluster as follows.

```java
String brokerUrl = "localhost:8000";
VertxPinotClientTransport transport = new VertxPinotClientTransport(vertx);
VertxConnection connection = VertxConnectionFactory.fromHostList(vertx, List.of(brokerUrl), transport);
```

Now we can use the `Connection` to query the cluster. Here is an example to query the top 10 home run scorers from
the sample quickstart dataset.

```java
String query = "select playerName, sum(homeRuns) AS totalHomeRuns from baseballStats where homeRuns > 0 group by playerID, playerName ORDER BY totalHomeRuns DESC limit 10";
connection
.execute(query)
.onSuccess(resultSetGroup -> {
ResultSet results = resultSetGroup.getResultSet(0);
System.out.println("Player Name\tTotal Home Runs");
for (int i = 0; i < results.getRowCount(); i++) {
System.out.println(results.getString(i, 0) + "\t" + results.getString(i, 1));
}
})
.onFailure(Throwable::printStackTrace);
```

## Demo

A more real-life demo is available at [`pizza-shop-demo`](https://github.com/lucifer4j/pizza-shop-demo). It includes a
pizza delivery service specializing in pizzas with Indian toppings. They have a dashboard built using Apache Pinot
to monitors their orders in realtime. The dashboard deals with three types of data: products, users, and orders. The
dashboard uses the Vert.x Pinot Client to [`query`](https://github.com/lucifer4j/pizza-shop-demo/blob/main/dashboard/backend/src/main/java/pizzashop/DashboardVerticle.java)
the Pinot cluster.

![Order Dashboard Statistics](/images/blog/soc-vertx-pinot-client/demo-dashboard-1.png)
![Order Dashboard Latest and Popular Items](/images/blog/soc-vertx-pinot-client/demo-dashboard-2.png)

## Challenges

I worked on this project as a part of the [Google Summer of Code](summerofcode.withgoogle.com/) - 2023 program and
during the implementation, I encountered several challenges.

#### Package-private `org.apache.pinot.client.BrokerResponse` class

The [`org.apache.pinot.client.BrokerResponse`](https://github.com/apache/pinot/blob/6e235a4ec2a16006337da04e118a435b5bb8f6d8/pinot-clients/pinot-java-client/src/main/java/org/apache/pinot/client/BrokerResponse.java#L27)
class is package-private. However several [`executeQuery/executeQueryAsync`](https://github.com/apache/pinot/blob/6e235a4ec2a16006337da04e118a435b5bb8f6d8/pinot-clients/pinot-java-client/src/main/java/org/apache/pinot/client/PinotClientTransport.java#L29)
methods in the `PinotClientTransport` return this type. This makes it difficult to implement the interface in an external package.

To work around this issue, we created a `org.apache.pinot.client` package inside our wrapper project. The [`VertxPinotClientTransport`](https://github.com/lucifer4j/pinot-client/blob/70dc4152cc77fd6ae521f94fb8c59bf4c7f9d12a/pinot-client/src/main/java/org/apache/pinot/client/VertxPinotClientTransport.java#L37)
implementation currently lives inside this package for the same reason. This workaround only works for projects running in `classpath`
mode but not for projects using `modulepath` because Java Modules disallow package splitting. To fix the issue as source,
I opened a [`PR`](https://github.com/apache/pinot/pull/10932) in `apache/pinot` repository. The PR is now merged. Once a new
release of Pinot is available, I will be updating the client wrapper as well.

#### `java.concurrent.util.Future` return type

Some methods of the `PinotClientTransport` [interface](https://github.com/apache/pinot/blob/6e235a4ec2a16006337da04e118a435b5bb8f6d8/pinot-clients/pinot-java-client/src/main/java/org/apache/pinot/client/PinotClientTransport.java#L32)
return a `java.concurrent.util.Future` future. However, `Future.get` is a blocking call which is undesirable. To work around the
issue, we wrap every request and query call in an [`executeBlocking` block](https://github.com/reactiverse/pinot-client/blob/main/pinot-client/src/main/java/io/reactiverse/pinot/client/impl/Utils.java#L24-L34).
To avoid blocking calls altogether, we discussed replacing `Future` with `CompletabeFutre`. The same change has already been [merged](https://github.com/apache/pinot/pull/10326)
now and will be available in the next release. Once that is available, we can update the wrapper client to adapt `CompletableFuture`s
to `Vert.x Futures` without much hassle and without having to offload some of the tasks to the Vert.x worker pool.

#### Missing `ConnectionFactory` method

The [`ConnectionFactory`](https://github.com/apache/pinot/blob/6e235a4ec2a16006337da04e118a435b5bb8f6d8/pinot-clients/pinot-java-client/src/main/java/org/apache/pinot/client/ConnectionFactory.java#L30)
class is used to create `Connection` instances to interact with the Pinot Cluster. There are multiple ways to interact with
the cluster, using zookeeper, controller or a hard-coded broker list. Most of the methods feature an overload to accept
a custom `PinotClientTransport` implementation but the same was notable missing for the controller variant. Further, the
constructors of the `Connection` class are package-private preventing us from directly adding the missing method in our
wrapper.

Again to fix this issue, I created a [`bridge class`](https://github.com/lucifer4j/pinot-client/blob/70dc4152cc77fd6ae521f94fb8c59bf4c7f9d12a/pinot-client/src/main/java/org/apache/pinot/client/ConnectionFactoryBridge.java#L28)
in the `org.apache.pinot.client` package. Simultaneously, I opened a [`PR`](https://github.com/apache/pinot/pull/11013) in
`apache/pinot` repository. The PR is also now merged and the Vert.x client will be updated once a new release is available.

## Bonus Fixes

#### Missing double-checked locking in `ConnectionFactory`

While working on the tasks outlined earlier, I (more accurately my IDE's static checking) discovered the double-checked
locking mechanism in `ConnectionFactory` class in `pinot-client` was broken. I opened a [`PR`](https://github.com/apache/pinot/pull/11014) to fix it.

#### Broken Docker health checks in `pizza-shop-demo`

The Docker Health Checks in the original `pizza-shop-demo` were broken. The health checks had been written using `nc` utility
but the same is not included in the `apache/pinot` docker image. It took me a lot of head scratching to discover that but
the same is now fixed. The PR for fixing it upstream is available [`here`](https://github.com/startreedata/pizza-shop-demo/pull/9).
32 changes: 32 additions & 0 deletions blog/2023-08-30-eclipse-vert-x-4-4-5.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
---
title: Eclipse Vert.x 4.4.5 released!
category: releases
authors:
- name: Julien Viet
github_id: vietj
summary: >-
Eclipse Vert.x version 4.4.5 has just been released. It fixes a few bugs that have been reported by the community
---

We are extremely pleased to announce that Eclipse Vert.x version 4.4.5 has been released.

Since the release of Vert.x 4.4.4, quite a few bugs have been reported. We would like to thank you all for reporting these issues.

The [4.4.5 release notes](https://github.com/vert-x3/wiki/wiki/4.4.5-Release-Notes) as well as
[deprecations and breaking changes](https://github.com/vert-x3/wiki/wiki/4.4.5-Deprecations-and-breaking-changes)
can be found on the wiki.

You can bootstrap a Vert.x 4 project using [start.vertx.io](https://start.vertx.io).

The release artifacts have been deployed to [Maven Central](https://search.maven.org/search?q=g:io.vertx%20AND%20v:4.4.5) and
you can get the distribution on [Maven Central](https://repo1.maven.org/maven2/io/vertx/vertx-stack-manager/4.4.5/).

The [Vert.x 4 eventbus JavaScript client library](https://www.npmjs.com/package/@vertx/eventbus-bridge-client.js) is now available
in a single location and it now usable standalone or it can easily be integrated with any frontend build tool.

The Vert.x distribution is available from [SDKMan](https://sdkman.io) and our
[HomeBrew TAP](https://github.com/vertx-distrib/homebrew-tap).

Docker images are available on [Docker Hub](https://hub.docker.com/u/vertx/).

That's it! Happy coding and see you soon on our user or dev [channels](https://vertx.io/channels).
37 changes: 37 additions & 0 deletions blog/2023-10-11-eclipse-vert-x-4-4-6.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
---
title: Eclipse Vert.x 4.4.6 released!
category: releases
authors:
- name: Julien Viet
github_id: vietj
summary: >-
Eclipse Vert.x version 4.4.6 has just been released. It fixes a few bugs that have been reported by the community as well
as provides protection for CVE-2023-44487
---

We are extremely pleased to announce that Eclipse Vert.x version 4.4.6 has been released.

Since the release of Vert.x 4.4.5, quite a few bugs have been reported. We would like to thank you all for reporting these issues.

In addition, the [CVE-2023-44487](https://www.cve.org/CVERecord?id=CVE-2023-44487) has been disclosed, it affects
HTTP/2 servers. Vert.x upgrades to [Netty 4.1.100.Final](https://netty.io/news/2023/10/10/4-1-100-Final.html)
which provides protection for this flood DDOS.

The [4.4.6 release notes](https://github.com/vert-x3/wiki/wiki/4.4.6-Release-Notes) as well as
[deprecations and breaking changes](https://github.com/vert-x3/wiki/wiki/4.4.6-Deprecations-and-breaking-changes)
can be found on the wiki.

You can bootstrap a Vert.x 4 project using [start.vertx.io](https://start.vertx.io).

The release artifacts have been deployed to [Maven Central](https://search.maven.org/search?q=g:io.vertx%20AND%20v:4.4.6) and
you can get the distribution on [Maven Central](https://repo1.maven.org/maven2/io/vertx/vertx-stack-manager/4.4.6/).

The [Vert.x 4 eventbus JavaScript client library](https://www.npmjs.com/package/@vertx/eventbus-bridge-client.js) is now available
in a single location and it now usable standalone or it can easily be integrated with any frontend build tool.

The Vert.x distribution is available from [SDKMan](https://sdkman.io) and our
[HomeBrew TAP](https://github.com/vertx-distrib/homebrew-tap).

Docker images are available on [Docker Hub](https://hub.docker.com/u/vertx/).

That's it! Happy coding and see you soon on our user or dev [channels](https://vertx.io/channels).
33 changes: 33 additions & 0 deletions blog/2023-11-16-eclipse-vert-x-4-5-0.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
---
title: Eclipse Vert.x 4.5.0 released!
category: releases
authors:
- name: Julien Viet
github_id: vietj
summary: >-
Eclipse Vert.x version 4.5.0 has just been released. It comes with a set of new exciting features, including virtual threads support!
---

We are extremely pleased to announce that Eclipse Vert.x version 4.5.0 has been released.

Vert.x 4.5 comes with a set of new [exciting features](https://vertx.io/blog/whats-new-in-vert-x-4-5/), including
virtual threads support.

The [4.5.0 release notes](https://github.com/vert-x3/wiki/wiki/4.5.0-Release-Notes) as well as
[deprecations and breaking changes](https://github.com/vert-x3/wiki/wiki/4.5.0-Deprecations-and-breaking-changes)
can be found on the wiki.

You can bootstrap a Vert.x 4 project using [start.vertx.io](https://start.vertx.io).

The release artifacts have been deployed to [Maven Central](https://search.maven.org/search?q=g:io.vertx%20AND%20v:4.5.0) and
you can get the distribution on [Maven Central](https://repo1.maven.org/maven2/io/vertx/vertx-stack-manager/4.5.0/).

The [Vert.x 4 eventbus JavaScript client library](https://www.npmjs.com/package/@vertx/eventbus-bridge-client.js) is now available
in a single location and it now usable standalone or it can easily be integrated with any frontend build tool.

The Vert.x distribution is available from [SDKMan](https://sdkman.io) and our
[HomeBrew TAP](https://github.com/vertx-distrib/homebrew-tap).

Docker images are available on [Docker Hub](https://hub.docker.com/u/vertx/).

That's it! Happy coding and see you soon on our user or dev [channels](https://vertx.io/channels).
Loading

0 comments on commit 26d76b1

Please sign in to comment.