How to (mis)use Virtual Threads

Tags:

How to enable Virtual Threads?

During my talks about Java 19 (which have the mandatory part dedicated to Virtual Threads), I’ve been asked: how do I enable Virtual Threads in the JVM? quite a few times.

This question carries an assumption that there’s some kind of switch, which has to be activated or hit to “magically” turn good ol' Threads into Virtual Threads. AFAICT there is no such thing right now, and I rather doubt there will be one in the future. There is no system property, environment variable or java parameter I know of, that would make Virtual Threads be used instead of Platform Threads. (Sure, to be able to use VTs we need to java --enable-preview, but that applies to all preview features, it’s not VTs specific.)

The Virtual Threads can be activated only by an explicit change in the source code. Basically, the places where Threads are created have to be revisited and changed so that VTs are created instead.

There are at least a few ways to use Virtual Threads instead of Platform Threads.

If we create Threads “by hand” in our program, we have to change new Thread(...) into Thread.ofVirtual()..., pretty much as I described in the previous installment.

If we’re using an ExecutorService, it might be wise to create one using Executors.newVirtualThreadPerTaskExecutor()

Which in turn relies on Thread.ofVirtual().factory() that can be used if you need a ThreadFactory.

I guess the Structured Concurrency is worth taking a look too. (I’ll write about it some other time).

Okay, but

How to misuse Virtual Threads?

There are two simple scenarios related to how not use Virtual Threads, and they’re listed in JEP-425.

  1. You shall not cache/reuse Virtual Threads.

    Really, they are meant to be used only once, so you don’t want to cache them. They aren’t heavy. Or you shall not use them to throttle / handle backpressure. There are better ways to achieve that in the concurrency toolbox.

  2. You don’t want to call blocking operations or native code from a synchronized section or method, because such a call pins a VirtualThread making such a call to the Platform Thread carrying it.

    You shall not pin Virtual Threads!

    Virtual Threads aren’t magic. They’re implemented as continuations, and they use (i.e. are mounted to) Platform Threads when they’re active.

    Native calls are pretty easy to understand, I believe. We don’t know what the native code is doing, so the Virtual Thread making such a call can’t unmount the Platform Thread, and needs to be pinned.

    And currently the same applies to blocking I/O calls, although this might be lifted in the future.

    So basically, if you’re making blocking calls or native calls, you don’t want to wrap them in synchronized, at least not for long. If you really need to make sure there are no concurrent calls, use locks/semaphores/whatever to ensure that.

Okay, but how can I know

Well, we shall test our code, right? And do that before it gets into production ;-)

By default, Java 19 will emit a new JFR event, jdk.VirtualThreadPinned, every time a Virtual Thread gets pinned, with threshold of 20ms. So after we start creating and using VTs in our code, we might want to run the code and check if there are no such events.

Manually? Nah, we’re going to automate this, because a) automation is good, b) we can, thanks to JfrUnit! YAY!!! A big, big THANK YOU to Mr. Gunnar Morling and other maintainers.

Let’s say our code is making HTTP calls, and we’re afraid that the VTs might get pinned. We’re going to use JUnit 5 and emulate the service our client is calling. It looks like this:

38@BeforeEach
39void setUp() throws Exception {
40    server = HttpServer.create(new InetSocketAddress(8080), 1_000_000);
41    server.createContext("/", exchange -> {
42        int sum = random.ints(15_000_000).sum();
43        var responseBytes = (sum + "").getBytes(StandardCharsets.UTF_8);
44        exchange.getResponseHeaders().add("content-type", "text/plain");
45        exchange.sendResponseHeaders(200, responseBytes.length);
46        exchange.getResponseBody().write(responseBytes);
47        exchange.close();
48    });
49    server.setExecutor(Executors.newSingleThreadExecutor());
50    server.start();
51}
52
53@AfterEach
54void tearDown() {
55    server.stop(1);
56}

The server is extremely dumb: it’s single-threaded and summarizing 15 million ints (to keep it busy without Thread.sleep()), and then serving the sum to the client as plain text. (Depending on your hardware, you may need to tune this number.) What we want to achieve here is to make HTTP call to this server take some time, and thus block the client.

Then, the actual test case may look like this: 

58@Test
59@EnableEvent("jdk.VirtualThreadPinned")
60public void shouldNotPin() throws InterruptedException {
61
62    var client = HttpClient.newBuilder().connectTimeout(Duration.ofSeconds(5)).build();
63    var request = HttpRequest.newBuilder(URI.create("http://localhost:8080")).GET().build();
64
65    VirtThreads.getGreetings(client, request, 20);
66
67    jfrEvents.awaitEvents();
68    Assertions.assertTrue(jfrEvents.events().findAny().isEmpty(), "there should be no pinned events");
69}

This test is also quite simple, I hope. First, in line 59 we tell JfrUnit to capture jdk.VirtualThreadPinned JFR events. Then we run the System Under Test. Then in line 89 we stop capturing the events, to check in the very next line if there are no events recorded. Because if there were any VTs pinned, we shall see the events in jfrEvents.events().

Luckily, we don’t encounter any, if you simply checkout the demo project from GitHub. However, if you decide to play a bit and e.g. turn the getGreeting(...) method into a synchronized one, such a test should tell us that we’re misusing Virtual Threads by pinning them.

That’s all for now. And yes, I understand this approach has certain flaws, so please stay tuned for the next post.

​