zoomcraft

Realistic audio presence, in Minecraft.

Are you working from home? Spending all day on Zoom calls?

Missing that presence that you feel when everybody’s working in the same
office?

Well, why not recapture that feeling by bringing everybody together—in
Minecraft!

zoomcraft is a single-container service that augments your Minecraft server
with real-time 3D audio presence over WebRTC. It works
with any Minecraft server with RCON capabilities (vanilla servers since 1.0.0).

It uses web audio spatialization APIs
to map the audio from other players to their in-game position, in order to
create a realistic virtual presence.

Check out a screencast of zoomcraft‘s 3D audio
capabilties. Use headphones! (or make sure your
speakers support stereo audio).

Usage

Run the Docker image on a server with access to the Minecraft game
server (vanilla / any variant with RCON access).

docker run \
  -p 8080:8080 \
  -e RCON_ADDRESS=http://localhost:25575 \
  -e RCON_PASSWORD=minecraft \
  stevenxie/zoomcraft

Join the Minecraft server.
Visit http://localhost:8080 and enter your player username to begin
conferencing.
Expose port 8080 on a public address to allow other players to
connect to zoomcraft. Have fun!

If you experience any connection issues or other bugs, please
submit an issue so
that I can look into it!

Caveats

Not all browser fully support the web audio spec. This service was developed
and tested on Chrome, so that is the recommended browser to use with
zoomcraft.
Some bluetooth headphones have limited audio channels, so when the microphone
is active, stereo audio output is disabled (meaning that 3D audio effects will
not work). If this happens, change your computer’s audio input source to be
something other than the bluetooth headphones (e.g. your computer’s internal
speakers).
Some Chrome browser extensions may interfere with the WebRTC connection
negotiation process. Run zoomcraft in an incognito window if you
encounter connection issues.
The first connection attempt sometimes takes much longer than future
attempts. Be prepared for other players to be connecting for up to 30
seconds. This should be a lot better starting from v1.1.1 due to the
implementation of WebRTC negotiation timeouts. Still happening?
File an issue!

Architecture

zoomcraft consists of three components: backend, gateway, and client:

backend is responsible for querying the Minecraft server for
world and player data using RCON. It exposes this
information over a GraphQL API.

Interested in forking zoomcraft to support another game? This is the
code that you should probably change!

In particular, check out
backend/minecraft/player_service.go
for an implementation of getting game data through RCON, and
backend/graphql/minecraft.resolvers.go
to see how that service is called from the GraphQL layer.
gateway serves both client and backend, and takes care of
connection routing. In particular, it:
- Routes /api/graphql and /api/graphiql to backend.
- Routes /* to client.
- Serves a socket.io server at /api/socket to
  relay WebRTC connection information between clients.
client is a React frontend that
exchanges audio with other clients using WebRTC, and
applies 3D effects with the
web audio API
using data from backend.

Advanced Usage

Virtual Player

Couldn’t manage to convince any friends to hang out with you on Minecraft?
Create a virtual player to test the platform!

When added, the VIRTUAL player will spawn at your current player location,
and make intermittent sounds so that you can test the platform’s 3D audio
capabilities during solo testing / development.

Client Overrides

The following global variables can be used to alter the behavior on client,
by typing them into the browser console.

You must apply them before connecting in order for them to take effect.

To skip player-validation in order to test audio conferencing capabilities
without Minecraft:
```
ZOOMCRAFT_SKIP_VALIDATION = true;
```
To change the maximum audible distance (after which other players are no
longer audible):
```
ZOOMCRAFT_MAX_DISTANCE = /* distance in blocks */
```
To change the rate at which player position data is updated:
```
ZOOMCRAFT_POLL_INTERVAL = /* duration in milliseconds */
```

To use custom ICE servers for WebRTC:

ZOOMCRAFT_ICE_SERVERS = [
  {
    urls: [
      /* ... */
    ],
  },
];

To change the WebRTC negotiation timeout:

ZOOMCRAFT_NEGOTIATION_TIMEOUT = /* duration in milliseconds */

TODO

Create a UI for changing the maximum audible distance.
Implement “rooms” that restrict communication to the players within an
in-game geofence.