MojangPipe is a proxy & relay server to Mojang's UUID -> Profile, Name -> UUID, UUID -> Name history APIs.
As MCHeads keeps growing, the task of contacting Mojang's API locally gets more complicated, causing high database traffic and slow response times. MojangPipe solves these issues by acting as a middleman between MCHeads and Mojang while caching recent requests and using HTTP proxies when contacting Mojang's API to avoid various rate limits.
- SparkJava Web app micro framework
- Squid HTTP proxy server
- OkHTTP 3
- Redis
- Order of operation per request:
- Checking if there was already a failed attempt to serve this request within the last 4 hours - if there was, just return 204 status code with a blank body.
- If there was not - checking if this request was already successfully served within the last 30 minutes - if it was, serve the cached version.
- If not, use the current proxy in rotation to make the necessary API request.
- Serve, cache the request and record statistics.
We use Squid as our HTTP proxy server, hosted locally on the same machine as the MojangPipe instance. Here is a quick guide on how to install Squid, and the config file we used for squid.
Since Squid is configured to send requests through each one of its IPs depending on which port it was connected to, and squid is hosted locally on the same machine is the relay server, i didn't bother writing MojangPipe in a way that would let each server address in the proxies hash field to be customized, instead, it just stores the ports on which squid runs and selects a port based on the timestamp of the last time this proxy was in rotation.
Currently, in the production environment, this system has managed to reduce the rate of 429 status codes from 6% (of all outgoing requests to Mojang), to 0.5 - 0.02%.
To contribute:
- Fork the project.
- Make a branch for each thing you want to do (don't put everything in your master branch: I don't want to cherry-pick and I may not want everything).
- Send a pull request.