Loading deck

Real-Time Capabilities for PHP, Serverless and Beyond

Kévin Dunglas

  • Les-Tilleuls.coop Founder
  • Symfony Core Team
  • API Platform Creator
  • Mercure Creator
  • Panther Creator

@dunglas

Les-Tilleuls.coop

🌍 API, web and cloud experts

👷 Consultancy, training, dev

✊ Self-managed, worker-owned cooperative

🦄 45 people, 1,000% growth in 6 years

💌 contact@les-tilleuls.coop

Real-time?

Mercure:
Push from Server to Clients

  • Push notifications

  • Synchronize connected devices in real-time

  • Collaborative editing (Google Docs-like)

  • Notify users when an async task has finished

 

Modern and high level alternative to WebSocket

Why a New Protocol?

Pros

  • Plain old HTTP
    • easy to implement
    • easy to debug
  • Short-lived connections (compatible with serverless, PHP…)
  • Supported by all browsers

Cons

  • Simplex communication
  • Inefficient
  • High power and resources consumption
  • Not true real-time

Timer-based Polling

Pros

  • High-level: built-in
    • re-connection
    • state reconciliation
  • Native implem in JS
  • Built on top of HTTP
    • Perfect with HTTP/2 & 3
  • Battery efficient
  • Connection-less push proxies (such as OMA push)

Cons

  • Simplex communication

  • Persistent connections

Server-Sent Events / EventSource (W3C)

Pros

  • Full-duplex communication
  • Low level: full control

Cons

  • Low level: no native
    • auth
    • re-connection
    • state reconciliation
    • events history
  • Obsoleted by HTTP/2 & 3
  • Hard to secure
  • Persistent connections

WebSocket (RFC 6455)

Web Push (RFC 8030)

👍 Works even when the webapp isn’t open

👎 Mostly designed for the Notifications API

👎 Restrictive quotas

👎 Limited payload size

👎 Proprietary hubs (Google Play & Apple Push)

WebSub (W3C)

👍 High level (auto-discoverable, auth...)

👍 Hub (open)

👎 Server-to-server only

Other Alternatives

The Persistent Connections Problem

  • WebSocket and SSE rely on persistent connections
  • Serverless platforms (AWS Lambda, Cloud Run, Azure functions…), PHP, FastCGI… are designed for short-lived connections
  • Persistent, long-lived, concurrent connections are better handled by dedicated and optimized software and hardware

The Mercure Protocol

Mercure, at a Glance

  • Full-duplex, but plain old HTTP
  • Native browser support, no SDK
  • Publish: HTTP POST
  • Subscribe: SSE
  • Built-in: reconnection, retrieving of lost messages, history
  • Auto-discoverable: designed for REST and GraphQL
  • JWT-based authorization mechanism (private updates)
  • Designed for serverless, PHP, FastCGI…
  • End-2-End encryption support

Mercure and HTTP/2+

HTTP/2 Multiplexing

HTTP/2 Support:

95% of All Internet Users

...and SSEs are also compatible with HTTP/1.x

SSE Native Support:

93% of All Internet Users

SSE/Mercure with Polyfill:

~100% of All Internet Users

HTTP/3

Get Started!

Start The Mercure Hub

# Run this command in the directory containing the assets of the demo interface
# Grab the hub corresponding to your system on
# https://github.com/dunglas/mercure/releases

$ ADDR=':3000' \
  JWT_KEY='!ChangeMe!' \
  ALLOW_ANONYMOUS=1 \
  CORS_ALLOWED_ORIGINS='http://localhost:8000' \
  DEMO=1 \
  mercure

# ADDR: The address to listen on (localhost:3000 here)
# JWT_KEY: The key used to sign the JWT, more about this later
# ALLOW_ANONYMOUS: Allow anonymous subscribers
# CORS_ALLOWED_ORIGINS: Allow Cross Origin connections from https://localhost:8000
# DEM0: Enable the demo web interface 

According to The 12-factor App methodology, configuration is done through environment variables.

The Demo Web Interface

Chrome's DevTools SSE Debugger

The Reference Hub

  • Implements 100% of the Mercure protocol
  • Fast, written in Go
  • Works everywhere: static binaries
  • Docker images and Kubernetes charts provided
  • Automatic HTTP/2 and HTTPS in prod (Let’s Encrypt)
  • CORS support, CSRF protection
  • Cloud Native (12Factor App)
  • Free Software (as a free speech - AGPL)
  • Managed and High Availability versions available in private beta (contact me)
  • Optional: a server can implement directly the protocol

Fast, really

  • Open Source version (EC2 t3.micro)
    • 40k concurrent connections
  • HA version (on premise)
    • 200k concurrent connections
  • Open Source load test provided

Alternative Hubs

  • NodeJS Hub: Ilshidur/node-mercure
  • Managed and on premise HA versions
  • And remember, the Hub is optional!
    • standalone Go library
    • standalone NodeJS library

Publishing

A Simple POST Request

curl \
  -d 'topic=http://example.com/my-resource' \
  -d 'data=My payload' \
  -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJtZXJjdXJlIjp7InB1Ymxpc2giOltdfX0.Oo0yg7y4yMa1vr_bziltxuTCqb8JVHKxp-f_FwwOim0' \
  -v \
  localhost:3000/hub

Using Plain PHP

<?php

// The publisher must be authenticated, more about that later!
define('JWT', 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJtZXJjdXJlIjp7InB1Ymxpc2giOltdfX0.Oo0yg7y4yMa1vr_bziltxuTCqb8JVHKxp-f_FwwOim0');

// simple form-urlencoded format
$postData = http_build_query([
    'topic' => 'http://example.com/my-resource',
    'data' => 'My payload',
]);

$headers = [
    'Content-type: application/x-www-form-urlencoded',
    'Authorization: Bearer '.JWT,
];

echo file_get_contents('http://localhost:3000/hub', false, stream_context_create(['http' => [
    'method'  => 'POST',
    'header'  => implode("\r\n", $headers),
    'content' => $postData,
]]));

Start a Web Server

# Disable HTTPS (you MUST use it in prod!)
$ symfony server:ca:uninstall
$ symfony serve -d

# Alternatively, use:
php -S localhost:8000 -t public/

The Mercure Symfony  component

<?php

// composer require symfony/mercure

define('JWT', 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJtZXJjdXJlIjp7InB1Ymxpc2giOltdfX0.Oo0yg7y4yMa1vr_bziltxuTCqb8JVHKxp-f_FwwOim0');

require __DIR__.'/../vendor/autoload.php';

use Symfony\Component\Mercure\Update;
use Symfony\Component\Mercure\Publisher;
use Symfony\Component\Mercure\Jwt\StaticJwtProvider;

$publisher = new Publisher('http://localhost:3000/hub', new StaticJwtProvider(JWT));

$update = new Update('http://example.com/my-resource', 'My payload');
echo $publisher($update);
  • Standalone library
  • Uses Symfony HttpClient internally

Subscribing

Subscribe

<script>
    const es = new EventSource('http://localhost:3000/hub?topic=http://example.com/my-resource');
    // Yes! EventSource is a native class, no NPM packages needed!
    es.onmessage = function (e) {
        console.log(e.data);
    };
</script>

Subscribe...

Using Modern JS

<script>
    const url = new URL('http://localhost:3000/hub');
    url.searchParams.append('topic', 'http://example.com/my-resource');

    const es = new EventSource(url);
    es.onmessage = ({data}) => console.log(data); // 🤯?
</script>

Subscribe to Several Topics

<script>
    const url = new URL('http://localhost:3000/hub');
    url.searchParams.append('topic', 'http://example.com/my-resource');
    url.searchParams.append('topic', 'http://example.com/another-resource');
    // topic=http://example.com/my-resource&topic=http://example.com/another-resource

    const es = new EventSource(url);
    es.onmessage = ({data}) => console.log(data);
</script>

Subscribe to Patterns

<script>
    const url = new URL('http://localhost:3000/hub');
    url.searchParams.append('topic', 'http://example.com/resources/{id}');
    // URI templates (RFC 6560)
    // http://example.com/resources/22 and http://example.com/resources/foo match

    const es = new EventSource(url);
    es.onmessage = ({data}) => console.log(data);
</script>

Authorization

Authorization

  • Uses JSON Web Token (JWT - RFC 7519)
  • An update can be intended for one or several targets
  • Publisher: must be authenticated
  • Subscriber:
    • Can be anonymous (if allowed by the config)
    • Must be authenticated to receive private updates
  • Two transports: cookie and Authorization header

The Mercure Claim

Root Access

Cookie-based authorization

Publisher

<?php
// composer require lcobucci/jwt
// Never implement JWT by yourself!!

require __DIR__.'/../../vendor/autoload.php';

use Lcobucci\JWT\Builder;
use Lcobucci\JWT\Signer\Hmac\Sha256;
use Symfony\Component\Mercure\Update;
use Symfony\Component\Mercure\Publisher;

$targets = [
    'http://example.com/users/1234',
    'http://example.com/groups/abc',
];

// Usually, you'll use pre-generated token for the server
$jwtProvider = function () use ($targets) : string {
    return (new Builder())
        ->set('mercure', ['publish' => $targets])
        // It must be the same key than the one used by the Hub (JWT_KEY)
        ->sign(new Sha256(), '!ChangeMe!')
        ->getToken();
};

$publisher = new Publisher('http://localhost:3000/hub', $jwtProvider);
$update = new Update('http://example.com/my-private-resource', 'My private payload', $targets);
echo $publisher($update);

Subscriber

<?php
require __DIR__.'/../../vendor/autoload.php';

use Lcobucci\JWT\Builder;
use Lcobucci\JWT\Signer\Hmac\Sha256;

$jwt = (new Builder())
        // set other appropriate JWT claims, such as an expiration date
        // try, then change the target, and check that you don't receive the update anymore
        ->set('mercure', ['subscribe' => ['http://example.com/users/1234']])
        // It must be the same key than the one used by the Hub (JWT_KEY)
        ->sign(new Sha256(), '!ChangeMe!')
        ->getToken();

setcookie('mercureAuthorization', $jwt, [
    'path' => '/hub',
    //'secure' => true, // only HTTPS, be sure to uncomment this line in prod
    'httponly'=> true, // not accessible in JavaScript
    'samesite' => 'strict', // CSRF protection
]);
?>
<script>
    const url = new URL('http://localhost:3000/hub');
    url.searchParams.append('topic', 'http://example.com/my-private-resource');

    // withCredentials is mandatory for the cookie to be send!
    const es = new EventSource(url, { withCredentials: true });
    es.onmessage = ({data}) => console.log(data);
</script>

The Discovery Mechanism

A Web API

<?php
header('Content-Type: application/ld+json');
header('Link: <http://localhost:3000/hub>; rel="mercure"', false);

echo json_encode(['@id' => 'http://example.com/my-resource', 'foo' => 'bar']);
// Better use API Platform!

Using the Link Header

<script>
    fetch('api.php') // Has this header `Link: <http://localhost:3000/hub>; rel="mercure"`
        .then(response => {
            // Extract the hub URL from the Link header
            const hubUrl = response.headers.get('Link').match(/<([^>]+)>;\s+rel=(?:mercure|"[^"]*mercure[^"]*")/)[1];

            return response.json().then(json => ({json, hubUrl}));
        })
        .then(({json, hubUrl}) => {
            console.log(json); // The document fetched from the API

            const url = new URL(hubUrl);
            url.searchParams.append('topic', json['@id']);

            const es = new EventSource(url);
            es.onmessage = ({data}) => console.log(data);
        })
    ;
</script>

Integrations with your Preferred Frameworks

Symfony

<?php

// composer require messenger
// set the correct env vars in .env
//
namespace App\Controller;

use Symfony\Bundle\FrameworkBundle\Controller\AbstractController;
use Symfony\Component\HttpFoundation\Response;
use Symfony\Component\Mercure\Update;
use Symfony\Component\Routing\Annotation\Route;

class MercureController extends AbstractController
{
    /**
     * @Route("/mercure", name="mercure")
     */
    public function index(): Response
    {
        // Sync, or async using RabbitMQ etc
        $this->dispatchMessage(new Update('http://example.com/my-resource', 'Payload'));

        return new Response('ok');
    }
}

API Platform

<?php

// composer require api
// bin/console doctrine:database:create
// bin/console doctrine:schema:create

namespace App\Entity;

use ApiPlatform\Core\Annotation\ApiResource;
use Doctrine\ORM\Mapping as ORM;

/**
 * @ApiResource(mercure=true)
 * @ORM\Entity
 */
final class Book
{
    /**
     * @ORM\Id
     * @ORM\Column(type="integer")
     * @ORM\GeneratedValue(strategy="AUTO")
     */
    public $id;

    /**
     * @ORM\Column(type="text")
     */
    public $title;
}

API Platform (subscriber)

<script>
    const url = new URL('http://localhost:3000/hub');
    url.searchParams.append('topic', 'http://localhost:8000/api/books/{id}');

    const es = new EventSource(url);
    es.onmessage = ({data}) => console.log(data);

    // As an exercise, try to use the discovery mechanism, it is supported by API Platform!
</script>

API Platform: clients

Thanks!

Any questions? 🍺