Loading deck

Kévin Dunglas

  • Les-Tilleuls.coop Founder
  • Symfony Core Team
  • API Platform Creator
  • Mercure Creator
  • Panther Creator

@dunglas

Les-Tilleuls.coop

🌍 API, web and cloud experts

👷 Consultancy, training, dev

✊ Self-managed, worker-owned cooperative

🦄 45 people, 1,000% growth in 6 years

💌 contact@les-tilleuls.coop

Our Project:
A Hacker News Clone

Get Started

Create the Symfony App

# Use symfony/website-skeleton
$ symfony new --full my-hn-clone
$ cd my-hn-clone

# Start the Symfony web server
$ symfony serve -d

Configure and Test the DB

# Edit .env
DATABASE_URL=mysql://admin@127.0.0.1:3306/pantherws

# Then create the DB:
$ bin/console doctrine:database:create

Make the CRUD!

Symfony MakerBundle

  • Generate boilerplate code for controllers, forms, commands, tests...
  • Doctrine entity, CRUD, API (Platform)... makers
  • Panther support 😻

Make the Doctrine Entity

$ bin/console make:entity

 Class name of the entity to create or update (e.g. FierceGnome):
 > Item

 Mark this class as an API Platform resource (expose a CRUD API for it) (yes/no) [no]:
 > no

 [snip]

 New property name (press <return> to stop adding fields):
 > storyLink

 Field type (enter ? to see all types) [string]:
 > string

 Field length [255]:
 > 2048

 Can this field be null in the database (nullable) (yes/no) [no]:
 > no

[snip]

 Add another property? Enter the property name (or press <return> to stop adding fields):
 > publishedAt          

 Field type (enter ? to see all types) [datetime]:
 > datetime

 Can this field be null in the database (nullable) (yes/no) [no]:
 > no

Create and Run the Migration

$ bin/console make:migration
$ bin/console doctrine:migrations:migrate
# Also run the migration in the test database
$ bin/console doctrine:migrations:migrate -e test

Make the CRUD

$ bin/console make:crud Item

OK we need CSS...

But that was easy!

 

Testing

a Symfony App

3 components

to Emulate a Browser

A Functional Test

Make a Functional Test Skeleton

$ bin/console make:functional-test ItemControllerTest

Your First Test!

<?php

namespace App\Tests;

use Symfony\Bundle\FrameworkBundle\Test\WebTestCase;

class ItemControllerTest extends WebTestCase
{
    public function testCreateItem(): void
    {
        $client = static::createClient();
        // Load the index
        $client->request('GET', '/item/');
        $client->followRedirects();

        $this->assertPageTitleSame('Item index');
        $client->clickLink('Create new');

        // On the creation form
        $this->assertPageTitleSame('New Item');
        $this->assertSelectorTextSame('body > h1', 'Create new Item');

        // Fill and submit the form
        $client->submitForm('Save', [
            // item[storyLink] is the name of the form field
            'item[storyLink]' => 'https://masterclass.les-tilleuls.coop',
            'item[publishedAt][date][month]' => '9',
            'item[publishedAt][date][day]' => '18',
            'item[publishedAt][date][year]' => '2019',
        ]);

        $this->assertSelectorTextContains('table', 'https://masterclass.les-tilleuls.coop');
    }
}

Configure PHPStorm

Run The Test

Click ▶️ in PHPStorm, or run:

$ vendor/bin/simple-phpunit

✅ all green!

Functional Tests in SF

  • KernelTestcase:

    • extends PHPUnit’s TestCase
    • boots the Symfony app
    • allows to access to the Symfony kernel
  • WebTestCase:
    • extends KernelTestcase
    • uses BrowserKit to simulate a browser (that uses DomCrawler and CssSelector)
    • creates HTTP Foundation’s Request and allows to inspect Response through a clean API

Oops, I Made a Mistake!

{# templates/item/new.html.twig #}

{% extends 'base.html.twig' %}

{% block stylesheets %}
<style>
    /* That's a big bug */
    #item {
        visibility: hidden;
    }
</style>
{% endblock %}

{# ... #}

Run The Test Again

Click ▶️ in PHPStorm, or run:

$ vendor/bin/simple-phpunit

Still all green 😿!

BrowserKit is a pure PHP lib with no support for

  • CSS rendering
  • JavaScript
  • Canvas, WebGL, WebSocket, SSE...

Introducing Panther

Symfony Panther

  • Executes real web browsers
  • Same features than browsers, JS and CSS
  • Leverages the WebDriver Protocol
  • Implements (most of) BrowserKit’s API
  • Provide convenient methods for JS testing:
    wait, execute script, screenshot…
  • Standalone lib, for testing and web scraping

Install

$ composer require panther
<!-- phpunit.xml.dist -->

     <!-- ... -->
+    <extensions>
+        <extension class="Symfony\Component\Panther\ServerExtension" />
+    </extensions>
 </phpunit>

The extension is optional but:

  • improves performance
  • allows to use the interactive mode

That's All!

  • Nothing to install
  • Nothing to configure
    • neither Selenium,
    • nor a web server

Panther just works!

Usage

// tests/ItemControllerTest.php
 
 namespace App\Tests;
 
-use Symfony\Bundle\FrameworkBundle\Test\WebTestCase;
+use Symfony\Component\Panther\PantherTestCase;
 
-class ItemControllerTest extends WebTestCase
+class ItemControllerTest extends PantherTestCase
 {
     public function testCreateItem(): void
     {
-        $client = static::createClient();
+        $client = static::createPantherClient();
         // Load the index

😺

Debug and Fix

$ PANTHER_NO_HEADLESS=1 ./vendor/bin/simple-phpunit --debug

Hacker News, it's

all about comments

A Better Comment System for HN

  • Lazy loading
  • No page reload after post
  • Nice UI...

 

We need JavaScript!

Install API Platform

$ composer require api

Create a Comment API

bin/console make:entity

 Class name of the entity to create or update (e.g. FierceChef):
 > Comment

 Mark this class as an API Platform resource (expose a CRUD API for it) (yes/no) [no]:
 > yes

[snip]

 New property name (press <return> to stop adding fields):
 > body

 Field type (enter ? to see all types) [string]:
 > text

 Can this field be null in the database (nullable) (yes/no) [no]:
 > no

 updated: src/Entity/Comment.php

 Add another property? Enter the property name (or press <return> to stop adding fields):
 > author

 Field type (enter ? to see all types) [string]:
 > string

 Field length [255]:
 > 

 Can this field be null in the database (nullable) (yes/no) [no]:
 > no

[snip]

 Add another property? Enter the property name (or press <return> to stop adding fields):
 > publishedAt

 Field type (enter ? to see all types) [datetime]:
 > datetime

 Can this field be null in the database (nullable) (yes/no) [no]:
 > no

 updated: src/Entity/Comment.php

 Add another property? Enter the property name (or press <return> to stop adding fields):
 > item 

 Field type (enter ? to see all types) [string]:
 > relation

 What class should this entity be related to?:
 > Item

[snip]

 Relation type? [ManyToOne, OneToMany, ManyToMany, OneToOne]:
 > ManyToOne

 Is the Comment.item property allowed to be null (nullable)? (yes/no) [yes]:
 > no

 Do you want to add a new property to Item so that you can access/update Comment objects from it - e.g. $item->getComments()? (yes/no) [yes]:
 > yes

 A new property will also be added to the Item class so that you can access the related Comment objects from it.

 New field name inside Item [comments]:
 > comments

 Do you want to activate orphanRemoval on your relationship?
 A Comment is "orphaned" when it is removed from its related Item.
 e.g. $item->removeComment($comment)
 
 NOTE: If a Comment may *change* from one Item to another, answer "no".

 Do you want to automatically delete orphaned App\Entity\Comment objects (orphanRemoval)? (yes/no) [no]:
 > yes

$ bin/console make:migration
$ bin/console doctrine:migrations:migrate

The Comment API is (Almost) Ready!

Disable Pagination, Default Publication Date

// src/Entity/Comment.php
// ...

/**
- * @ApiResource()
+ * @ApiResource(paginationEnabled=false)
  * @ORM\Entity(repositoryClass="App\Repository\CommentRepository")
  */
 class Comment

// ...

     private $item;
 
+    public function __construct()
+    {
+        $this->publishedAt = new \DateTime();
+    }
+

A Read Only API for Items, and a Subresource

// src/Entity/Item.php
 
 namespace App\Entity;
 
+use ApiPlatform\Core\Annotation\ApiResource;
+use ApiPlatform\Core\Annotation\ApiSubresource;
 use Doctrine\Common\Collections\ArrayCollection;
 use Doctrine\Common\Collections\Collection;
 use Doctrine\ORM\Mapping as ORM;
 
 /**
  * @ORM\Entity(repositoryClass="App\Repository\ItemRepository")
+ * @ApiResource(
+ *     itemOperations={"get"},
+ *     collectionOperations={"get"}
+ * )
  */
 class Item

[...]

     /**
+     * @ApiSubresource
      * @ORM\OneToMany(targetEntity="App\Entity\Comment", mappedBy="item", orphanRemoval=true)
      */
     private $comments;

The Latest

Trendy JS "Framework"?

Not Today 😼

Vanilla JS Comment System

{# templates/item/show.html.twig #}
{% extends 'base.html.twig' %}

{% block javascripts %}
    {# Better use Symfony Webpack Encore to generate cacheable external JS files #}
    <template id="hn-comments-template">
        <h1>Comments</h1>

        <div id="comments">Loading...</div>

        <form id="post-comment">
            <style>.hidden { display: none; }</style>
            <div id="status" class="hidden">Comment published!</div>
            <textarea name="body" placeholder="Your comment"></textarea>
            <input name="author" placeholder="Your name"/>

            <input type="submit" value="Post">
        </form>
    </template>

    <template id="hn-comment-template">
        <div class="comment">
            <article><!-- body --></article>
            <p><!-- author --></p>
            <time><!-- publishedAt --></time>
        </div>
    </template>
    <script>
        class HNComments extends HTMLElement {
            constructor() {
                super();

                this.itemID = this.getAttribute('item-id');

                const rootNode = document.getElementById('hn-comments-template').content.cloneNode(true);
                this.appendChild(rootNode);
                this.comments = document.getElementById('comments');

                this.commentTmpl = document.getElementById('hn-comment-template');

                const form = document.getElementById('post-comment');
                const status = document.getElementById('status');
                const self = this;

                // post a new comment
                form.onsubmit = function (e) {
                    e.preventDefault();

                    fetch(`/api/comments`, {
                        method: 'POST',
                        headers: {'Content-Type': 'application/ld+json'},
                        body: JSON.stringify({
                            item: `/api/items/${self.itemID}`,
                            author: this.author.value,
                            body: this.body.value,
                        })
                    })
                    // handle server-side errors, network failures etc...
                    .then(() => self.fetchAndDisplayComments()) // Refresh list
                    .then(() => {
                        status.classList.replace('hidden', 'displayed');
                        form.reset();
                    })
                }
            }

            connectedCallback() {
                // Called every time the custom element is inserted into the DOM.
                this.fetchAndDisplayComments();
            }

            fetchAndDisplayComments() {
                return fetch(`/api/items/${this.itemID}/comments`)
                    .then(response => response.json())
                    .then(json => {
                        if (json['hydra:member'].length === 0) {
                            this.comments.innerText = 'No comments';

                            return;
                        }

                        this.comments.innerText = ''; // Reset
                        json['hydra:member'].forEach(comment => {
                            const commentNode = this.commentTmpl.content.cloneNode(true);
                            commentNode.querySelector('article').innerText = comment.body;
                            commentNode.querySelector('p').innerText = comment.author;
                            commentNode.querySelector('time').innerText = comment.publishedAt;

                            this.comments.appendChild(commentNode);
                        })
                    });
            }
        }

        customElements.define('hn-comments', HNComments);
    </script>
{% endblock %}

{% block title %}Item{% endblock %}

{% block body %}
    {# ... #}
    <hn-comments item-id="{{ item.id|escape('html_attr') }}"></hn-comments>
{% endblock %}

Test JS Apps With Panther!

namespace App\Tests;

use Symfony\Component\Panther\PantherTestCase;

class ItemControllerTest extends PantherTestCase
{
    // ...
    public function testComment(): void
    {
        $client = static::createPantherClient();
        $client->request('GET', '/item/1'); // 🔍🙀

        // Panther's magic: wait for the form to appear!
        $client->waitFor('#post-comment');

        $client->submitForm('Post', [
            'body' => 'Very interesting!',
            'author' => 'bob',
        ]);

        // Wait for the post to be processed server-side, fetched and displayed
        $client->waitFor('#status.displayed');

        $this->assertSelectorTextContains('#comments', 'Very interesting!');
    }
}

How About Shadow DOM?

  • Shadow DOM plays well with Web Components...
  • ... but it's not easy (yet) to access it using WebDriver
  • Still, workarounds exist
  • At some point, we'll surely add dedicated helpers in Panther to access it (PR welcome!)

Database Testing

Database Testing

Alice: Panther's Best Friend

What If the DB is empty?

$client->request('GET', '/item/1'); // 🔍🙀

We Need Test Fixtures!

Meet Alice

$ composer require --dev alice
  • Alice: fixtures generator, with an expressive syntax
  • Faker: the pseudo-random data generator used by Alice
  • Support for Doctrine ORM, ODM and Eloquent
  • DB testing helpers

Write the Fixtures

# fixtures/items.yaml

App\Entity\Item:
  item_{0..10}:
    storyLink: '<url()>'
    publishedAt: '<dateTime()>'

App\Entity\Comment:
  comment_{0..200}:
    body: '<paragraph()>'
    author: '<name()>'
    publishedAt: '<dateTime()>'
    item: '@item_*'
$ bin/console hautelook:fixtures:load

And Load Them

ReloadDatabaseTrait

 namespace App\Tests;
 
+use Hautelook\AliceBundle\PhpUnit\ReloadDatabaseTrait;
 use Symfony\Component\Panther\PantherTestCase;
 
 class ItemControllerTest extends PantherTestCase
 {
+    // The database will be purged and the fixtures loaded between all tests
+    use ReloadDatabaseTrait;

Use a Standalone DB for Panther

# By default, Panther will run the app under test
# using the "panther" Symfony environment.
# You can configure this env in .env.panther
DATABASE_URL=mysql://admin@127.0.0.1:3306/my-test-db

Inside the Panther

ChromeDriver

  • Panther ships ChromeDriver binaries
  • ChromeDriver is a standalone server, exposing a Selenium/W3C compliant WebDriver API
  • ChromeDriver automatically finds and drives the local Chrome install
  • By default, Panther starts Chrome in headless mode
    • Yes, it also works with Docker, Vagrant, Travis, GitLab CI...

Panther and WebDriver

  • ChromeDriver is not mandatory
  • Panther has experimental support for Geckodriver (same as ChromeDriver, but for Firefox)
  • Panther is compatible with any WebDriver server, including the Selenium server and Selenium Grid
  • Compatible with remote testing services such as:

The Built-in Web Server

  • Panther transparently serves your project using the builtin PHP web server
    • finds the PHP binary
    • finds the document root (public/ with Flex)
    • starts the project in the "panther" environment
      • Yes, even on Windows!
  • Using your custom web server (NGINX, Apache...) is also supported!

The APIs you Love

// the Panther Client implements:

// BrowserKit and DomCrawler APIs
$crawler = $client->request('GET', '/');
$crawler->filter('body > p')->siblings();
$crawler->filterXPath('//body/p')->attr('class');

// Docs:
//   * https://symfony.com/doc/current/components/browser_kit.html
//   * https://symfony.com/doc/current/components/dom_crawler.html

// Facebook's PHP WebDriver API:
$client->findElement('#my-elem')->click();
$cookieContent = $client->executeScript('return document.cookie;');
// Using the previous one, you can workaround all WebDriver limitations!

// Docs:
//   * https://github.com/facebook/php-webdriver/wiki

Some Sexy Extensions

// Screenshot
$client->takeScreenshot('export.png');

// Mouse API
$mouse = $client->getMouse();
mouse->mouseMoveTo('#my-element');
mouse->clickTo('#my-element');

Be Ready For Tomorrow

Using Panther with Mercure

Install the Mercure support and Start the Hub

# Install the Symfony integration
$ composer require mercure

# Start the Mercure Hub
$ ADDR=':3000' \
    JWT_KEY='!ChangeMe!' \
	ALLOW_ANONYMOUS=1 \
	CORS_ALLOWED_ORIGINS='https://localhost:8000,http://127.0.0.1:9080' \
	./mercure/mercure

# http://127.0.0.1:9080 is the URL of Panther's internal web server

Enable Mercure

# .env.php
MERCURE_PUBLISH_URL=http://localhost:3000/hub
// src/Entity/Comment.php

/**
- * @ApiResource(paginationEnabled=false)
+ * @ApiResource(
+ *     paginationEnabled=false,
+ *     mercure=true
+ * )
  * @ORM\Entity(repositoryClass="App\Repository\CommentRepository")
  */
 class Comment

Subscribe to Updates

{# templates/item/show.html.twig #}
{% block javascripts %}
{# ... #}
    <script>
        const commentTmpl = document.getElementById('hn-comment-template');

        const hubURL = new URL('http://localhost:3000/hub');
        hubURL.searchParams.append('topic', `${window.origin}/api/comments/{id}`);

        const es = new EventSource(hubURL);
        es.onmessage = function (e) {
            const comment = JSON.parse(e.data);
            if (comment.item !== '/api/items/{{ item.id|escape('js') }}') {
                // Comment of another post
                return;
            }

            const commentNode = commentTmpl.content.cloneNode(true);
            commentNode.querySelector('.comment').classList.add('mercure');
            commentNode.querySelector('article').innerText = comment.body;
            commentNode.querySelector('p').innerText = comment.author;
            commentNode.querySelector('time').innerText = comment.publishedAt;

            document.getElementById('comments').appendChild(commentNode);
        }
    </script>

// You must also comment this line:
.then(() => self.fetchAndDisplayComments()) // Refresh list

You Can Use Several Browsers in Your Tests!

class ItemControllerTest extends PantherTestCase
{
// ...
    public function testComment(): void
    {
        $client = static::createPantherClient();
        $client->request('GET', '/item/1');

        // Another isolated browser
        $client2 = static::createAdditionalPantherClient();
        $client2->request('GET', '/item/1');

        $client->waitFor('#post-comment');

        $client->submitForm('Post', [
            'body' => 'Very interesting!',
            'author' => 'bob',
        ]);

        $client->waitFor('.mercure');
        $this->assertSelectorTextContains('.mercure', 'Very interesting!');

        $crawler2 = $client2->waitFor('.mercure');
        $this->assertStringContainsString(
            'Very interesting!',
            $crawler2->filter('.mercure')->text()
        );
    }
}