Hi

Branislav Bujisic, product engineer

Family guy

Bad accent included

The goal:

ensure quailty of software,
through frequent and comprehensive testing,
while maintaining high productivity

Quality control

• Coding standards
  because we need consistency
• Unit tests
  cover individual componets, smallest possible things to test
• Integration tests
  cover the interaction between components
• Acceptance tests
  evaluate the compliance with the business requirements
• E2E tests
  cover the entire flow of an app, from the start to the end

What can be automated?

Good quality control

Continuous Integration allows you to integrate the code into a shared repository and build and test each change automatically, as early as possible, usually several times a day.

Source: https://about.gitlab.com/product/continuous-integration

Even better quality control

Continuous Delivery ensures that the software can be released to production at any time, often by automatically pushing changes to a staging system.

Source: https://about.gitlab.com/product/continuous-integration

Quality control that you really trust

Continuous Deployment takes the process a step further and pushes changes to the production automatically.

Source: https://about.gitlab.com/product/continuous-integration

The pipeline

Source: https://about.gitlab.com/product/continuous-integration

Code review tools are logical candidates to run CI/CD pipelines

Lots of tools

Setting up Gitlab

Ubuntu 18.04, 2 cores, 8GB RAM

curl -LO https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh
sudo bash script.deb.sh
sudo apt install gitlab-ce

# /etc/gitlab/gitlab.rb
external_url 'https://example.com'
letsencrypt['contact_emails'] = ['admin@example.com']

sudo gitlab-ctl reconfigure

Source: https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-gitlab-on-ubuntu-18-04

Gitlab runners

• An isolated machine (a VM, a VPS, a bare-metal machine, a docker container, or a cluster of containers).
• Picks up jobs from the coordinator API of GitLab and runs them.
• Ideally, it should not run on the same server as GitLab.

Source: https://docs.gitlab.com/runner/

Setting up a GitLab runner

curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh | sudo bash
sudo apt-get install gitlab-runner

sudo gitlab-runner register

Sources:
https://docs.gitlab.com/runner/install/linux-repository.html
https://docs.gitlab.com/runner/register/index.html

Docker image

docker pull bbujisic/drupal8-phpunit:1.0

• PHP 7.2, composer, Drush
• PHPCS, Drupal Coder, PHPLint
• PHP Unit
• Open source

The recipe

image: bbujisic/drupal8-phpunit:1.0

stages:
  - test

phpcs:
  stage: test
  script:
    - phpcs --standard=Drupal web/modules/custom web/themes/custom

Our job today is to build the functional pipeline

• You decide: group or individual work?
• You choose the tests you need and design the pipeline

Preparation

• Open a Platform.sh account, use voucher code DDD2019
• Open a GitLab account
• Create a Drupal Umami at Platform.sh
• This deck as a reference: https://slides.bujisic.com/slides/gitlab-ci/workshop.html

Some suggestions for tasks to be automated

• PHP Parallel Lint
• PHP Code Sniffer
• Sensio Security Checker
• PHPStan
• PHP Unit
• Behat tests
• Deployment

PHP Parallel Lint

• Static analysis
• PHP Syntax checker

Installation and usage

composer require --dev "jakub-onderka/php-parallel-lint"

./bin/parallel-lint web/modules/custom/

Usage with GitLab CI

# .gitlab-ci.yml

image: bbujisic/drupal8-phpunit:latest
stages:
  - pre-deploy-test

parallel-lint:
  stage: pre-deploy-test
  script:
    -  parallel-lint web/modules/custom web/themes/custom

PHP Code Sniffer

• Static analysis
• Detects violations of the defined coding standards

Installation and usage

# For Drupal developers:
composer global require "drupal/coder"
# For non-Drupal developers:
composer global require "squizlabs/php_codesniffer=*"

phpcs --standard=Drupal web/modules/custom web/themes/custom

Usage with GitLab CI

# .gitlab-ci.yml

image: bbujisic/drupal8-phpunit:latest
stages:
  - pre-deploy-test

parallel-lint:
  stage: pre-deploy-test
  script:
    - phpcs --standard=Drupal web/modules/custom web/themes/custom

SensioLabs Security Checker

• Static analysis
• Runs composer.lock against SensioLabs database of vulnerable libraries

Installation and usage

php -r "readfile('http://get.sensiolabs.org/security-checker.phar');" > /usr/local/bin/security-checker
chmod +x /usr/local/bin/security-checker

security-checker security:check

Usage with GitLab CI

# .gitlab-ci.yml
image: bbujisic/drupal8-phpunit:latest
stages:
  - pre-deploy-test

security:
  stage: pre-deploy-test
  script:
    - security-checker security:check

PHPStan

• Static analysis
• finds errors in the code without actually running it
• Requires core and contrib dependencies to be present

Installation and usage

# For Drupal developers:
composer require "phpstan/phpstan"
# For non-Drupal developers:
composer require "mglaman/phpstan-drupal"

# phpstan.neon

includes:
  - vendor/mglaman/phpstan-drupal/extension.neon

vendor/bin/phpstan analyse web/modules/custom web/themes/custom

Usage with GitLab CI

# phpstan.neon
includes:
  - %rootDir%/../../mglaman/phpstan-drupal/extension.neon
parameters:
  excludes_analyse:
    - *Test.php
    - *TestBase.php
  level: 1

# .gitlab-ci.yml
image: bbujisic/drupal8-phpunit:latest
stages:
  - pre-deploy-test

phpstan:
  stage: pre-deploy-test
  script:
    - composer install
    - phpstan web/modules/custom web/themes/custom

PHPUnit

• Executes PHP code
• Unit and integration tests
• Requires composer install
• Doesn't require the state of the app

Installation and usage

composer global require phpunit/phpunit

phpunit -c phpunit.xml web/modules/custom

Usage with GitLab CI

# .gitlab-ci.yml
image: bbujisic/drupal8-phpunit:latest
stages:
  - pre-deploy-test

phpunit:
  stage: pre-deploy-test
  script:
    - composer install
    - phpunit web/modules/custom

Behat

• Executes PHP
• Needs the entire state of the app

Installation (1)

// composer.json
"require": {
  "drupal/drupal-extension": "^3.4",
},
"config": {
    "bin-dir": "bin/"
}

composer update
mkdir behat
cd behat
../bin/behat --init

Installation (2)

# behat/behat.yaml
default:
  suites:
    default:
      contexts:
        - FeatureContext
        - Drupal\DrupalExtension\Context\DrupalContext
        - Drupal\DrupalExtension\Context\MinkContext
        - Drupal\DrupalExtension\Context\MessageContext
        - Drupal\DrupalExtension\Context\DrushContext
  extensions:
    Behat\MinkExtension:
      goutte: ~
      selenium2: ~
      base_url: http://mysite.local
    Drupal\DrupalExtension:
      blackbox: ~
      api_driver: 'drupal'
      drush:
        alias: 'local'
      drupal:
        drupal_root: '../web/'
      region_map:
        footer: "#footer"

Usage

# behat/features/hello.feature
@api
  Scenario: An anonymous should see the hello page
    Given I am an anonymous user
    When I go to "hello"
    Then I should see "Hello world"

cd behat; behat

Usage with GitLab CI

• You need a working Drupal installation in a Docker container!
• Your Docker container should be as close to production as possible
• Long and boring setup; long and boring execution of each of the tests
• Alternative: deploy to your staging environment first, and then run the tests

Usage with GitLab CI and Platform.sh (1)

# behat/behat.yaml
default:
  extensions:
    Behat\MinkExtension:
      base_url: http://mysite.local

To get the environment URL, run:

platform url --pipe | head -n 1

Then use the BEHAT_PARAMS environment variable to override parts of the behat.yaml file.

{"extensions": {
  "Behat\\MinkExtension":
    {"base_url":"[platform-url]"}
  }
}

Usage with GitLab CI and Platform.sh (2)

# .gitlab-ci.yml

image: bbujisic/drupal8-phpunit:latest
stages:
  - post-deploy-test

behat:
  stage: post-deploy-test
  script: |
    bash scripts/gitlab/setup-cli.sh
    platform variable:create \
      --name=env:BEHAT_PARAMS \
      --value="{\"extensions\":{\"Behat\\\\MinkExtension\":{\"base_url\":\"`platform url --environment=$CI_BUILD_REF_NAME --pipe | head -n 1`\"}}}" \
      --level=environment --json=true \
      --environment=$CI_BUILD_REF_NAME \
      --yes || true
    platform ssh "cd behat; behat" -e $CI_BUILD_REF_NAME

Deploy to staging

Platform.sh solves the problem of staging environments and stakeholder acceptance

A new staging environment

$ git push platform my-feature-branch
$ platform environment:activate
                    

$ platform push
                    

Gitlab + Platform.sh

1. Use the environment variables to store the ssh keys
2. Setup ssh on docker container build
3. Push to platform

Gitlab environment variables

SSH setup and push

stages:
  # ...
  - deploy
# ...
psh-deploy:
  stage: deploy
  script:
    - mkdir -p $HOME/.ssh
    - echo "$SSH_KEY" > $HOME/.ssh/id_rsa
    - echo "$SSH_KEY_PUB" > $HOME/.ssh/id_rsa.pub
    - echo "$SSH_KNOWN_HOSTS" > $HOME/.ssh/known_hosts
    - chmod go-r $HOME/.ssh/id_rsa
    - platform project:set-remote "$PSH_PROJECT_ID"
    - platform push --force --activate --target=$CI_BUILD_REF_NAME

setup-cli.sh file: a bit of cleanup

#!/usr/bin/env bash
set -e

# Set up SSH credentials for pushing to external Git repositories, via GitLab CI
# environment variables.

if [ -n "$SSH_KEY" ]; then
  mkdir -p $HOME/.ssh
  echo "$SSH_KEY" > $HOME/.ssh/id_rsa
  echo "$SSH_KEY_PUB" > $HOME/.ssh/id_rsa.pub

  chmod go-r $HOME/.ssh/id_rsa
  unset SSH_KEY

  echo "Created SSH key: .ssh/id_rsa"
fi

# Set up SSH known hosts file.
if [ -n "$SSH_KNOWN_HOSTS" ]; then
  mkdir -p $HOME/.ssh
  echo "$SSH_KNOWN_HOSTS" > $HOME/.ssh/known_hosts
fi

platform project:set-remote "$PSH_PROJECT_ID"