Feed aggregator

Foster Interactive: DrupalNorth.org - Tales from the Trenches, Building the First Multilingual Conference Site in Drupal 8 Beta

Planet Drupal - Wed, 12/08/2015 - 10:00
Learn about our experience building DrupalNorth.org in Drupal 8 Beta. We were very impressed with D8's new features, but also shed a few (Many) tears with some very frustrating challenges in the build. Read our first impressions on Twig, Multilingual Support, Beta-to-Beta migrations, & Configuration Management. I've also added few tips for diving into D8, and highlighted some challenges I think will make scare off some early adopters from learning D8.
Categories: Elsewhere

Modules Unraveled: 144 Using the Open Sourced Red Test Framework Instead of Simpletest for Faster Drupal 7 Testing with Neerav Mehta - Modules Unraveled Podcast

Planet Drupal - Wed, 12/08/2015 - 07:00
Published: Wed, 08/12/15Download this episodeAutomated Testing
  • Before we actually get started talking about Red Test, can we take a step back and talk about what automated testing is in general?
    • I am sure you are all aware of manual testing and QA. Once you build a site or a feature in the site, you go to the browser and test it out. As an example, you added a workflow to your blog post. You will go to the browser and test whether the blog you create gets the draft state. Then you will log in as an editor and you will move it to published state. Now you will log out and check whether the blog post is visible to anonymous users. What you are doing is manual testing. It works very well for smaller projects.
    • For bigger projects, and especially for projects with multiple developers, what starts happening is adding a feature may break an older feature. Let’s take the example above. Earlier all the blog posts used to be published by the writer directly as soon as he saved the form and an email used to go out to all your blog subscribers that a new blog has been created. Now after adding the functionality to save the blog post as a draft state, you want the email to go out only after the blog post has been published. Suppose the developer changed and tested the workflow functionality without changing the email sending rule and pushed the feature to live site. You will soon be flooded by complaints from your blog subscribers.
    • This is a very simple example but such problems increase exponentially when the software grows in size over time. Such problems can be prevented by having automation testing in place. For each feature in your software, you write test code and you run all these tests often, and especially after working on a feature. You push your new feature to production only after all the tests pass. This way, as long as you have ample test coverage, you minimize your chances of finding a bug on the production site.
  • Who generally does this testing, the developer? site-builder? site-owner?
    • There are different schools of thought here. In practice, I have seen two different approaches:
      • If you are following test-driven development, then the developer writes these automated tests. In fact, he writes the tests before he even writes the code for the feature and makes sure that the tests fail. Then he implements the feature and makes sure that all the tests pass. In our company, our developers write the tests but that’s usually done after he has implemented the feature. We are experimenting with test driven development as of now.
      • The other approach I have seen in having a separate QA team for testing. They write automated tests as well as do manual QA. The advantage of this approach is that you can hire QA person, who is usually much cheaper than a developer. The problem I have seen in this approach is miscommunication between the developer and tester. Because of the delay due to communication and also since tester writes the tests after developer has completed the task, as compared to in parallel, it takes more time for a feature to get out of the door.
  • What do these automated tests test for?
    • Ideally everything but practically that’s not possible. As everything in life, this follows pareto rule. 20% of tests will cover 80% of the functionality and you should definitely write these 20% tests. In addition to that, we follow a rule that no bug should appear on production twice. So if a bug is found on production, we write an automated test for it so that it doesn’t appear on production again. What these tests should cover is very dependent on your application. If you are building an information blog site with a beautiful theme and you are making changes mostly to the theme, then you should write regressions tests for your CSS and JS. On the other hand, if your site has a lot of custom business logic, access permissions and rules, then focus on testing the logic.
  • I always assumed tests were for functionality, can you give me an example of something you would test on the theme layer?
  • I have to admit that I haven’t ever written an automated test for any of the sites I’ve built. I did have an experience a little over a year ago where the videos on my site (that were supposed to be behind a pay-wall) weren’t because I made a change to panels that messed up the access control. I didn’t realize it for two months! So, if I had had tests in place to check the access to those videos, I would have been in better shape. So, even though my site is a small site in the grand scheme of things, even I can bennefit from writting appropriate tests.
  • Red Test, as I understand it, is an open source integration testing framework for Drupal that you developed at Red Crackle. Can you tell us briefly what it does and why we should use it?
    • Red Test is an open-source integration testing framework for testing your Drupal applications. If you have multiple developers working on a big Drupal application stretching into months, you know that you need automated testing to keep adding more and more functionality without breaking old code. Red Test is ideal for testing:
      • Site structure and configuration
      • Custom business logic of your application
      • User access, roles and permissions
      • Rules
  • Drupal 7 supports Simpletest by default. Simpletest has unit testing and functional testing. Why do we need another automated testing solution? How is Red Test different from Simpletest and why should people use it?
    • You correctly mentioned that Drupal 7 supports Simpletest by default. In real life, when you are working on a big project, there are quite a few challenges when you test Drupal sites.
      • Drupal assumes that there is a persistent database connection available so any hook or function is free to access the database at will. This means that any function that accesses the database, unit testing is not possible. You can obviously refactor your code but that takes a long time. Also since Drupal stores all the configuration in the database, most of your custom code will need to access the database anyway.
      • For every test, simpletest creates a new test environment that has only minimal modules enabled. So if you want to test your current site with all the configuration, you have to first set all that configuration in code and then run the tests. That is too much of an overhead.
      • Functional testing in Simpletest is very slow because it uses a headless browser and every request needs to bootstrap Drupal. It’s not uncommon for a test suite on a large site to take multiple hours to finish.
    • Red Test alleviates these problems. It is an integration testing framework so it tests your site with the current configuration. It actually depends on the database so there is no need to refactor your code to make it work with Red Test. In fact, Red Test code is totally separate from Drupal code. We have created Red Test so that it runs in parallel on multiple processors of your machine and bootstraps Drupal only once at the start so it is 60 times faster than Simpletest.
  • A lot of Drupal developers have started using Behat which helps in testing your site functionally through a browser. With Behat gaining traction, is there still a need for Red Test?
    • Behat is an excellent tool and in fact, we use it in addition to Red Test. Since Red Test is an integration testing framework written in PHP and resides on the server, it can’t really test the javascript. So wherever JS functionality needs to be tested, Behat is the right tool. On the other hand, for testing all the other business logic in the application, we use Red Test. If you have used Behat on a big project, you will know that creating and especially maintaining Behat tests takes a long time. Every time an HTML id changes, Behat test needs to be changed.
    • Similarly, when you add a new required field in a form, Behat test needs to be updated to fill that field in the form. Red Test, on the other hand, knows about Drupal and its entities, nodes, people, roles and permissions. So it does a lot of tasks automatically in the background. As an example, if you added a new required field to a node form, Red Test will automatically fill suitable values in that field without you having to change anything in your tests. This makes it very easy for you to create and develop tests on Red Test.
    • In fact, we have done some measurement over the last couple of months and found that with the latest version of Red Test, creating and developing automated tests takes about 12% of total development time.
  • Is it easy to get started with writing automated tests using Red Test?
    • Yes, we are using all the standard PHP tools so it’s pretty easy for a developer to get started. Red Test uses Composer for installation and is based on PHPUnit. In fact, we measured how much time it takes a newbie to create the first integration test using Red Test and it comes to be a little less than 15 minutes. Below is the blog post you can follow to get started: http://redcrackle.com/red-test/documentation/first-integration-test
Use Cases
  • What are some concrete examples of what you should test?
    • Red Test is ideal for testing:
      • Site structure and configuration
      • Custom business logic of your application
      • User access, roles and permissions
      • Rules
    • If you are building just a basic informational site on Drupal without much functionality, you may not want to use Red Test. Use it only if you are building a large Drupal application.
  • What’s in the future for Red Test? Do you have improvement plans, or plans to integrate with D8?
    • Currently a developer needs to create tests using Red Test. We are considering whether enhancing it so that it works using Gherkin language makes sense. The next obvious step is to migrate it to Drupal 8.
Episode Links: Neerav on drupal.orgRed Crackle on TwitterBlog postWrite a Red Test in under 15 minutesTesting blocks using Red TestTags: Automated Testingplanet-drupal
Categories: Elsewhere

Realityloop: Automating image optimision with Drupal to improve performance

Planet Drupal - Wed, 12/08/2015 - 05:45
12 Aug Brian Gilbert

Recently I blogged 6 tips for manually optimising images, now I’m going to show you how to do it automatically with Drupal!

Some time ago we undertook a performance audit for the website of a substantial local news service. One of the areas I found that we could increase the performance of their website was to compress their numerous images.

After identifying the commands to apply using the various optimising programs we found that the ImageAPI Optimize module got us some of the way there. We were then able to develop a patch to implement the extra binaries and later gain co-maintainer status, allowing us to improve the module further.

What is the ImageAPI Optimize module

The ImageAPI Optimize module allows you to preprocess and optimise uploaded JPG and PNG images that get generated via an Image Style rather than use Drupal’s inbuilt compression which only affects JPG files.

Despite the name, the D7 version of this module does not depend on ImageAPI. It depends only on the core image.module.

It allows the use of multiple binaries, some of which are lossy compressors, so note that you may want to only use some of the tools listed below.

      • Download and enable the required module
      • Configure appropriately
        Navigate to /admin/config/media/image-toolkit in your site and select the ImageAPI Optimize radio button, then click Save.

        Now you will need to set your ImageAPI Optimize Base Toolkit, you’ll if you have Imagemagick on your server you may want to select that instead of GD.

        Then you want to configure the ImageAPI Optimize Service settings.

        To find the path for each of the binaries you can run the following commands in terminal on your server, one at a time, and copy the output into the relevant fields, if there is no output then you can follow the instructions in step 3 to install the binary in question.

        which advpng which optipng which pngcrush which advpng which advdef which pngout which jpegoptim which jfifremove

        Then in the advanced settings for the jpeg tools I recommend you set the output to be a progressive jpeg. For more on why this is a good idea see the following links:

      • Set-up image compressors on your webserver & development computer if they weren’t on your server

        I will only provide instructions for Debian based Linux variants, and OS X (where homebrew is installed) here.

        advpng (lossless) which is part of the advancecomp package http://www.advancemame.it/comp-readme.html
            Linux: apt-get install advancecomp
            OS X: brew install advancecomp

        OptiPNG (lossless) http://optipng.sourceforge.net
            Linux: apt-get install optipng (ensure you are using a version 0.7.5 or higher)
            OS X: brew install optipng

        Pngcrush (lossless) http://pmt.sourceforge.net/pngcrush/
            Linux: apt-get install pngcrush (ensure you are using a version 0.7.5 or higher)
            OS X: brew install pngcrush

        pngout (lossless) http://www.jonof.id.au/kenutils
            Linux: download binary archive and extract to /usr/local/bin
            OS X: download binary archive and extract to /usr/local/bin

        pngquant (lossy) https://pngquant.org/
            Linux: apt-get install pngcrush (ensure you are using a version 0.7.5 or higher)
            OS X: brew install pngquant

        jpegoptim (lossless & lossy) https://github.com/tjko/jpegoptim
            Linux: apt-get install jpegoptim
            OS X: brew install jpegoptim

        jpegtran (lossless) which is part of jpeglib package http://jpegclub.org/
            Linux: apt-get install libjpeg-progs
            OS X: brew install libjpeg

        jfifremove (lossless) http://lyncd.com/files/imgopt/jfifremove.c
        You need to compile it and then copy it into your path:

        wget http://lyncd.com/files/imgopt/jfifremove.c gcc -o jfifremove jfifremove.c mv jfifremove /usr/local/bin
What is the real world effect?

This acts on any image that is displayed via an image preset. The original images are not affected. I can’t personally think of a case any time recently where we are displaying original uploaded images so for our usage this is perfectly acceptable.

Filesize saving in a really simple test of an iPhone6+ screenshot in PNG format showed a reduction in size of around a quarter.

What to do If you are using shared hosting..

Odds are that you won’t be able to install the tools above using the method shown. You’ll likely need to create statically linked binaries so that you can upload them to the server.

A statically linked binary is an executable that doesn’t require any support libraries as they are included in the binary itself. This results in a binary that will run on any system of the right CPU type (i386, x86_64) The downside of this is that the resulting binary ends up being larger.

Normally, when you download a tarball of source code, you’ll do the standard “configure; make; make install” to build it. If you want a statically linked binary, replace the plain “make” with:

    make SHARED=0 CC='gcc -static'

You will need to do this on a machine with the same architecture as your shared hosting.

Pro tip

It’s possibly worth using the ImageCache Actions module to automatically convert gif’s to png, because there isn’t really any good case for using gif’s now, especially with the above in place, unless your site expects animated gifs to be uploaded but if displayed via an image style these animations would become static anyway.

drupal planet
Categories: Elsewhere


Subscribe to jfhovinne aggregator