Planet Drupal

Subscribe to Planet Drupal feed
Drupal.org - aggregated feeds in category Planet Drupal
Updated: 34 min 30 sec ago

Drupal Commerce: Using OpenID Connect for Single Sign-On with Drupal

Tue, 17/02/2015 - 16:03

At Commerce Guys we provide a varied range of services, including our cloud PaaS Platform.sh, this Drupal Commerce community website, support, and the Commerce Marketplace.

Our users may need to log in to any of these services, and sometimes several at the same time. So we needed to have a shared authentication system, a way of synchronizing user accounts, and single sign-on (SSO) functionality.

After a lot of research on the existing methods, such as CAS, we found that there was no generic open-source solution which would cover all of our current needs and would also allow us to grow and scale in the future when adding new features or applications.

We decided to implement the OAuth 2.0 and OpenID Connect protocols, which were designed to be flexible, yet simple and standardized - exactly what we wanted.

Categories: Elsewhere

Drupal @ Penn State: Autopost to Facebook

Tue, 17/02/2015 - 15:15

I ran into an issue with the Drupal for Facebook module, both for D6 and D7, where I wanted articles to auomatically be posted to Facebook when they are submitted.  There appeared to be no way to do this via the module and I had played around with Rules to see if that would work, but no luck.

Categories: Elsewhere

Colan Schwartz: Integrating remote data into Drupal 7 and exposing it to Views

Mon, 16/02/2015 - 20:45
Topics: 

Drupal's strength as a content management framework is in its ability to effectively manage and display structured content through its Web user interface. However, the out-of-the-box system assumes all data is local (stored in the database). This can present challenges when attempting to integrate remote data stored in other systems. You cannot, by default, display non-local records as pages. While setting this up is in itself a challenge, it is an even bigger challenge to manipulate, aggregate and display this data through Views.

I've split this article into the following sections and subsections. Click on any of these to jump directly to the one of them.

  1. Introduction
  2. What's Changed
  3. Architecture
    1. Remote entity definition
    2. Access to remote properties
    3. Remote property definition
    4. Entity instances as Web pages
    5. Web services integration
    6. Temporary local storage
    7. Implementing the remote connection class
    8. Implementing the remote query class
  4. Views support
    1. Basic set-up
    2. Converting from an EntityFieldQuery
  5. Alternatives
  6. References
Introduction

This exposition is effectively a follow-up to some excellent articles from years past:

I'd recommend reading them for background information.

The first article (written in the Drupal 6 days) describes a "Wipe/rebuild import" method (Method 3) to bring remote data into Drupal. That's basically what we'll be discussing here, but there is now a standard method for doing so. What's interesting is that future plans mentioned there included per-field storage engines (with some being remote). The idea never made it very far. This is most likely because grabbing field data from multiple locations is far too inefficient (multiple Web-service calls) compared to fetching an entire record from a single location.

Taking a look at the second article, you can now see that Drupal 7 is dominant, and we have more tools at our disposal, but at the time this one was written, we still didn't have all of them in place. We did, however, have the following APIs for dealing with entities.

  1. The entity API in Drupal Core
  2. The Entity API contributed module
What's Changed

We now have another API, the Remote Entity API, which was inspired by Florian's article. As you can imagine, this API is dependent on the Entity API which is in turn dependent on the Drupal Core's entity functionality.

I recently added support for this new API to EntityFieldQuery Views Backend, the module allowing Views to work with data stored outside of the local SQL database. Previously, it supported non-SQL data, but still assumed that this data was local. Tying these two components together gives us what we need to achieve our goal.

Architecture

So we really need to take advantage of the three (3) entity APIs to load and display individual remote records.

  1. The entity API in Drupal Core
  2. The Entity API contributed module
  3. The Remote Entity API contributed module

The first provides basic entity functionality in Drupal. The second adds enhanced functionality for custom entities. The third and final API adds additional handling mechanisms for working with any remote data.

We'll need the following contributed modules to make all of this work.

In addition to the above, a new custom module is necessary. I recommend something like siteshortname_entities_remote for the machine name. You can have another one, siteshortname_entities_local, for local entities without all of the remote code if necessary. In the .info file, add remote_entity (the Remote Entity API) as a dependency.

You'll want to divide your module file into at least three (3) parts:

  1. Entity APIs: Code for defining remote entities through any of the above entity APIs. (Part I)
  2. Drupal Core: Code for implementing Drupal Core hooks. This is basically a hook_menu() implementation with some helper functions to get your entity instances to show up at specific paths based on the remote entity IDs. (Part II)
  3. Web Service Clients: Code for implementing what's necessary for the Web Service Clients module, a prerequisite for the Remote Entity API. It's essentially the external communications component for accessing your remote data. (Part III)

Most of the code will be in PHP class files you'll want in a classes subdirectory (autoloaded by defining these in your .info file), but you'll still need some code in your main module file.

We'll be adding only one new entity in this exercise, but the code is extensible enough to allow for more. Once one of these is set up, adding more is (in most cases) trivial.

Remote entity definition

Your basic remote entity definitions will exist in the Entity APIs section of your module file, Part I. Within the hook_entity_info() implementation, you'll see that different properties within the definition will be used by different layers, the three APIs.

For the following examples, let's assume we have a remote event data type.

<?php
/****************************************************************************
 ** Entity APIs
 ****************************************************************************/

/**
 * Implements hook_entity_info().
 *
 * @todo Add 'bundles' for different types of remote content.
 * @todo Add 'entity keys' => 'needs remote save' if remote saving required.
 * @todo Remove 'static cache' and 'field cache' settings after development.
 */
function siteshortname_entities_remote_entity_info() {
  $entities['siteshortname_entities_remote_event'] = array(

    // Core properties.
    'label' => t('Event'),
    'controller class' => 'RemoteEntityAPIDefaultController',
    'base table' => 'siteshortname_entities_remote_events',
    'uri callback' => 'entity_class_uri',
    'label callback' => 'remote_entity_entity_label',
    'fieldable' => FALSE,
    'entity keys' => array(
      'id' => 'eid',
      'label' => 'event_name',
    ),
    'view modes' => array(
      'full' => array(
        'label' => t('Full content'),
        'custom settings' => FALSE,
      ),
    ),
    'static cache' => FALSE,
    'field cache' => FALSE,

    // Entity API properties.
    'entity class' => 'SiteshortnameEvent',
    'module' => 'siteshortname_entities_remote',
    'metadata controller class' => 'RemoteEntityAPIDefaultMetadataController',
    'views controller class' => 'EntityDefaultViewsController',

    // Remote Entity API properties.
    'remote base table' => 'siteshortname_entities_remote_events',
    'remote entity keys' => array(
      'remote id' => 'event_id',
      'label' => 'event_name',
    ),
    'expiry' => array(
      // Number of seconds before a locally cached instance must be refreshed
      // from the remote source.
      'expiry time' => 600,
      // A boolean indicating whether or not to delete expired local entities
      // on cron.
      'purge' => FALSE,
    ),
  );

  // Get the property map data.
  $remote_properties = siteshortname_entities_remote_get_remote_properties();

  // Assign each map to its corresponding entity.
  foreach ($entities as $key => $einfo) {
    $entities[$key]['property map'] =
      drupal_map_assoc(array_keys($remote_properties[$key]));
  }

  // Return all of the entity information.
  return $entities;
}
?>
Notes
  1. Just like the entity type node, which is subdivided into content types (generically referred to as bundles in Drupal-speak), we can subdivide remote entities into their own bundles. In this case, we could have a "High-school event" bundle and a "College event" bundle that vary slightly, but instances of both would still be members of the entity type Event. We won't be setting this up here though.
  2. In this article, we won't be covering remote saving (only remote loading), but it is possible through the remote API.
  3. Make sure to adjust the cache settings properly once development is complete.
  4. Detailed documentation on the APIs is available for the Core entity API, the Entity API, and the the Remote Entity API.
Access to remote properties

As we're not using the Field API to attach information to our entities, we need to do it with properties. The code below exposes the data we'll define shortly.

<?php
/**
 * Implements hook_entity_property_info_alter().
 *
 * This is needed to use wrappers to access the remote entity
 * data in the entity_data property of remote entities.
 *
 * @see: Page 107 of the Programming Drupal 7 Entities book.  The code below is
 *   a variation on it.
 * @todo: Remove whenever this gets added to the remote_entity module.
 */
function siteshortname_entities_remote_entity_property_info_alter(&$info) {

  // Set the entity types and get their properties.
  $entity_types = array(
    'siteshortname_entities_remote_event',
  );

  $remote_properties = siteshortname_entities_remote_get_remote_properties();

  // Assign the property data to each entity.
  foreach ($entity_types as $entity_type) {
    $properties = &$info[$entity_type]['properties'];
    $entity_data = &$properties['entity_data'];
    $pp = &$remote_properties[$entity_type];
    $entity_data['type'] = 'remote_entity_' . $entity_type;

    // Set the default getter callback for each property.
    foreach ($pp as $key => $pinfo) {
      $pp[$key]['getter callback'] = 'entity_property_verbatim_get';
    }

    // Assign the updated property info to the entity info.
    $entity_data['property info'] = $pp;
  }
}
?>
Remote property definition

This is where we define the field (or in this case property) information, the data attached to each entity, that we exposed above.

<?php
/**
 * Get remote property information for remote entities.
 *
 * @return
 *   An array of property information keyed by entity type.
 */
function siteshortname_entities_remote_get_remote_properties() {

  // Initialize a list of entity properties.
  $properties = array();

  // Define properties for the entity type.
  $properties['siteshortname_entities_remote_event'] = array(

    // Event information.
    'event_id' => array(
      'label' => 'Remote Event ID',
      'type' => 'integer',
      'description' => 'The remote attribute "id".',
      'views' => array(
        'filter' => 'siteshortname_entities_remote_views_handler_filter_event_id',
      ),
    ),
    'event_date' => array(
      'label' => 'Date',
      'type' => 'date',
      'description' => 'The remote attribute "date".',
      'views' => array(
        'filter' => 'siteshortname_entities_remote_views_handler_filter_event_date',
      ),
    ),
    'event_details' => array(
      'label' => 'Details',
      'type' => 'text',
      'description' => 'The remote attribute "details".',
    ),
  );

  // Return all of the defined property info.
  return $properties;
}
?>
Notes
  1. Try to remember the distinction between local and remote entity IDs. At the moment, we're only interested in remote properties so we don't don't need to worry about local IDs just yet.
  2. Don't worry too much about the Views filters. These are Views filter handler classes. They're only necessary if you'd like custom filters for the respective properties.
Entity instances as Web pages

This starts the Core Hooks section of the module file, Part II. In this section, we're providing each remote data instance as a Web page just like standard local content within Drupal via nodes.

The hook_menu() implementation responds to hits to the event/EVENT_ID path, loads the object, themes all of the data, and then returns it for display as a page. We're assuming all of your HTML output will be in a template in the includes/siteshortname_entities_remote.theme.inc file in your module's directory.

<?php
/****************************************************************************
 ** Drupal Core
 ****************************************************************************/

/**
 * Implements hook_menu().
 */
function siteshortname_entities_remote_menu() {
  $items = array();

  $items['event/%siteshortname_entities_remote_event'] = array(
    'title' => 'Remote Event',
    'page callback' => 'siteshortname_entities_remote_event_view',
    'page arguments' => array(1),
    'access arguments' => array('access content'),
  );

  return $items;
}

/**
 * Menu autoloader wildcard for path 'event/REMOTE_ID'.
 *
 * @see hook_menu() documentation.
 * @param $remote_id
 *   The remote ID of the record to load.
 * @return
 *   The loaded object, or FALSE on failure.
 */
function siteshortname_entities_remote_event_load($remote_id) {
  return remote_entity_load_by_remote_id('siteshortname_entities_remote_event', $remote_id);
}

/**
 * Page callback for path 'event/%remote_id'.
 *
 * @param $event
 *   The auto-loaded object.
 * @return
 *   The themed output for the event page.
 */
function siteshortname_entities_remote_event_view($event) {
  $fullname = $event->name;
  drupal_set_title($fullname);
  $event_output = theme('siteshortname_entities_remote_event', array(
    'event' => $event,
  ));
  return $event_output;
}

/**
 * Implements hook_theme().
 */
function siteshortname_entities_remote_theme() {
  return array(
    'siteshortname_entities_remote_event' => array(
      'variables' => array('event' => NULL),
      'file' => 'includes/siteshortname_entities_remote.theme.inc',
    ),
  );
}
?>

There's one more thing to do here. In our hook_entity_info() implementation, we stated the following:

<?php
    'entity class' => 'SiteshortnameEvent',
?>

We could have used Entity here instead of SiteshortnameEvent, but we want a custom class here so that we can override the URL path for these entities. So add the following class:

<?php
class SiteshortnameEvent extends Entity {
  /**
   * Override defaultUri().
   */
  protected function defaultUri() {
    return array('path' => 'event/' . $this->remote_id);
  }
}
?>
Web services integration We're now onto Part III, setting up Web-service endpoints and associating remote resources with entities. This is done through the implementation of a few Web Service Clients hooks. <?php
/****************************************************************************
 ** Web Service Clients
 ****************************************************************************/

/**
 * Implements hook_clients_connection_type_info().
 */
function siteshortname_entities_remote_clients_connection_type_info() {
  return array(
    'our_rest' => array(
      'label'  => t('REST Data Services'),
      'description' => t('Connects to our data service using REST endpoints.'),
      'tests' => array(
        'event_retrieve_raw' => 'SiteshortnameEntitiesRemoteConnectionTestEventRetrieveRaw',
      ),
      'interfaces' => array(
        'ClientsRemoteEntityInterface',
      ),
    ),
  );
}

/**
 * Implements hook_clients_default_connections().
 */
function siteshortname_entities_remote_clients_default_connections() {

  $connections['my_rest_connection'] = new clients_connection_our_rest(array(
    'endpoint' => 'https://data.example.com',
    'configuration' => array(
      'username' => '',
      'password' => '',
    ),
    'label' => 'Our REST Service',
    'type' => 'our_rest',
  ), 'clients_connection');

  return $connections;
}

/**
 * Implements hook_clients_default_resources().
 */
function siteshortname_entities_remote_clients_default_resources() {
  $resources['siteshortname_entities_remote_event'] = new clients_resource_remote_entity(array(
    'component' => 'siteshortname_entities_remote_event',
    'connection' => 'my_rest_connection',
    'label' => 'Resource for remote events',
    'type' => 'remote_entity',
  ), 'clients_resource');

  return $resources;
}
?>

In the first function, we're adding metadata for the connection. In the second one, we're setting the endpoint and its credentials. The third function is what ties our remote entity, defined earlier, with the remote resource. There's some information on this documentation page, but there's more in the README file.

Temporary local storage

We'll need to store the remote data in a local table as a non-authoritative cache. The frequency with which it gets refreshed is up to you, as described earlier in this article. We'll need one table per entity. The good news is that we don't need to worry about the details; this is handled by the Remote Entity API. It provides a function returning the default schema. If you want to do anything different here, you are welcome to define your own.

The argument provided in the call is used for the table description as "The base table for [whatever you provide]". This will go in your siteshortname_entities_remote.install file.

<?php
/**
 * Implementation of hook_schema().
 */
function siteshortname_entities_remote_schema() {
  $schema = array(
    'siteshortname_entities_remote_events' => remote_entity_schema_table('our remote event entity type'),
  );

  return $schema;
}
?>

If you don't actually want to save one or more of your remote entities locally (say because you have private data you'd rather not have stored on your publicly-accessible Web servers), you can alter this default behaviour by defining your own controller which overrides the save() method.

<?php
/**
 * Entity controller extending RemoteEntityAPIDefaultController
 *
 * For most of our cases the default controller is fine, but we can use
 * this one for entities we don't want stored locally.  Override the save
 * behaviour and do not keep a local cached copy.
 */
class SiteshortnameEntitiesRemoteNoLocalAPIController extends RemoteEntityAPIDefaultController {

  /**
   * Don't actually save anything.
   */
  public function save($entity, DatabaseTransaction $transaction = NULL) {
    $entity->eid = uniqid();
  }
}
?>
Implementing the remote connection class

Create a file for the connection class.

<?php
/**
 * @file
 * Contains the clients_connection_our_rest class.
 */

/**
 * Set up a client connection to our REST services.
 *
 *  @todo Make private functions private once development is done.
 */
class clients_connection_our_rest extends clients_connection_base
  implements ClientsConnectionAdminUIInterface, ClientsRemoteEntityInterface {

}
?>

We'll now divide the contents of said file into three (3) sections, ClientsRemoteEntityInterface implementations, clients_connection_base overrides and local methods.

ClientsRemoteEntityInterface implementations

As you can see below, we've got three (3) methods here.

  • remote_entity_load() will load a remote entity with the provided remote ID.
  • entity_property_type_map() is supposedly required to map remote properties to local ones, but it wasn't clear to me how this gets used.
  • getRemoteEntityQuery() returns a query object, either a "select", "insert" or "update" based on whichever one was requested.
<?php
  /**************************************************************************
   * ClientsRemoteEntityInterface implementations.
   **************************************************************************/

  /**
   * Load a remote entity.
   *
   * @param $entity_type
   *   The entity type to load.
   * @param $id
   *   The (remote) ID of the entity.
   *
   * @return
   *  An entity object.
   */
  function remote_entity_load($entity_type, $id) {
    $query = $this->getRemoteEntityQuery('select');
    $query->base($entity_type);
    $query->entityCondition('entity_id', $id);
    $result = $query->execute();

    // There's only one. Same pattern as entity_load_single().
    return reset($result);
  }

  /**
   * Provide a map of remote property types to Drupal types.
   *
   * Roughly analogous to _entity_metadata_convert_schema_type().
   *
   * @return
   *   An array whose keys are remote property types as used as types for fields
   *   in hook_remote_entity_query_table_info(), and whose values are types
   *   recognized by the Entity Metadata API (as listed in the documentation for
   *   hook_entity_property_info()).
   *   If a remote property type is not listed here, it will be mapped to 'text'
   *   by default.
   */
  function entity_property_type_map() {
    return array(
      'EntityCollection' => 'list<string>',
    );
  }

  /**
   * Get a new RemoteEntityQuery object appropriate for the connection.
   *
   * @param $query_type
   *  (optional) The type of the query. Defaults to 'select'.
   *
   * @return
   *  A remote query object of the type appropriate to the query type.
   */
  function getRemoteEntityQuery($query_type = 'select') {
    switch ($query_type) {
      case 'select':
        return new OurRestRemoteSelectQuery($this);
      case 'insert':
        return new OurRestRemoteInsertQuery($this);
      case 'update':
        return new OurRestRemoteUpdateQuery($this);
    }
  }
?>
Parent overrides

The only method we need to worry about here is callMethodArray(). Basically, it sets up the remote call.

<?php
  /**************************************************************************
   * clients_connection_base overrides
   **************************************************************************/

  /**
   * Call a remote method with an array of parameters.
   *
   * This is intended for internal use from callMethod() and
   * clients_connection_call().
   * If you need to call a method on given connection object, use callMethod
   * which has a nicer form.
   *
   * Subclasses do not necessarily have to override this method if their
   * connection type does not make sense with this.
   *
   * @param $method
   *  The name of the remote method to call.
   * @param $method_params
   *  An array of parameters to passed to the remote method.
   *
   * @return
   *  Whatever is returned from the remote site.
   *
   * @throws Exception on error from the remote site.
   *  It's up to subclasses to implement this, as the test for an error and
   *  the way to get information about it varies according to service type.
   */
  function callMethodArray($method, $method_params = array()) {

    switch ($method) {
      case 'makeRequest':

        // Set the parameters.
        $resource_path = $method_params[0];
        $http_method = $method_params[1];
        $data = isset($method_params[2]) ? $method_params[2] : array();

        // Make the request.
        $results = $this->makeRequest($resource_path, $http_method, $data);
        break;
    }

    return $results;
  }
?>
Local methods We're assuming REST here, but you can use any protocol.

We have a makeRequest() method, which actually performs the remote call, and handleRestError() which deals with any errors which are returned.

<?php
  /**************************************************************************
   * Local methods
   **************************************************************************/

  /**
   * Make a REST request.
   *
   * Originally from clients_connection_drupal_services_rest_7->makeRequest().
   * Examples:
   * Retrieve an event:
   *  makeRequest('event?eventId=ID', 'GET');
   * Update a node:
   *  makeRequest('node/NID', 'POST', $data);
   *
   * @param $resource_path
   *  The path of the resource. Eg, 'node', 'node/1', etc.
   * @param $http_method
   *  The HTTP method. One of 'GET', 'POST', 'PUT', 'DELETE'. For an explanation
   *  of how the HTTP method affects the resource request, see the Services
   *  documentation at http://drupal.org/node/783254.
   * @param $data = array()
   *  (Optional) An array of data to pass to the request.
   * @param boolean $data_as_headers
   *   Data will be sent in the headers if this is set to TRUE.
   *
   * @return
   *  The data from the request response.
   *
   *  @todo Update the first two test classes to not assume a SimpleXMLElement.
   */
  function makeRequest($resource_path, $http_method, $data = array(), $data_as_headers = FALSE) {

    // Tap into this function's cache if there is one.
    $request_cache_map = &drupal_static(__FUNCTION__);

    // Set the options.
    $options = array(
      'headers' => $this->getHeaders(),  // Define if you need it.
      'method'  => $http_method,
      'data'    => $data,
    );

    // If cached, we have already issued this request during this page request so
    // just use the cached value.
    $request_path = $this->endpoint . $context_path . '/' . $resource_path;

    // Either get the data from the cache or send a request for it.
    if (isset($request_cache_map[$request_path])) {
      // Use the cached copy.
      $response = $request_cache_map[$request_path];
    } else {
      // Not cached yet so fire off the request.
      $response = drupal_http_request($request_path, $options);

      // And then cache to avoid duplicate calls within the page request.
      $request_cache_map[$request_path] = $response;
    }

    // Handle any errors and then return the response.
    $this->handleRestError($request_path, $response);
    return $response;
  }

  /**
   * Common helper for reacting to an error from a REST call.
   *
   * Originally from clients_connection_drupal_services_rest_7->handleRestError().
   * Gets the error from the response, logs the error message,
   * and throws an exception, which should be caught by the module making use
   * of the Clients connection API.
   *
   * @param $response
   *  The REST response data, decoded.
   *
   * @throws Exception
   */
  function handleRestError($request, $response) {

    // Report and throw an error if we get anything unexpected.
    if (!in_array($response->code, array(200, 201, 202, 204, 404))) {

      // Report error to the logs.
      watchdog('clients', 'Error with REST request (@req). Error was code @code with error "@error" and message "@message".', array(
        '@req'      => $request,
        '@code'     => $response->code,
        '@error'    => $response->error,
        '@message'  => isset($response->status_message) ? $response->status_message : '(no message)',
      ), WATCHDOG_ERROR);

      // Throw an error with which callers must deal.
      throw new Exception(t("Clients connection error, got message '@message'.", array(
        '@message' => isset($response->status_message) ? $response->status_message : $response->error,
      )), $response->code);
    }
  }
?>
Implementing the remote query class

This is where the magic happens. We need a new class file, OurRestRemoteSelectQuery.class.php, that will assemble the select query and execute it based on any set conditions.

Class variables and constructor

First, let's define the class, its variables and its constructor. It's a subclass of the RemoteEntityQuery class. Most of the standard conditions would be added to the $conditions array, but conditions handled in a special way (say those dealing with metadata) can be set up as variables themselves. In the example below, the constructor sets the active user as it can affect which data is returned. You can, however, set whatever you need to initialize your subclass, or leave it out entirely.

<?php
/**
 * @file
 * Contains the OurRestRemoteSelectQuery class.
 */

/**
 * Select query for our remote data.
 *
 * @todo Make vars protected once no longer developing.
 */
class OurRestRemoteSelectQuery extends RemoteEntityQuery {

  /**
   * Determines whether the query is RetrieveMultiple or Retrieve.
   *
   * The query is Multiple by default, until an ID condition causes it to be
   * single.
   */
  public $retrieve_multiple = TRUE;

  /**
   * An array of conditions on the query. These are grouped by the table they
   * are on.
   */
  public $conditions = array();

  /**
   * The from date filter for event searches
   */
  public $from_date = NULL;

  /**
   * The to date filter for event searches
   */
  public $to_date = NULL;

  /**
   * The user id.
   */
  public $user_id = NULL;

  /**
   * Constructor to generically set up the user id condition if
   * there is a current user.
   *
   * @param $connection
   */
  function __construct($connection) {
    parent::__construct($connection);
    if (user_is_logged_in()) {
      global $user;
      $this->useridCondition($user->name);
    }
  }
}
?>
Setting conditions

We have three (3) methods which set conditions within the query. entityCondition() sets conditions affecting entities in general. (The only entity condition supported here is the entity ID.) propertyCondition() sets conditions related to properties specific to the type of data. For example, this could be a location filter for one or more events. Finally, we have useridCondition() which sets the query to act on behalf of a specific user. Here we simply record the current Drupal user.

<?php
  /**
   * Add a condition to the query.
   *
   * Originally based on the entityCondition() method in EntityFieldQuery, but
   * largely from USDARemoteSelectQuery (Programming Drupal 7 Entities) and
   * MSDynamicsSoapSelectQuery.
   *
   * @param $name
   *  The name of the entity property.
   */
  function entityCondition($name, $value, $operator = NULL) {

    // We only support the entity ID for now.
    if ($name == 'entity_id') {

      // Get the remote field name of the entity ID.
      $field = $this->entity_info['remote entity keys']['remote id'];

      // Set the remote ID field to the passed value.
      $this->conditions[$this->remote_base][] = array(
        'field' => $field,
        'value' => $value,
        'operator' => $operator,
      );

      // Record that we'll only be retrieving a single item.
      if (is_null($operator) || ($operator == '=')) {
        $this->retrieve_multiple = FALSE;
      }
    }
    else {

      // Report an invalid entity condition.
      $this->throwException(
        'OURRESTREMOTESELECTQUERY_INVALID_ENTITY_CONDITION',
        'The query object can only accept the \'entity_id\' condition.'
      );
    }
  }

  /**
   * Add a condition to the query, using local property keys.
   *
   * Based on MSDynamicsSoapSelectQuery::propertyCondition().
   *
   * @param $property_name
   *  A local property. Ie, a key in the $entity_info 'property map' array.
   */
  function propertyCondition($property_name, $value, $operator = NULL) {

    // Make sure the entity base has been set up.
    if (!isset($this->entity_info)) {
      $this->throwException(
        'OURRESTREMOTESELECTQUERY_ENTITY_BASE_NOT_SET',
        'The query object was not set with an entity type.'
      );
    }

    // Make sure that the provided property is valid.
    if (!isset($this->entity_info['property map'][$property_name])) {
      $this->throwException(
        'OURRESTREMOTESELECTQUERY_INVALID_PROPERY',
        'The query object cannot set a non-existent property.'
      );
    }

    // Adding a field condition (probably) automatically makes this a multiple.
    // TODO: figure this out for sure!
    $this->retrieve_multiple = TRUE;

    // Use the property map to determine the remote field name.
    $remote_field_name = $this->entity_info['property map'][$property_name];

    // Set the condition for use during execution.
    $this->conditions[$this->remote_base][] = array(
      'field' => $remote_field_name,
      'value' => $value,
      'operator' => $operator,
    );
  }

  /**
   * Add a user id condition to the query.
   *
   * @param $user_id
   *   The user to search for appointments.
   */
  function useridCondition($user_id) {
    $this->user_id = $user_id;
  }
?>
Executing the remote query

The execute() method marshals all of the conditions, passes the built request to the connection's makeRequest() that we saw earlier, calls parseEventResponse() (which we'll investigate below) and then returns the list of remote entities that can now be used by Drupal.

Feel free to ignore the authentication code if it's not required for your implementation. I left it in as an extended example of how this could be done.

<?php
  /**
   * Run the query and return a result.
   *
   * @return
   *  Remote entity objects as retrieved from the remote connection.
   */
  function execute() {

    // If there are any validation errors, don't perform a search.
    if (form_set_error()) {
      return array();
    }

    $querystring = array();

    $path = variable_get($this->base_entity_type . '_resource_name', '');

    // Iterate through all of the conditions and add them to the query.
    if (isset($this->conditions[$this->remote_base])) {
      foreach ($this->conditions[$this->remote_base] as $condition) {
        switch ($condition['field']) {
          case 'event_id':
            $querystring['eventId'] = $condition['value'];
            break;
          case 'login_id':
            $querystring['userId'] = $condition['value'];
            break;
        }
      }
    }

    // "From date" parameter.
    if (isset($this->from_date)) {
      $querystring['startDate'] = $this->from_date;
    }

    // "To date" parameter.
    if (isset($this->to_date)) {
      $querystring['endDate'] = $this->to_date;
    }

    // Add user id based filter if present.
    if (isset($this->user_id)) {
      $querystring['userId'] = $this->user_id;
    }

    // Assemble all of the query parameters.
    if (count($querystring)) {
      $path .= '?' . drupal_http_build_query($querystring);
    }

    // Make the request.
    try {
      $response = $this->connection->makeRequest($path, 'GET');
    } catch (Exception $e) {
      if ($e->getCode() == OUR_REST_LOGIN_REQUIRED_NO_SESSION) {
        drupal_set_message($e->getMessage());
        drupal_goto('user/login', array('query' => drupal_get_destination()));
      }
      elseif ($e->getCode() == OUR_REST_LOGIN_REQUIRED_TOKEN_EXPIRED) {

        // Logout
        global $user;
        module_invoke_all('user_logout', $user);
        session_destroy();

        // Redirect
        drupal_set_message($e->getMessage());
        drupal_goto('user/login', array('query' => drupal_get_destination()));
      }
    }

    switch($this->base_entity_type) {
      case 'siteshortname_entities_remote_event' :
        $entities = $this->parseEventResponse($response);
        break;
    }

    // Return the list of results.
    return $entities;
  }
?>
Unmarshalling the response data and returning it

Here, in the parseEventResponse method, we decode the response data (if there is any), and do any additional work required to get each entity's data into an object. They're all returned as a single list (array) of entity objects. If the response provides information on the format (XML, JSON, etc.), you can unmarshal the data differently based on what the server returned.

<?php
  /**
   * Helper for execute() which parses the JSON response for event entities.
   *
   * May also set the $total_record_count property on the query, if applicable.
   *
   * @param $response
   *  The JSON/XML/whatever response from the REST server.
   *
   * @return
   *  An list of entity objects, keyed numerically.
   *  An empty array is returned if the response contains no entities.
   *
   * @throws
   *  Exception if a fault is received when the REST call was made.
   */
  function parseEventResponse($response) {

    // Fetch the list of events.
    if ($response->code == 404) {
      // No data was returned so let's provide an empty list.
      $events = array();
    }
    else /* we have response data */ {

      // Convert the JSON (assuming that's what we're getting) into a PHP array.
      // Do any unmarshalling to convert the response data into a PHP array.
      $events = json_decode($response->data, TRUE);
    }

    // Initialize an empty list of entities for returning.
    $entities = array();

    // Iterate through each event.
    foreach ($events as $event) {
      $entities[] = (object) array(

        // Set event information.
        'event_id' => isset($event['id']) ? $event['id'] : NULL,
        'event_name' => isset($event['name']) ? $event['name'] : NULL,
        'event_date' => isset($event['date']) ? $event['date'] : NULL,
      );
    }

    // Return the newly-created list of entities.
    return $entities;
  }
?>
Error handling

We provide a helper method dealing with errors raised in other methods. It records the specific error message in the log and throws an exception based on the message and the code.

<?php
  /**
   * Throw an exception when there's a problem.
   *
   * @param string $code
   *   The error code.
   *
   * @param string $message
   *   A user-friendly message describing the problem.
   *
   * @throws Exception
   */
  function throwException($code, $message) {

    // Report error to the logs.
    watchdog('siteshortname_entities_remote', 'ERROR: OurRestRemoteSelectQuery: "@code", "@message".', array(
      '@code' => $code,
      '@message' => $message,
    ));

    // Throw an error with which callers must deal.
   throw new Exception(t("OurRestRemoteSelectQuery error, got message '@message'.", array(
      '@message' => $message,
    )), $code);
  }
?>

Everything we've covered so far gets our remote data into Drupal. Below, we'll expose it to Views.

Views support Basic set-up

At the beginning of this article, I stated that we required the EntityFieldQuery Views Backend module. This allows us to replace the default Views query back-end, a local SQL database, with one that supports querying entities fetchable through the Remote Entity API. Make sure to add it, efq_views, to your custom remote entity module as a dependency.

For the curious, the changes I made to EFQ Views Backend to add this support can be found in the issue Add support for remote entities.

I added official documentation for all of this to the Remote Entity API README (via Explain how to integrate remote querying through Views). As it may not be obvious, when creating a new view of your remote entities, make sure that the base entity is the EntityFieldQuery version, not simply the entity itself. When selecting the entity type on which to base the view, you should see each entity twice: the standard one (via the default query back-end) and the EFQ version.

As stated in the documentation, you need to a add a buildFromEFQ() method to your RemoteEntityQuery subclass (which we went over in the previous section). We'll review why this is necessary and give an example next.

Converting from an EntityFieldQuery

As EFQ Views only builds EntityFieldQuery objects, we need to convert that type of query to an instance of our RemoteEntityQuery subclass. If EFQ Views stumbles upon a remote query instead of a local one, it will run the execute() method on one of these objects instead.

So we need to tell our subclass how to generate an instance of itself when provided with an EntityFieldQuery object. The method below handles the conversion, which EFQ Views calls when necessary.

<?php
  /**
   * Build the query from an EntityFieldQuery object.
   *
   * To have our query work with Views using the EntityFieldQuery Views module,
   * which assumes EntityFieldQuery query objects, it's necessary to convert
   * from the EFQ so that we may execute this one instead.
   *
   * @param $efq
   *   The built-up EntityFieldQuery object.
   *
   * @return
   *   The current object.  Helpful for chaining methods.
   */
  function buildFromEFQ($efq) {

    // Copy all of the conditions.
    foreach ($efq->propertyConditions as $condition) {

      // Handle various conditions in different ways.
      switch ($condition['column']) {

        // Get the from date.
        case 'from_date' :
          $from_date = $condition['value'];
          // Convert the date to the correct format for the REST service
          $result = $from_date->format('Y/m/d');
          // The above format() can return FALSE in some cases, so add a check
          if ( $result ) {
            $this->from_date = $result;
          }
          break;

        // Get the to date.
        case 'to_date':
          $to_date = $condition['value'];
          // Convert the date to the correct format for the REST service
          $result = $to_date->format('Y/m/d');
          // The above format() can return FALSE in some cases, so add a check
          if ( $result ) {
            $this->to_date = $result;
          }
          break;

        // Get the user ID.
        case 'user_id':
          $this->user_id = $condition['value'];
          break;

        default:
          $this->conditions[$this->remote_base][] = array(
            'field' => $condition['column'],
            'value' => $condition['value'],
            'operator' => isset($condition['operator']) ? $condition['operator'] : NULL,
          );
          break;
      }
    }

    return $this;
  }
?>

That should be it! You'll now need to spend some time (if you haven't already) getting everything connected as above to fit your specific situation. If you can get these details sorted, you'll then be ready to go.

Alternatives

At the time of this writing, there appears to be only one alternative to the Remote Entity API (not including custom architectures). It's the Web Service Data suite. The main difference between the modules is that Web Service Data doesn't store a local cache of remote data; the data is always passed through directly.

If this more closely matches what you'd like to do, be aware that there is currently no EntityFieldQuery support:

Support for EntityFieldQuery (coming soon) will allow developers to make entity field queries with web service data.

This is very clearly stated on the main project page, but I wasn't able to find an issue in the queue tracking progress. So if you choose this method, you may have to add EFQ support yourself, or you may not be able to use Views with your remote entities.

References

This article, Integrating remote data into Drupal 7 and exposing it to Views, appeared first on the Colan Schwartz Consulting Services blog.

Categories: Elsewhere

DrupalCon News: Making website magic with the DrupalCon site building track

Mon, 16/02/2015 - 19:45

In honor of this year’s DrupalCon in Tinseltown, we invite you to indulge in a bit of Drupal movie magic.

Imagine the scene…

NARRATOR
You are about to enter another dimension, a dimension not only of configuration and security but of UI. A journey into a wondrous land of complex sites without custom development. Next stop, the Drupal Zone!

THE SCENE
Intl. Acme, Inc. Meeting Room - it is day

FADE IN

Categories: Elsewhere

Chromatic: Atomic Drupal Development: Building Pieces Before Pages

Mon, 16/02/2015 - 18:12

Many designers are praising the benefits of Atomic Design. Rather than designing pages, Atomic Design focuses on designing systems of individual, reusable components. Designers aren’t – or at least shouldn’t be – the only ones thinking this way. From content strategy to QA, the entire team must be on the same atomic page.

Development is one area of a project that stands to benefit the most from this change in thought. Organizing a codebase by individual components keeps developers out of each other’s hair, reducing the code and effort overlap that often occurs when building by page or section. It also makes the codebase much easier to understand and maintain. Developers will know where to find code and how to fix, alter, or extend it, regardless of the original author. After enforcing coding standards, only git’s history will know who wrote what. This all saves time and money.

Because there are many ways to do anything in Drupal, building every component with the same approach is crucial. In the Drupal world, this approach is known as “the Drupal way”.

Building a component the Drupal way

Individual blocks, panel panes, or other UI elements would be examples of a component in Drupal. They are placed into regions within layouts to build pages. Other pages may use the same component in the same or different regions. A given component may vary across pages, but the design and intended functionality are similar. A simple search form is a good example, but they can be much more complex.

Design deliverables often arrive as complete pages. If the designers haven’t already, identify the components that each page consists of. Break up the page’s layout into regions and those regions into components. Determine which components live on more than one page and if they vary between them. It also helps to identify different components that share design or functionality with others. It’s important to recognize early if they will be sharing code.

Before writing a line of code, determine where in the codebase the component will live. Organize custom modules by content types or sections and add relevant components to the same modules. A module exported with Features should be treated no differently than one created by hand; don’t be afraid to add custom code to them (please do). The end goal is to have all back-end and (most) front-end code for a given component living in the same module.

Warning: This article is about to move fast and cover more ground than it should. It will move from back-end to front-end. There are many wonderful resources about each topic covered below, so they will be linked to rather than recreated. This will instead provide a high level overview of how they fit together and will highlight the most important pieces.

Component containers and placement

The most common container for a custom component is a block, created with a series of hooks. Contributed modules like Context can help place them on the page. More complex projects may choose to build pages with the Panels module. For pages built with Panels, custom panel page plugins are a component’s container of choice.

The decision between blocks and Context, Panels, or another approach is important to make early in the project. It is also important to stick with the same approach for every component. This article will focus less on this decision and more on how to construct the markup within the container of choice.

View modes and entity_view()

If the component displays information from a node or another type of entity, render it with a view mode. View modes can render different information from the same entity in different ways. Among other benefits, this helps display content in similar ways among different components.

Create a view mode with hook_entity_info_alter() or with the Entity view modes contributed module. This module also provide template suggestions for each entity type in each view mode. Render an individual piece of information with a view mode inside of a component using entity_view() (you’ll need the Entity API module) or node_view(). Alter the entity’s information as needed using a preprocess function and adjust the markup in a template. Those pieces will be discussed later.

If a component lists more than one entity or node, build a view with the Views contributed module. It is best if the view renders content with view modes using the Format options. Create Views components with the Block (or Content pane for Panels) display(s). Views also provides template suggestions to further customize the markup of the component. The exported view should live in the same module as the code that customizes it. EntityFieldQuery might be worth considering as an alternative to using Views.

hook_theme() and render arrays

If the component does not display information from an entity, such as a UI element, build it with hook_theme(). Drupal core and contributed modules use hook_theme() to build elements like links and item lists. This allows other modules to override and alter the information used to render the element. Default theme functions and templates can also be overridden to alter their markup.

Choose a name for the element that will identify it throughout the codebase. Outline what information the element will need to build the desired output. Use these decisions to define it using hook_theme(). Again, keep this hook in the same custom module as the rest of the code for the component.

To render a hook_theme() implementation, construct a render array. This array should contain the name of the implementation to render and any data it needs as input. Build and return this array to render the element as markup. The theme() function is a common alternative to render arrays, but it has been deprecated in Drupal 8. There are advantages to using render arrays instead, as explained in Render Arrays in Drupal 7.

Custom templates

Drupal renders all markup through templates and theme functions. Use templates to construct markup instead of theme functions. Doing so makes it easier for front-end developers to build and alter the markup they need.

Templates place variables provided by entity_view(), render arrays, and preprocess functions into the markup. They should live in the “templates” directory of the same module as the rest of the component’s code. The name of a template will come from theme hook suggestions. Underscores get replaced with dashes. Tell hook_theme() about the template for each element it defines.

There should be no logic in the template and they should not have to dig deep into Drupal’s objects or arrays. They should only use an if statement to determine if a variable has a value before printing its markup and value. They can also use a foreach to loop through an array of data. Further manipulation or function calls should happen in a preprocess function.

Preprocess functions

Use preprocess functions to extract and manipulate data such as field values and prepare them for the template. They are the middleman between the input and the output.

Preprocess functions follow the naming convention of hook_theme() implementations. Common base themes often use Drupal core’s preprocess functions, such as hook_preprocess_node(), in their template.php file. Keeping all preprocess functions in one file will create a mess in no time. Instead, place preprocess functions in the modules that define the parts their working with. This might be the custom feature that contains the exported content type.

jQuery/JavaScript files

Create a separate JavaScript file for each component that needs custom JavaScript. Place it in a “js” directory within the module and name the file after the component. Be sure to use the Drupal behavior system and name the behavior after the module and component.

Add the JavaScript file to each page the component will appear on. If the component appears on most pages, it might be best to just add it to every page. This will cause less HTTP requests with JavaScript aggregation enabled. The best way to do so is with hook_page_build(). JavaScript files can also be attached to entities rendered through view modes within hook_entity_view(). The best way to add JavaScript to a hook_theme() implementation is by attaching it to the render array.

Sass components

When using a CSS preprocessor like Sass, there isn’t much of a penalty to dividing the CSS into many files. Create a new Sass partial for each component and give the file the same name as the component. Keep them in a “components” directory within the Sass folder structure. Unlike all other code mentioned in this article, it is often best to keep all CSS for these components within the theme. Only keep CSS that supports the core behavior of the component in the module. Consider what styles should persist if it were a contributed module used with other themes.

In the component’s template, base the class names off of the component’s name as well. This makes it easy to find the component’s Sass after inspecting the element in the source. Follow the popular BEM / SMACSS / OOCSS methodologies from there.

Coming up for air

As mentioned, there are often endless ways to complete the same task in Drupal. This makes learning best practices difficult and “the Drupal way” will vary in the minds of different experts. The best way to grasp what works best is to start building something with other people and learn from mistakes. The approach outlined in this article aligns with common practice, but mileage will vary per project.

Regardless of approach, focusing on components before pages will only become more important. Drupal content is already displayed on everything from watches to car dashboards. The web is not made of pages anymore. Designers have begun to embrace this and Drupal developers should too; everyone will benefit!

Categories: Elsewhere

Makak Media: Taking BackDrop For A Test Drive

Mon, 16/02/2015 - 17:41

So the first BackDrop release is out there in the wild ready for a quick test drive! We're excited to see where this fork of Drupal 7 leads as we believe it to be a good complementary system to Drupal with a long term future.

First off we checked under the hood to get things configured and found the settings.php file in the root folder, which makes for easier access. Also all those txt files have been removed including the CHANGELOG.txt file, which we remove by default, as it supplies useful info to any hacker out there!

Naturally the installation process is very similar to Drupal but with a few less settings giving it a simpler feel.

Upon installation you're presented with a responsive admin menu with a slightly different structure to the standard Drupal menu. Responsiveness out of the box is great and the new menu again has a simpler look.

read more

Categories: Elsewhere

DrupalDare: G-WAN as a static Drupal file server

Mon, 16/02/2015 - 17:26
So now that we have concluded that it's easy to setup distribution of files on a separate subdomain, what about using a completely other web server (or in this case an application server)? Will it blend?
Categories: Elsewhere

Acquia: Development based on Drupal's Fundamental Particles - Brad Czerniak

Mon, 16/02/2015 - 13:34
Language Undefined

Presenter Brad Czerniak caught my eye with a blog post entitled "10 things I learned using Drupal at a hackathon," based on his experiences taking part in the #hackDPL (Detroit Public Library) competitive hackathon. In our podcast interview we talk about that – before moving on to Brad's session about the Drupal development best practices he and his team use at Commercial Progression in Michigan.

Categories: Elsewhere

Annertech: Enlightening - The Dark Art of Solr Search with Drupal

Mon, 16/02/2015 - 12:41
Enlightening - The Dark Art of Solr Search with Drupal Why this blog post?

Often when I add a search function to a Drupal website using Apache Solr, I'm amazed at how complex some people think this is. Many developers/site builders are of the belief that this is some kind of very-hard-to-master black art. They could not be more wrong.

So what I want to contribute back to the Drupal community is an understanding of how Solr works, why/how it differs from Drupal Core Search module, and the benefits Solr has over core search.

Categories: Elsewhere

lakshminp.com: The Drupal 8 plugin system - part 2

Mon, 16/02/2015 - 11:38

We saw in part 1 how plugins help us in writing reusable functionality in Drupal 8. There are a lot of concepts which plugins share in common with services, like:

  1. limited scope. Do one thing and do it right.
  2. PHP classes which are swappable.

Which begs the question, how exactly are plugins different from services?
If your interface expects implementations to yield the same behaviour, then go for services. Otherwise, you should write it as a plugin. This needs some explaining.
For instance, if you are creating an interface to store data in a persistent system, like MySQL or MongoDB, then it would be implemented as a service. The save() function in your interface interface will be implemented differently for both the services, but the behaviour will be the same, i.e., it takes data as input parameters, stores them in the respective data store and returns a success message.

On the other hand, if you are creating an image effect, it needs to be a plugin. (It already is. Check image effects as plugins). The core concept of image plugins is to take in an image, apply an effect on it and return the modified image. Different image effects yield different behaviours. An image scaling effect might not produce the same behaviour as that of an image rotating effect. Hence, each of these effects need to be implemented as a plugin. If any module wants to create a new image effect, it needs to write a new plugin by extending the ImageEffectBase class.

Plugins used in core

Let's take a look at the major plugin types provided by Drupal 8 core. An example plugin of each plugin types will be the subjects of future blog posts.

  1. Blocks
    Drupal 8 finally got blocks right. Custom blocks can be created from the BlockBase class.

  2. Field Types, Field Widgets and Field Formatters
    Check part 1 for how this is done in Drupal 8.

  3. Actions
    Drupal 8 allows module developers to perform custom actions by implementing the ActionBase class. Blocking a user, unpublishing a comment, making a node sticky etc. are examples of actions.

  4. Image Effects
    Image effects are plugins which manipulate an image. You can create new image effects by extending ImageEffectBase. Examples of core image effects are CropImageEffect and ScaleImageEffect.

  5. Input filters
    User submitted input is passed through a series of filters before it is persisted in the database or output in HTML. These filters are implemented as plugins by implementing the FilterBase class.

  6. Entity Types
    In Drupal parlance, entities are objects that persist content or configuration in the database. Each entity is an instance of an entity type. New entity types can be defined using the annotation discovery mechanism.

  7. Views related plugins
    A large collection of different plugin types are employed by views during the querying, building and rendering stages.

Plugin Discovery

Plugin discovery is the process by which Drupal finds plugins written in your module. Drupal 8 has the following plugin discovery mechanisms:

  1. Annotation based. Plugin classes are annotated and have a directory structure which follows the PSR-4 notation.

  2. Hooks. Plugin modules need to implement a hook to tell the manager about their plugins.

  3. YAML files. Plugins are listed in YAML files. Drupal Core uses this method for discovering local tasks and local actions.

  4. Static. Plugin classes are registered within the plugin manager class itself. This is useful if other modules should not create new plugins of this type.

Annotation based discovery is the most popular plugin discovery method in use. We will briefly look at how we create a new plugin type using this method in the next part.

Categories: Elsewhere

DrupalDare: CDN, Cookieless Requests and Subdomains

Mon, 16/02/2015 - 10:52
In this text I will go in to the topic of using a separate domain for serving your static files to avoid the client sending unnecessary cookies in the headers and why it may be or may not be a solution to speed up your website.
Categories: Elsewhere

Drupal core announcements: Drupal core security release window on Wednesday, February 18

Mon, 16/02/2015 - 04:37
Start:  2015-02-18 (All day) America/New_York Online meeting (eg. IRC meeting) Organizers:  David_Rothstein

The monthly security release window for Drupal 6 and Drupal 7 core will take place on Wednesday, February 18.

This does not mean that a Drupal core security release will necessarily take place on that date for either the Drupal 6 or Drupal 7 branches, only that you should prepare to look out for one (and be ready to update your Drupal sites in the event that the Drupal security team decides to make a release).

There will be no bug fix release on this date; the next window for a Drupal core bug fix release is Wednesday, March 4.

For more information on Drupal core release windows, see the documentation on release timing and security releases, and the discussion that led to this policy being implemented.

Categories: Elsewhere

Drupalpress, Drupal in the Health Sciences Library at UVA: two new drupal distros – one for voting, one for 3d printing e-commerce

Sun, 15/02/2015 - 21:29

Two new drupal distributions available on github

** https://github.com/alibama/cvillecouncilus is the distribution behind https://www.cvillecouncil.us - it’s an attempt to run a political campaign through a virtual proxy…

** https://github.com/alibama/rapid-prototyping-ecommerce-drupal – this is the code behind http://rpl.mae.virginia.edu/ it’s an e-commerce solution for 3d printing… A lot of this is implemented in rules and other well-standardized code thanks to Joe Pontani - a talented developer here in Virginia.  Joe integrated several third party tools, and set up the UVa payment gateway through Nelnet.

Both sites are getting updates over the next few months – the Charlottesville Council website also has a drupalgap implementation on it – absolutely awesome toolset…

18F API compliance is another feature I’m pretty stoked about… I got most of that done with the oauth2 server, views datasource, services and a couple of great notification features done with rules + views  i’ll get that feature out asap = it’s really convenient – matching a profile2 taxonomy field onto content taxonomy fields for notifications with new content.

any questions – please drop a line in the comments below

Categories: Elsewhere

DrupalOnWindows: Bypassing Form Validations and Required Fields in Drupal: the BFV module.

Sun, 15/02/2015 - 07:00
Language English

Required or not required? To validate or not to validate? That is the question. So you've setup (the site builder's way, no custom forms) your required fields and custom validations for Node types, just to get this feedback from the customer:

That field we defined as mm..... as required (something trivial and not really critical such as an image file) is actually not always required. Users X and Y should be able to bypass that restriction.

More articles...
Categories: Elsewhere

Drupal @ Penn State: A window into our Community

Sat, 14/02/2015 - 17:22
Intro

Something that inspired me recently to write about DUG, are the efforts of MediaCurrent. Media Current has recently been pushing forward a series of postings talking about how they are giving back and being a lot more open about use of time to give back (which is awesome).

Categories: Elsewhere

Angie Byron: Webchick's "plain Drupal English" Guide to the Remaining Drupal 8 Critical Issues: DrupalCon Bogotá Edition

Sat, 14/02/2015 - 10:09

(Apologies for the atrocious state of the HTML that follows; this content is originally from this Google Doc.)

Webchick's "plain Drupal English" Guide to the Remaining Drupal 8 Critical Issues: DrupalCon Bogotá Edition

DrupalCon Bogotá just finished up, and critical issue-wise we've managed to stay in the 50s for a few days (down from a high of 150 last summer!), so now seems like as good a time as any to write down what's left to ship Drupal 8!

This post will attempt to document all of the remaining 55 criticals (as of this writing), and attempt to offer a somewhat "plain English" (or at least "Drupal English" ;)) description of each, loosely categorized into larger areas in which we could really use extra help. There are over 2,600 contributors to Drupal 8 at this time, please join us!

(Note: These descriptions might not be 100% accurate; this is my best approximation based on the issue summary and last few comments of each issue. If I got the description of your pet issue wrong, please update your issue summary. ;))

Table of contents

Quick vocabulary lesson

Current state of critical issues

Security

Security Parity with Drupal 7

Session and User Authentication API

REST

New security improvements

Performance

Profiling

Fix regressions relative to Drupal 7

Entity Field API

Views

Configuration system

"Fix it, or else"

General house-keeping

Other

Thrilling conclusion! (also known as "TL;DR")

Quick vocabulary lesson

Within this list, there are numerous "markers" used to signify that some of the issues in this list are more important to fix ASAP. These are:

  • D8 upgrade path: An issue tagged D8 upgrade path (currently, 13) means it blocks a beta-to-beta upgrade path for Drupal 8, generally because they materially impact the data schema or they impact security. Once we resolve all of these blockers, early adopters will no longer need to reinstall Drupal between beta releases, but can just run the update.php script as normal. This is currently our biggest priority.
  • Blocker: An issue tagged blocker (currently, 5) means it blocks other issues from being worked on. This is currently our second-biggest priority (or 0th priority in the case an issue blocks a D8 upgrade path issue :D). I've noted these as "sub-bullets" of the issues that are blocking them.
  • Postponed: Issues that are marked postponed (currently, 9) are either currently blocked by one of the "Blocker" issues, or we've deliberately chosen to leave off until later.
  • >30 days: These patches have a patch more than 30 days old, and/or were last meaningfully commented on >30 days ago. If you're looking for a place to start, re-rolling these is always helpful!
  • No patch: This issue doesn't have a patch yet. Oh the humanity! Want to give it a shot?

Other weird core issue nomenclature:

  • "meta" means a discussion/planning issue, with the actual patch action happening in related/child issues.
  • "PP-3" means "this issue is postponed on 3 other issues" (PP-1 means 1 other issue; you get the drift).
Current state of critical issues

Sections roughly organized from "scariest" to "least scary" in terms of how likely they are to make Drupal 8 take a longer time to come out.

Security

Because Drupal 8 hasn't shipped yet, it's not following Drupal's standard Security Advisory policy, so there are still outstanding, public security issues (13 as of this writing). We need to resolve most of these prior to providing a Drupal 8 beta-to-beta upgrade path, as this is the time when we signal to early adopters that it's an OK time to start cautiously building real sites on Drupal 8.

Skills needed: Various

Security Parity with Drupal 7

This class of security issue is to ensure that when Drupal 8 ships, it won't have any regressions security-wise relative to Drupal 7.

  • Port SA-CONTRIB-2013-096 to D8 (D8 upgrade path) Here's one such issue for Entity Reference module. SA-CONTRIB-2013-096 addressed a relatively esoteric remote access bypass bug, and the patch needs to be forward-ported to Drupal 8.
  • Port SA-CONTRIB-2015-039 to D8 (D8 upgrade path)  SA-CONTRIB-2015-039 addressed two issues in Views module, a redirect and default permissions for disabled views. The first was fixed in D8, but access checks are still missing from a few views for the second.

Session and User Authentication API

Because of various intricate dependencies, the authentication part of Drupal 8 isn't yet converted to object-oriented code, and prevents us from further optimizing bootstrap. This set of issues fixes various problems with this part of the code, and ensures these important security APIs are complete and ready to ship.

REST
  • REST user updates bypass tightened user account change validation (D8 upgrade path) Since Drupal 7, when you edit your user account, you have to provide the existing password when you want to change the password or e-mail. This security feature is currently by-passed by REST user updates as you can change the password or e-mail without providing the password.
  • External caches mix up response formats on URLs where content negotiation is in use (>30 days) Drupal 8's request processing system is currently based on content negotiation (which allows you to serve multiple versions of a document at the same URI based on what headers are sent e.g. Accept: text/html or Accept: application/json). This is generally considered the "right way" to do REST. However, various external caches and CDNs have trouble with this mechanism, and can mix them up and can send random formats back. The issue proposes changing from content negotiation to separate, distinct paths such as /node/1.json.

New security improvements

These issues affect new security improvements we want to make over and above what Drupal 7 does.

  • [meta] Document or remove every SafeMarkup::set() call One of the big security improvements in Drupal 8 is the introduction of Twig's autoescape feature, which ensures that all output to the browser is escaped by default. However, this is quite a big change that requires all of the code that was previously escaping content to stop doing that, else it gets double-escaped (so you start seeing &lt; and &quot; and whatnot in the UI). We originally introduced the ability to manually mark markup safe with SafeMarkup::set(), but the recommended approach is actually to use Twig everywhere, so this issue is to ensure that all remaining instances of the manual way are fixed, or at least documented to explain why they're using the non-recommended method.
  • Passing in #markup to drupal_render is problematic (>30 days) Another issue in the Twig autoescape space, we need to ensure that markup set by the "#markup" in e.g. form definitions is properly escaped.
  • Limit PDO MySQL to executing single statements if PHP supports it Remember SA-CORE-2014-005? Yeah, so do we. ;) This issue is to make sure that if another SQL injection vulnerability is ever found again, the damage it can do is more limited by eliminating the ability for MySQL to execute multiple queries per PDO statement.

Performance

Tied with security, 13 of the remaining issues are tagged Performance. While it may seem odd/scary to have this be a big chunk of the work left, it's a common practice to avoid premature optimization, and instead focus on optimization once all of the foundations are in place.

Skills needed: Profiling, caching, optimization, render API

Profiling

Here are a sub-set of issues where we need performance profiling to determine what gives us the biggest bang for our effort.

Fix regressions relative to Drupal 7
  • [meta] Resolve known performance regressions in Drupal 8 This is the main tracking issue in this space. During the 8.x cycle we've introduced several known performance regressions compared to Drupal 7 (sometimes to make progress on features/functionality, other times because we introduced changes that we hoped would buy us better scalability down the line), which we need to resolve before release so that Drupal 8 isn't slower than Drupal 7. The performance team meets weekly and tracks their progress in a detailed spreadsheet.
Entity Field API

Tracked under the Entity Field API tag (currently 6 issues).

Skills needed: Entity/Field API, Form API, Schema API

  • Schema for newly defined entity types is never created (D8 upgrade path) When you first install a module that defines an entity type (for example, Comment), its database tables are correctly generated. However, if an entity definition is later added by a developer to an already-installed module, the related database schema won't get created, nor will it be detected in update.php as an out-of-date update to run.
  • FileFormatterBase should extend EntityReferenceFormatterBase (D8 upgrade path) Entity Reference fields define a EntityReferenceFormatterBase class, which contains logic about which entities to display in the lookup, including non-existing entities and autocreated entities. File field's FileFormatterBase class currently duplicates that logic, except it misses some parts, including access checking, which makes this a security issue. The issue proposes to simply make File field's base class a sub-class of Entity Reference's, removing the need of "sort of but not quite the same" code around key infrastructure.
  • FieldTypePluginManager cannot instantiate FieldType plugins, good thing TypedDataManager can instantiate just about anything Currently, you get a fatal error if you attempt to use Drupal 8's Plugin API to create a new instance of a field type. The current code in core is avoiding this problem by going roundabout via the Typed Data API instead. This issue's critical because these are two of the most central APIs in Drupal 8, and they should work as expected.
  • [META] Untie content entity validation from form validation Despite all the work to modernize Drupal 8 into a first-class REST server, there still remain places where validation is within form validation functions, rather as part of the proper entity validation API, which means REST requests (or other types of workflows that bypass form submissions) are missing validation routines. This meta issue tracks progress of moving the logic to its proper place.
  • Entity forms skip validation of fields that are edited without widgets (>30 days) If a field can be edited with a form element that is not a Field API widget, we do not validate its value at the field-level (i.e., check it against the field's constraints). Fixing this issue requires ensuring that all entity forms only use widgets for editing field values.
  • Entity forms skip validation of fields that are not in the EntityFormDisplay (No patch, >30 days) Drupal 8 has a new feature called "form modes" (basically analogous to "view modes" in Drupal 7, except allowing you to set up multiple forms for a given entity instead). Currently, we're only validating fields that are displayed on a given form mode, even though those fields might have validation constraints on other fields that are not displayed. Critical because it could present a security issue.
Views

Views issues are generally tracked with the VDC tag. There are currently 6 criticals at this point which touch on Views (some already covered in earlier sections).

Configuration system

The configuration system is remarkably close to being shippable! Only 4 critical issues left. We're now working on finalizing the niggly bits around edge cases that involve configuration that depends on other configuration.

Skills needed: Configuration system, Entity Field API, Views

"Fix it, or else"

This subset of issues are things that are part of core currently, and we would really like to keep, but are willing to make some hard choices in the event they are among the last remaining criticals blocking release. The "postponed" among this list means "postponed until we're down to only a handful of criticals left." If these issues end up remaining in the list, we will move their functionality to contrib, and hope to add it back to core in a later point release if it gets fixed up.

Skills required: Various, but mainly low-level infrastructure and non-MySQL database skills.

  • [meta] Drupal.org (websites/infra) blockers to a Drupal 8 release (Blocker) This issue contains a "grab bag" of Drupal.org blockers that prevent an optimal Drupal 8 release, including things like semantic versioning support, testing support for multiple PHP/database versions, and support for Composer-based installations. If this issue is one of the last remaining criticals, we might choose to ship Drupal 8 anyway, and jettison one or more features in the process, such as…
  • [Meta] Make Drupal 8 work with PostgreSQL The meta/planning issue for fixing PostgreSQL (both in terms of functionality and in terms of failing tests). bzrudi71 is predominantly leading the charge here and making steady progress, but more hands would be greatly appreciated.
  • [meta] Database tests fail on SQLite (>30 days) Same deal as PostgreSQL but for SQLite. Unlike PostgreSQL though, this one doesn't have anyone leading the charge at this time, and it's also a lot harder to punt this to contrib, since we use it for various things such as testbot. Help wanted!

General house-keeping

These are all basic things we need to keep on top of between now and release, to ensure that when we're down to only a handful of criticals, we're ready to ship a release candidate. The good news is, these are also all generally really easy patches to make, and often also to test.

Skills needed: Basic patch rolling / reviewing / testing skills. (good for newbies!)

  • [meta] Ship minified versions of external JavaScript libraries (Postponed) Basically, in the Gilded Mobile Age™ we want to ensure that we're sending as little over the wire as possible, so scrunching various JS libraries down to the smallest possible file size needs to be the default. Separate issue from above because it needs to happen for both updated and existing JS libraries. Postponed because there'll be less work to do once all of the out-of-date JS libraries are updated and minified at the same time.
Other

I couldn't figure out a nice heading for these, so here's the rest.

  • Remove _system_path from $request->attributes Symfony provides a $request object, which has an "attributes" property for the purpose of storing various contextual bits. But the problem with $request->attributes->get('_MAGIC_KEY') is that the values are undocumented, there's no IDE autocompletion, and it's not clear which are internal vs. public properties, so we have an issue at [meta] Stop using $request->attributes->get(MAGIC_KEY) as a public API. to try and stop doing that.

    However, _system_path in particular is used a ton, since it's very common to want to know the path of the current request. The patch exposes a "CurrentPath" service instead, which eliminates all of those issues.
  • Potential data loss: concurrent node edits leak through preview Because the temp store that Drupal 8's new node preview system employs uses an entity's ID as the key, rather than something uniquely identifiable to a user, if two users are editing the same node and hit preview at the same time, one of them is going to lose data due to a race condition.
  • Ajax file uploads fail on IE 9 Pretty much exactly what it says on the tin. :P
Thrilling conclusion! (also known as "TL;DR")

Well, not so thrilling, but at least a conclusion. :)

  • Anywhere you see a blocker issue, attack it with fire. Those are holding other criticals up.
  • The biggest area of focus right now is D8 upgrade path blockers. Many of them are security issues.
  • Another big area is Performance, both fixing existing regressions, and profiling to determine where our biggest wins are.
  • Views and Entity Field API are tied in third place for number of remaining criticals. Let's have a race, shall we? ;)
  • The configuration system is looking pretty good, but still has a handful of sticky issues left.
  • There are a series of important features we'll lose if they're not fixed up in time.
  • If you're looking for something somewhat easy/mundane, help yourself to one of the general house-keeping issues.
  • Don't forget about the other miscellaneous issues I was too tired to categorize.

Sorry this post was so long (and probably has its share of inaccuracies) but I hope it will be helpful to some. It's basically what I needed to get back up to speed after taking a few months off of Drupal 8, so figured I'd document my way to understanding.

Now, let's get 'er done! :D

Tags: drupal 8drupaldrupal core diaries
Categories: Elsewhere

3C Web Services: Introduction to the Super Login Module for Drupal 7

Sat, 14/02/2015 - 00:13
Drupal’s default login page form is functional but does leave a lot to be desired. It’s pretty bland and, if left as-is, is always a telltale sign that your site is a Drupal website. The Super Login Module for Drupal 7 is a simple way to improve the look and functionality of Drupal's login page.
Categories: Elsewhere

Stanford Web Services Blog: Behat Custom Step Definition: Wait for Batch API to Finish

Fri, 13/02/2015 - 23:00

If you're using Behat and the Drupal Extension, you might find the following code snippet helpful if you want to add a step to wait for batch jobs to finish.

If one of your Behat scenarios kicks off a batch job (e.g., a Feeds import), and you want to wait for that batch job to finish before moving on to the next step, add this step definition in your FeatureContext.php file:

Categories: Elsewhere

DrupalDare: Nginx, Memcache, Drupal page cache #1

Fri, 13/02/2015 - 20:09
Reverse proxy caching is something that is almost a must have for any popular site today. For Drupalers Varnish is by far the most used reverse proxy since it's easy to use and works really well/stable. Nginx has been succesful as a reverse proxy with Drupal as well, but it has mainly been been used with the file system and modules like Boost. But there exists a Memcache module that can speak directly to Nginx as well. In this article I will do some benchmarking on how to save the data when applying memcache to Nginx.
Categories: Elsewhere

Commerce Guys: Drupal Commerce Site Spotlight: Novusbio.com

Fri, 13/02/2015 - 19:06

We're always on the lookout for great sites built with Drupal Commerce, our truly flexible software that's changing the face of eCommerce one site at a time.

Perhaps the biggest strength of Drupal Commerce is it's flexibility, and that's clearly at work on the Novus Bio web site, a niche eCommerce site that's servicing a unique need in BioTech. Novus Biologicals features a commerce suite with a multitude of products available internationally for buyers of many different languages. Not to mention they are selling "cells", How cool is that?

 

To see Drupal Commerce sites we've Spotlighted in previous weeks view the Other Spotlight Sites

Categories: Elsewhere

Pages