Elsewhere

Drupal Commerce: Upgrade paths between Drupal 8 module versions

Planet Drupal - ven, 22/04/2016 - 17:11

Over time, modules update their shipped configuration such as rules, views, node types. Users expect to get these updates when they update the modules. For example, Drupal Commerce 1.x provides over eleven default views and rules. From release to release, these Views and Rules were enhanced with new features or bug fixes reported by the community.

Take the order management view. In a future release of Drupal Commerce, we may add a newly exposed filter to the view. In Drupal 7 the view would automatically be updated unless it was overridden. That is not the case in Drupal 8.

Drupal 8 introduces the Configuration Management system. While robust, it changes how we work with the configuration in comparison to Drupal 7. Instead of a module owning configuration, configuration is now owned by the site. When a module is installed, its configuration is imported. On module update, no configuration updates are performed. Instead, the module must write an update hook to perform one of the following updates:

Catégories: Elsewhere

OpenLucius: 11 Cool Drupal modules for site builders | April 2016

Planet Drupal - ven, 22/04/2016 - 09:59

Let’s start straight away with the things that struck me about Drupal module updates of last month:

1. Image effects

The Drupal 8 and Drupal 7 core both have features to facilitate graphics, including scaling and cropping.

To perform other actions, Drupal 7 had additional modules: ImageCache Actions and Textimage:

  • Placing overlays, for example for round corners
  • Adding a watermark, for example your logo
  • Placing text over image, for example your company name
  • Making images lighter/darker

In Drupal 8 these additional image features are now available in this module.

Download

Catégories: Elsewhere

ActiveLAMP: Creating Layouts with the Layout Plugin Module in Drupal 8

Planet Drupal - ven, 22/04/2016 - 05:01

Writing custom layouts using the Layout Plugin module for D8 is really easy. This video will outline how to create a new layout in your theme using Foundation 6 as the base theme and how to extend the layout to add custom classes and id.

Read more...
Catégories: Elsewhere

Matthew Garrett: Circumventing Ubuntu Snap confinement

Planet Debian - ven, 22/04/2016 - 03:51
Ubuntu 16.04 was released today, with one of the highlights being the new Snap package format. Snaps are intended to make it easier to distribute applications for Ubuntu - they include their dependencies rather than relying on the archive, they can be updated on a schedule that's separate from the distribution itself and they're confined by a strong security policy that makes it impossible for an app to steal your data.

At least, that's what Canonical assert. It's true in a sense - if you're using Snap packages on Mir (ie, Ubuntu mobile) then there's a genuine improvement in security. But if you're using X11 (ie, Ubuntu desktop) it's horribly, awfully misleading. Any Snap package you install is completely capable of copying all your private data to wherever it wants with very little difficulty.

The problem here is the X11 windowing system. X has no real concept of different levels of application trust. Any application can register to receive keystrokes from any other application. Any application can inject fake key events into the input stream. An application that is otherwise confined by strong security policies can simply type into another window. An application that has no access to any of your private data can wait until your session is idle, open an unconfined terminal and then use curl to send your data to a remote site. As long as Ubuntu desktop still uses X11, the Snap format provides you with very little meaningful security. Mir and Wayland both fix this, which is why Wayland is a prerequisite for the sandboxed xdg-app design.

I've produced a quick proof of concept of this. Grab XEvilTeddy from git, install Snapcraft (it's in 16.04), snapcraft snap, sudo snap install xevilteddy*.snap, /snap/bin/xevilteddy.xteddy . An adorable teddy bear! How cute. Now open Firefox and start typing, then check back in your terminal window. Oh no! All my secrets. Open another terminal window and give it focus. Oh no! An injected command that could instead have been a curl session that uploaded your private SSH keys to somewhere that's not going to respect your privacy.

The Snap format provides a lot of underlying technology that is a great step towards being able to protect systems against untrustworthy third-party applications, and once Ubuntu shifts to using Mir by default it'll be much better than the status quo. But right now the protections it provides are easily circumvented, and it's disingenuous to claim that it currently gives desktop users any real security.

comments
Catégories: Elsewhere

Lullabot: “One Weird Trick” for Drupal Security... or Something

Planet Drupal - jeu, 21/04/2016 - 22:00
Matt & Mike talk with Drupal Security Team Lead Michael Hess, along with the former lead Greg Knaddison. We talk about the current state of Drupal security, along with their Drupalcon session, "Watch the Hacker Hack".
Catégories: Elsewhere

Palantir: What DrupalCon Means to Us

Planet Drupal - jeu, 21/04/2016 - 20:48

Each and every year we pack up our booth, swag, and people, and make the pilgrimage to a location somewhere in the US for DrupalCon North America. It's always an amazing week, filled with knowledge, sharing, fun, and, of course, business. This year it's in New Orleans the week of May 9th, and, per usual, we have a lot going on so we'd like to share a few highlights.

Sessions

We have a few sessions lead by our team this year, and we'd love to see you there! Better still, stop by our booth (no. 222) before or after each to talk more. We're an open book, and want to share.

D8 Module Acceleration Program (Workbench)

Ken Rickard
Ken is part of a D8 Module Acceleration Program panel for Workbench on Tuesday, May 10th at 2:15 pm in room no. 279.

Finding Your Purpose as a Drupal Agency

George DeMet
Founder and CEO George DeMet will be giving a talk on Finding Your Purpose as a Drupal Agency on Wednesday, May 11th at 4:45pm in room no. 262.

Booth

We'll be at booth no. 222 all week, and we'd like you to think of it as your basecamp. As you walk around the conference and attend sessions, if a question comes to mind or you need some clarification on some aspect of Drupal, web strategy, design, or development, please do come by and let's talk. And we mean it; we have couches...

Sponsorship

We always gladly sponsor both the Drupal Association and DrupalCon. It's good for the community, and it's good for our clients. Why? Because of the nature of the open source community, and the sheer amount of knowledge share to solve common problems as it relates to the web. We utilize and pass on this knowledge to keep the cycle going.

Trivia Night

Speaking of sponsorship, we've sponsored Trivia Night at DrupalCon for the last few years, and it's always a fantastic night of fun and Drupal nerdiness! We're proud sponsors again this year, so we hope you can join us for a night filled with trivia, hilarity, and plenty of prizes.

Thursday, May 12
9:00pm - 12:00am (doors at 8:00pm)
U.S. Freedom Pavilion at National World War II Museum
1043 Magazine Street, New Orleans, LA
Free to attend

Related content Everything you Wanted to Know About DrupalCon

DrupalCon is just a few weeks away in New Orleans, so our Account Manager Allison Manley is joined by our CEO and Founder George DeMet, Drupal veteran and PHP guru Larry "Crell" Garfield, and Senior Front-End Developer Lauren Byrwa on our podcast this time around. They share thoughts about the conference generally, what they're excited about specifically, and what they're expected from the Driesnote, among other topics.

Planning for Long Term Success on the Web

Designing, architecting, and building enterprise-level Web projects can feel like a Moonshot. They're often expensive, complex, and time-consuming undertakings that require the long-term commitment and dedication of the entire company or organization. In this way, many of the lessons of the Apollo program are directly applicable to the work that we undertake with our customers every day at Palantir.

Project Management: The Musical!

Description

Attending the conference? We'd love to say hello. Let's schedule a time to meet at DrupalCon.

Catégories: Elsewhere

Mediacurrent: What it really means to run a project the “Agile Scrum” way

Planet Drupal - jeu, 21/04/2016 - 20:36

 If you were to search “What is the Agile Scrum Methodology of project management?” you’d find this:

“…an alternative to traditional project management, typically used in software development. It helps teams respond to unpredictability through incremental, iterative work cadences, known as sprints. Scrum is one framework used to run a project using the Agile Methodology.”

Catégories: Elsewhere

Chapter Three: More Drupal 8 Sites in the Wild

Planet Drupal - jeu, 21/04/2016 - 20:11

Its been a big week for Drupal 8 here at Chapter Three.

As mentioned previously, at the beginning of the year Chapter Three made the decision to build all new projects with Drupal 8. Knowing we had the help and Drupal 8 expertise of Alex and Daniel to back us up if we encountered any errors or incomplete contrib ports gave us the confidence to leave Drupal 7 in the dust. Having already had a few successful client projects on 8 last year let me know I wouldn't have a developer revolt on my hands as a result of that decision.

The fruits of that decision are starting to ripen.

Catégories: Elsewhere

Metal Toad: What I Learned Today: Check Your Default Google Analytics Settings

Planet Drupal - jeu, 21/04/2016 - 19:48
What I Learned Today: Check Your Default Google Analytics Settings April 21st, 2016 Jonathan Jordan Google Analytics Module Settings

Drupal's Google Analytics Module is great. There are a few settings though that I recently found out you'll want to pay closer attention to. First is the "Pages" section of the configuration form, which allows you to only include/exclude Google Analytics tracking code on certain pages. The default settings are to exclude the code on the following pages.

admin admin/* batch node/add* node/*/* user/*/*

The other setting to keep in mind is in the "Privacy" section of the configuration form. In particular the "Universal web tracking opt-out" setting which is enabled by default. Those are pretty good defaults for most sites, but if you aren't aware of them, they can also cause you to lose some valuable analytics data.

The "Universal web tracking opt-out" setting allows users to setup their browsers to send a Do Not Track header. The Google Analytics module respects this header if the "Universal web tracking opt-out" setting is enabled. That is unless page caching is enabled. If the page is cached, then Google Analytics is always included in the cached page. So on production sites, this usually means that tracking is enabled except if a user is logged in and they setup their browser to send the Do Not Track header. Again, this seems like the reasonable and respectful default.

Introduce Custom Panel Pages Into the Mix

Now consider you have a Custom Panel page with the url: "node/%/my-sub-page", which is a special sub page for a particular content type on your site. The contents of the panel page doesn't matter much, what matters is that if you are using the default Google Analytics settings for googleanalytics_pages, this page will not have Google Analytics tracking on it. The fix is fairly simple, update your Google Analytics settings and replace "node/*/*" with node/*/edit, and maybe a few others that you might not care about (ie: node/*/devel, node/*/nodequeue if you use those modules).

This post is part of my challenge to never stop learning.
Catégories: Elsewhere

Phponwebsites: Redirect users after login in Drupal 7

Planet Drupal - jeu, 21/04/2016 - 18:42
    This blog describes about how to redirect users after logged into a site in Drupal 7. By default, Drupal redirects users to user page after logged into a site.  Suppose you want to redirect users into any other pages as you want. Then you can done that in Drupal 7.
  You can redirect users after login in Drupal using the following two ways:
1. Redirect users after logged into a site using hook_user_login()
2. Redirect users after logged into a site using custom form submit



Redirect users after logged into a site using hook_user_login:
     Drupal provides hook called hook_user_login to make changes while user login successfully. Let see the below code.

/**
 * Implement hook_user_login()
 */
function phponwebsites_user_login(&$form, &$form_state) {
 //add page here to where you want redirect users after login
  $form['redirect'] = '<front>';
}
    Now you can check whether you redirect to front page or not after login. Now  Drupal will be redirect you to front page.

Redirect users after logged into a site using custom form submit:
   Drupal have alternate method to redirect users after login. Ie, You need to add custom form submit handler to a form using hook_form_alter(). Then add a page to redirect users in that custom form submit handler in Drupal 7. Let see the below code.


/**
 * Implement hook_form_alter().
 */
function phponwebsites_form_alter(&$form, &$form_state, $form_id) {
  if ($form_id == "user_login" || $form_id == "user_login_block") {
    $form['#submit'][] = 'phponwebsites_custom_login_submit';
  }
}  
function phponwebsites_custom_login_submit(&$form, &$form_state) {
  //page to be redirect
  $form['redirect'] = '<front>';
}

Now you will be redirect to front page after logged into a drupal site. Now I’ve hope you should know how to redirect users after logged into a site in Drupal 7.

Related articles:
Add new menu item into already created menu in Drupal 7
Add class into menu item in Drupal 7
Create menu tab programmatically in Drupal 7
Add custom fields to search api index in Drupal 7
Clear views cache when insert, update and delete a node in Drupal 7
Create a page without header and footer in Drupal 7
Login using both email and username in Drupal 7
Catégories: Elsewhere

Pronovix: Documenting APIs mini-conference: Video recordings available!

Planet Drupal - jeu, 21/04/2016 - 17:41

The video recordings of the lectures held on the Documenting APIs mini-conference are now available on our dedicated event page.

On March 4, 2016 we helped organise a special whole-day meetup in London on the Write The Docs conference, in which the community discussed the tools and processes used to document APIs.

Catégories: Elsewhere

John Goerzen: Count me as a systemd convert

Planet Debian - jeu, 21/04/2016 - 15:45

Back in 2014, I wrote about some negative first impressions of systemd. I also had a plea to debian-project to end all the flaming, pointing out that “jessie will still boot”, noting that my preference was for sysvinit but things are what they are and it wasn’t that big of a deal.

Although I still have serious misgivings about the systemd upstream’s attitude, I’ve got to say I find the system rather refreshing and useful in practice.

Here’s an example. I was debugging the boot on a server recently. It mounts a bunch of NFS filesystems and runs a third-party daemon that is started from an old-style /etc/init.d script.

We had a situation where the NFS filesystems the daemon required didn’t mount on boot. The daemon then was started, and unfortunately it basically does a mkdir -p on startup. So it started running and processing requests with negative results.

So there were two questions: why did the NFS filesystems fail to start, and how could we make sure the daemon wouldn’t start without them mounted? For the first, journalctl -xb was immensely helpful. It logged the status of each individual mount, and it turned out that it looked like a modprobe or kernel race condition when a bunch of NFS mounts were kicked off in parallel and all tried to load the nfsv4 module at the same time. That was easy enough to work around by adding nfsv4 to /etc/modules. Now for the other question: refusing to start the daemon if the filesystems weren’t there.

With systemd, this was actually trivial. I created /etc/systemd/system/mydaemon.service.requires (I’ll call the service “mydaemon” here), and in it I created a symlink to /lib/systemd/system/remote-fs.target. Then systemctl daemon-reload, and boom, done. systemctl list-dependencies mydaemon will even show the the dependency tree, color-coded status of each item on it, and will actually show every single filesystem that remote-fs requires and the status of it in one command. Super handy.

In a non-systemd environment, I’d probably be modifying the init script and doing a bunch of manual scripting to check the filesystems. Here, one symlink and one command did it, and I get tools to inspect the status of the mydaemon prerequisites for free.

I’ve got to say, as someone that has occasionally had to troubleshoot boot ordering and update-rc.d symlink hell, troubleshooting this stuff in systemd is considerably easier and the toolset is more powerful. Yes, it has its set of poorly-documented complexity, but then so did sysvinit.

I never thought the “world is falling” folks were right, but by now I can be counted among those that feels like systemd has matured to the point where it truly is superior to sysvinit. Yes, in 2014 it had some bugs, but by here in 2016 it looks pretty darn good and I feel like Debian’s decision has been validated through my actual experience with it.

Catégories: Elsewhere

Drop Guard: Speaking Words of Wisdom, Let it Key - a guest post by Luke Probasco

Planet Drupal - jeu, 21/04/2016 - 14:00
Speaking Words of Wisdom, Let it Key - a guest post by Luke Probasco Gast (not verified) Thu, 21.04.2016 - 14:00

Encryption has gone mainstream. Thanks to the numerous data breaches (781 during 2015 in the U.S. alone) data security is a top priority for businesses of all sizes. Semi-vague language like “we ensure our data is protected” from IT teams is no longer good enough to satisfy the concerns of business executives and their customers. CEOs are losing their jobs and companies are suffering financial losses/fines that reach into the millions of dollars when poorly encrypted or un-encrypted data is lost.

Drupal Encryption Security Drupal Planet
Catégories: Elsewhere

Liip: State of Drupal Commerce for Drupal 8

Planet Drupal - jeu, 21/04/2016 - 13:12

The two biggest players in the Drupal 7 webshop field are Drupal Commerce (also known as DC1) and Übercart. DC1 actually started as an Übercart rewrite to make use of Drupal 7 APIs. After the split Übercart was ported to Drupal 7 too but it was still using Drupal 6 technologies.

Although still very much in development, it seems something similar will be true for Drupal 8 as well. The developers of DC2 (the Drupal 8 version of Drupal Commerce), lead by Bojan Živanović rewrote the whole system from scratch to make use of the huge changes in Drupal 8. They are active members of the Drupal developer community so they not only know but also form the actual best practices. While working on DC2 they have fixed many dozens of Drupal 8 core issues and much more in other contributed modules (such as Entity, Inline Entity Form, Profile).

A great realisation when rewriting Commerce was that several components of a webshop could be reused by other (not even necessarily webshop or Drupal) systems. Some typical examples are address formats, currencies and taxes. These components are usually a huge pain to maintain because of the small differences from country to country. So they have created standalone PHP libraries usually using authorative third party datasets such as CLDR for currency or Google’s dataset for address formats. Some of them are already used by other webshop solutions like Foxycart and developers even outside the Drupal community are giving feedback which makes maintaining them easier.

In the DC2 development process UI and UX has got a big emphasis already from the beginning. Based on research of existing webshop solutions the shop administration and checkout process has been redesigned by UX specialists. For example, the product creation process is quite confusing in DC1 and there’s not even a recommended way to do it. In DC2 this happens now in one single form which makes it super easy.

A new concept in DC2 is Stores. Stores represent billing locations and products can belong to one ore more stores. One use case is the need for different billing for customers from different countries. Another one is having a shop where sellers can open an account and sell their own products. In this case each seller has their own store.

There are many other new features and improvements like a new and flexible tax system (you can say things like: “from Jan 1st 2014 the VAT changes from 21% to 19%”), a redesigned checkout flow, different workflows for different order types etc.

DC2 is still in alpha phase and is not recommended for production use yet. Beta releases will already have upgrade paths between them and so can be considered for starting real sites with. Beta1 is expected for May.

Drupal Commerce is the most popular e-commerce solution for Drupal 7. Given the high quality code and responsiveness to developer, shop maintainer and customer needs I do not expect this to change in Drupal 8 either.

Sources:
Drupal Commerce 2 blog
Modules Unraveled podcast on Commerce

Catégories: Elsewhere

Valuebound: How to create Custom Rest Resources for POST methods in Drupal 8

Planet Drupal - jeu, 21/04/2016 - 11:07

One of the biggest changes in Drupal 8 is the integration of Rest services in the core. With use of Views it become very easy to create RESTful Services.

But there are certain situations when you need to create your own custom REST Resources. There are documentations available for creating a GET request using Views and using custom module. However there isn’t much documentation available for creating Rest Resources for POST methods.

In this article I will be sharing, how to create a REST Resource for POST methods. This Rest API will create an article in Drupal 8 site from an external…

Catégories: Elsewhere

Alessio Treglia: Corporate Culture in the Transformative Enterprise

Planet Debian - jeu, 21/04/2016 - 10:40

 

The “accelerated” world of the Western or “Westernized” countries seems to be fed by an insidious food, which generates a kind of psychological dependence: anxiety. The economy of global markets cannot help it, it has a structural need of it to feed their iron logic of survival. The anxiety generated in the masses of consumers and in market competitors is crucial for Companies fighting each other and now they can only live if men are projected to objective targets continuously moving forward, without ever allowing them to achieve a stable destination.

The consumer is thus constantly maintained in a state of perpetual breathlessness, always looking for the fresh air of liberation that could eventually reduce his tension. It is a state of anxiety caused by false needs generated by advertising campaigns whose primary purpose is to create a need, to interpret to their advantage a still confused psychological demand leading to the destination decided by the market…

<Read More…[by Fabio Marzocca]>

Catégories: Elsewhere

Sven Decabooter: Using Drupal 8 contact forms via REST

Planet Drupal - jeu, 21/04/2016 - 10:34
Using Drupal 8 contact forms via REST

While working on Narwal CMS, our hosted decoupled / headless CMS service built on Drupal 8, a feature request that has popped up multiple times is the ability to have (contact) forms in the frontend application or website be processed by the Drupal backend through REST.

Drupal 8 provides you with REST services and fieldable contact forms out of the box, but doesn't provide all the pieces to get this specific use case working. In the tutorial below I will show you how to implement this on your Drupal 8 website.

Tutorial:
  1. Prerequisites
    • Install your Drupal 8 website and make sure to enable the "Contact" and "Restful web services" modules, as we'll obviously need those.
    • Install and enable the REST UI module to easily manage which REST resources are activated on your site.
    • Install and enable the "Contact message REST" module.
    • You should also activate an authentication module such as "HTTP Basic Authentication" (in Drupal 8 core) or IP Consumer Auth to be able to restrict form submission access to your frontend application(s) only, and not leave your REST endpoints open to everyone.
  2. Set up REST resource
    • Go to /admin/config/services/rest and enable the "Contact message" resource, with path /contact_message/{entity}:
    • Configure the supported formats and authentication providers to your preferences. E.g.:
  3. Configure permissions
    • Go to /admin/people/permissions and grant selected roles access to the permission "Access POST on Contact message resource". This should be set up correctly so your frontend application can authenticate with the correct role and perform the POST method. Only grant anonymous users permissions if you have some method to restrict access to your frontend application, such as IP whitelisting or API keys.
  4. Create contact form
    • Go to /admin/structure/contact and create a contact form, or use the default "Website feedback" form. Take note of the machine name of the contact form, as we'll need it when performing the REST call.
  5. Submit contact form via REST
    • In your frontend application, you should do a POST method call to the /contact_message REST endpoint: e.g. POST http://mydomain.com/contact_message?_format=json. Make sure you have set up your headers correctly for authentication and the X-CSRF-Token.
    • The body of your method call should provide values for the contact form entity fields. When using the default contact form fields, your request would look like this (in JSON format): { "contact_form":[{"target_id":"YOUR_FORM_MACHINE_NAME"}], "name":[{"value":"SENDER_NAME_VALUE"}], "mail":[{"value":"SENDER_MAIL_VALUE"}], "subject":[{"value":"FORM_SUBJECT"}], "message":[{"value":"FORM_MESSAGE"}] }
    • For example:
Extra's:

You could also install the Contact Storage module if you wish to store the submitted form entries, rather then just sending them via mail. In my experience, the module currently still has some bugs that prevent it from being completely usable, but those might get ironed out soon.

Background about "Contact message REST" module

While trying to get the functionality above working with Drupal core, 2 issues came up that prevented it from working:

  • When trying to create a Contact message through the default /entity/contact_message/{contact_message} resource, errors are thrown, because a contact Message entity doesn't get an ID assigned to it (because it uses the ContentEntityNullStorage storage handler). Since this REST resource tries to load the created entity and return it, this fails.
  • The actual sending of the contact form mails happens in the \Drupal\contact\MessageForm submit handler, which doesn't get called when a Message entity gets created through a REST resource.

Hence I created the Contact message REST module to solve these issues. Hopefully they will be resolved in later versions of Drupal 8.x, so the module won't be needed anymore.

 

Sven Decabooter Thu, 04/21/2016 - 10:34
Catégories: Elsewhere

Mario Lang: Scraping the web with Python and XQuery

Planet Debian - jeu, 21/04/2016 - 10:30

During a JAWS for Windows training, I was introduced to the Research It feature of that screen reader. Research It is a quick way to utilize web scraping to make working with complex web pages easier. It is about extracting specific information from a website that does not offer an API. For instance, look up a word in an online dictionary, or quickly check the status of a delivery. Strictly speaking, this feature does not belong in a screen reader, but it is a very helpful tool to have at your fingertips.

Research It uses XQuery (actually, XQilla) to do all the heavy lifting. This also means that the Research It Rulesets are theoretically also useable on other platforms. I was immediately hooked, because I always had a love for XPath. Looking at XQuery code is totally self-explanatory for me. I just like the syntax and semantics.

So I immediately checked out XQilla on Debian, and found #821329 and #821330, which were promptly fixed by Tommi Vainikainen, thanks to him for the really quick response!

Unfortunately, making xqilla:parse-html available and upgrading to the latest upstream version is not enough to use XQilla on Linux with the typical webpages out there. Xerces-C++, which is what XQilla uses to fetch web resources, does not support HTTPS URLs at the moment. I filed #821380 to ask for HTTPS support in Xerces-C to be enabled by default.

And even with HTTPS support enabled in Xerces-C, the xqilla:parse-html function (which is based on HTML Tidy) fails for a lot of real-world webpages I tried. Manually upgrading the six year old version of HTML Tidy in Debian to the latest from GitHub (tidy-html5, #810951) did not help a lot either.

Python to the rescue

XQuery is still a very nice language for extracting information from markup documents. XQilla just has a bit of a hard time dealing with the typical HTML documents out there. After all, it was designed to deal with well-formed XML documents.

So I decided to build myself a little wrapper around XQilla which fetches the web resources with the Python Requests package, and cleans the HTML document with BeautifulSoup (which uses lxml to do HTML parsing). The output of BeautifulSoup can apparently be passed to XQilla as the context document. This is a fairly crazy hack, but it works quite reliably so far.

Here is how one of my web scraping rules looks like:

from click import argument, group @group() def xq(): """Web scraping for command-line users.""" pass @xq.group('github.com') def github(): """Quick access to github.com.""" pass @github.command('code_search') @argument('language') @argument('query') def github_code_search(language, query): """Search for source code.""" scrape(get='https://github.com/search', params={'l': language, 'q': query, 'type': 'code'})

The function scrape automatically determines the XQuery filename according to the callers function name. Here is how github_code_search.xq looks like:

declare function local:source-lines($table as node()*) as xs:string* { for $tr in $table/tr return normalize-space(data($tr)) }; let $results := html//div[@id="code_search_results"]/div[@class="code-list"] for $div in $results/div let $repo := data($div/p/a[1]) let $file := data($div/p/a[2]) let $link := resolve-uri(data($div/p/a[2]/@href)) return (concat($repo, ": ", $file), $link, local:source-lines($div//table), "---------------------------------------------------------------")

That is all I need to implement a custom web scraping rule. A few lines of Python to specify how and where to fetch the website from. And a XQuery file that specifies how to mangle the document content.

And thanks to the Python click package, the various entry points of my web scraping script can easily be called from the command-line.

Here is a sample invokation:

fx:~/xq% ./xq.py github.com Usage: xq.py github.com [OPTIONS] COMMAND [ARGS]... Quick access to github.com. Options: --help Show this message and exit. Commands: code_search Search for source code. fx:~/xq% ./xq.py github.com code_search Pascal '"debian/rules"' prof7bit/LazPackager: frmlazpackageroptionsdeb.pas https://github.com/prof7bit/LazPackager/blob/cc3e35e9bae0c5a582b0b301dcbb38047fba2ad9/frmlazpackageroptionsdeb.pas 230 procedure TFDebianOptions.BtnPreviewRulesClick(Sender: TObject); 231 begin 232 ShowPreview('debian/rules', EdRules.Text); 233 end; 234 235 procedure TFDebianOptions.BtnPreviewChangelogClick(Sender: TObject); --------------------------------------------------------------- prof7bit/LazPackager: lazpackagerdebian.pas https://github.com/prof7bit/LazPackager/blob/cc3e35e9bae0c5a582b0b301dcbb38047fba2ad9/lazpackagerdebian.pas 205 + 'mv ../rules debian/' + LF 206 + 'chmod +x debian/rules' + LF 207 + 'mv ../changelog debian/' + LF 208 + 'mv ../copyright debian/' + LF ---------------------------------------------------------------

For the impatient, here is the implementation of scrape:

from bs4 import BeautifulSoup from bs4.element import Doctype, ResultSet from inspect import currentframe from itertools import chain from os import path from os.path import abspath, dirname from subprocess import PIPE, run from tempfile import NamedTemporaryFile import requests def scrape(get=None, post=None, find_all=None, xquery_name=None, xquery_vars={}, **kwargs): """Execute a XQuery file. When either get or post is specified, fetch the resource and run it through BeautifulSoup, passing it as context to the XQuery. If find_all is given, wrap the result of executing find_all on the BeautifulSoup in an artificial HTML body. If xquery_name is not specified, the callers function name is used. xquery_name combined with extension ".xq" is searched in the directory where this Python script resides and executed with XQilla. kwargs are passed to get or post calls. Typical extra keywords would be: params -- To pass extra parameters to the URL. data -- For HTTP POST. """ response = None url = None context = None if get is not None: response = requests.get(get, **kwargs) elif post is not None: response = requests.post(post, **kwargs) if response is not None: response.raise_for_status() context = BeautifulSoup(response.text, 'lxml') dtd = next(context.descendants) if type(dtd) is Doctype: dtd.extract() if find_all is not None: context = context.find_all(find_all) url = response.url if xquery_name is None: xquery_name = currentframe().f_back.f_code.co_name cmd = ['xqilla'] if context is not None: if type(context) is BeautifulSoup: soup = context context = NamedTemporaryFile(mode='w') print(soup, file=context) cmd.extend(['-i', context.name]) elif isinstance(context, list) or isinstance(context, ResultSet): tags = context context = NamedTemporaryFile(mode='w') print('<html><body>', file=context) for item in tags: print(item, file=context) print('</body></html>', file=context) context.flush() cmd.extend(['-i', context.name]) cmd.extend(chain.from_iterable(['-v', k, v] for k, v in xquery_vars.items())) if url is not None: cmd.extend(['-b', url]) cmd.append(abspath(path.join(dirname(__file__), xquery_name + ".xq"))) output = run(cmd, stdout=PIPE).stdout.decode('utf-8') if type(context) is NamedTemporaryFile: context.close() print(output, end='')

The full source for xq can be found on GitHub. The project is just two days old, so I have only implemented three scraping rules as of now. However, adding new rules has been made deliberately easy, so that I can just write up a few lines of code whenever I find something on the web which I'd like to scrape on the command-line. If you find this "framework" useful, make sure to share your insights with me. And if you impelement your own scraping rules for a public service, consider sharing that as well.

If you have an comments or questions, send me mail. Oh, and by the way, I am now also on Twitter as @blindbird23.

Catégories: Elsewhere

Vardot: Project Manager’s Guide to Breaking Down a Drupal Site for Incremental Delivery

Planet Drupal - jeu, 21/04/2016 - 09:39
How to, Resources Read time: 8 minutes

TL;DR. Jump to the free template: Standard Drupal Work Breakdown Structure Template.

Building a new site on a content management system has always been a tricky project to manage for a project manager, when compared to building a site on a framework or from scratch. That is because you are dealing with building blocks that are provided as a standard from the CMS. A project manager should have the necessary knowledge of the CMS’s building blocks to be able to manage a successful project.

Put this in context of today’s Scrum management approach (an agile way to manage a project, usually software development) and you’ll end up with a puzzled project manager with several questions such as:

  1. What can my team deliver in the first sprint?

  2. How can I breakdown the project’s deliveries into sprints?

  3. What expectations of deliverables should I set with my project’s stakeholders (product owner, business owner, client)

  4. When do I deliver the homepage for my stakeholder to look at?

  5. Are we supposed to deliver on page by page basis?

 

Drupal disrupts the “page” methodology that we are used to thinking of. One naturally tends to think of a website as a folder, with sub-folders, and pages (.html) inside those folders. That’s the 90s. We’re in 2016. Drupal is a database-driven CMS that takes a content-first (or content-out) approach of building rich web experiences, instead of a page-first approach. See “Drupal is content first, and that's good” and “A Richer Canvas”.

   

Due to this approach of how Drupal works, we at Vardot have came up with a framework of planning the phases of building a Drupal site, to lead to an incremental development that can be broken down and fit into Scrum sprints. This will apply to almost all Drupal projects.

This standard way we'll call: The Standard Drupal Work Breakdown Structure

 

Why Do I Need a Breakdown of Work for Planning My Drupal Site?

Because this what project managers do. I have seen in the many Drupal projects that I was part of, that project managers (and/or coordinators) must understand how Drupal works, how the development process goes, and how do we get 80% of the site done in 20% of the time.

A work breakdown structure will help you (as a project manager) to understand how a Drupal site is built. It will also ease the process for getting high quality incremental deliveries to fit in your sprints. In this post, I will walk you through the high-level breakdown for any Drupal site.

 

Most importantly, the goals and outcomes of a breakdown are for you to understand and communicate to your project’s stakeholders your timeline of deliveries, and to be able to fit these deliveries into sprints.

To summarize, these goals are:

  1. Breakdown of deliverables. Define needed outcomes of initial sprints

  2. Provide a holistic view and analysis of the site’s functionality and its building blocks

  3. Remember, we are building a CMS, not a website. Therefore you need to architect your “CMS solution”, and not your “website solution”

 

Let’s Start With The How

Now we enforce these goals by implementing the The Standard Drupal Work Breakdown Structure, that will fit for almost all of the Drupal projects you will work with.

These phases will be divided into:

  1. Initialization Work Breakdown Structure: This phase is the cornerstone phase for starting right, it’s most probably a typical standard way that you should do in every project.

  2. Project’s Epics Work Breakdown Structure: Careful analysis of the site’s components and how it will be developed in the CMS will be implemented here.

  3. Finalization Work Breakdown Structure: This is the ending phase, where you make sure your site is ready for launch. Final preparations, tuning, and tweaks are carried out in this phase to prepare for your big day.

Note that you will be able to deliver something for your stakeholders to look at, in the “Initialization” phase.

This breakdown must happen after high-fidelity wireframes are done, or if you have the full visual mockups of a Drupal site done for your key pages.

It’s important to note that the visual mockups should use and adhere to Drupal’s design language and patterns. But what is Drupal’s design language and patterns? That’s for another article to discuss.

Now that we have designs handed to us with a clear communication of how the new website will look like. We are ready to breakdown our Drupal site for a successful delivery.

 

The Work Breakdown

Disclaimer: the terminology that I’m using below to name some components that make up your site is not an “official Drupal language”. No worries if you stick with the same terminology or use your own names, what really matters is just the breakdown structure.

So I’m categorizing what makes a (Drupal) site into six components:

  1. Wrapping components: Header and Footer.
    These are the components that provide your site with a wrapper for all your next components. Start with these as soon as you install Drupal; it will help you get through the easy stuff that makes up your site.
     
  2. Global components: Page title, Breadcrumbs, Tabs (a.k.a menu local tasks), System messages ...etc.
    These are the components that make up the uniformity of a CMS. These are your next target.
     
  3. Site-unified components: Ad blocks, Newsletter subscribe block, Social media feeds or “follow us” blocks, Static “about us” block ..etc.
    These are the components that most likely appear in the same style across multiple pages in the site.
     
  4. Full nodes and entities: Your “Full content” node/user/entity pages.
    Getting back to “content-out” approach, always start with the full-node or entities completion.
     
  5. Views, view modes, and other content: Views of recent content, Featured content, Node pages, Feeds integration, CRM integration, Single Sign On integration, ...etc.
    This is the major work; components that define your site.
     
  6. The annoying 20% of the site: This is where the built 80% of your site gets the final hidden work, iterative tweaking and enhancements to your site, whether it is requested by your QA team, the client, or the product owner.

In light of this breakdown of CMS’s categories, here’s an animated illustration of how a site can be made possible when following the flow of development based on the components above:

In this order, you can now think of a Drupal site to be developed according the following steps:

Initialization Work Breakdown Structure

  1. Delivering “1. Wrapping components

    1. Install Drupal (or the distribution you want to use), setup the development environment ..etc.

    2. Populate the things that make up the “Wrapping components”: Menus, logo, search ..etc.

    3. Create your theme, and theme the “Wrapping components”

  2. Delivering “2. Global components

    1. Just populate then and theme them.

  3. Delivering “3. Site-unified components

    1. Create and populate the things that make up your “site-unified components”

    2. Theme them

Project’s Epics Work Breakdown Structure

  1. Outline your content types starting from the “Full node” view modes. Identify other view modes for your content types. Start creating those into “Tasks”

  2. Do the same for other Drupal entities: Entities, Files, Comments ..etc.

  3. Deliver “4. Full nodes and entities

  4. Deliver “5. Views, view modes, and other content

  5. Deliver “6. The annoying 20% of the site

Finalization Work Breakdown Structure

  1. Final overall testing

  2. SEO, Printability, Performance, Security, and Accessibility tuning and configuration

  3. Your pre-launch checklists

  4. Go live!

 

FREEBIE: The Standard Drupal Work Breakdown Structure Template

Our Standard Drupal Work Breakdown Structure Template provides an outline of these phases and detailed tasks to be done that we use for every Drupal project. This template is made to be easily imported to JIRA. It contains:

  • a master sheet that aggregates the standard epics, tasks and stories to be easily imported to JIRA.

  • a sheet for defining the project’s own epics and stories

  • the standard Initialization and Finalization work breakdown structure that must not be missed for any project

All of this helps to reduce discrepancies in developing each project, not to miss important tasks and also allows our team to deliver a project fast, and incrementally (delivering in the first week of development).

Using The Template

The template is a Google Spreadsheet that you can easily clone and customize. To do so:

  1. Open the sheet and copy it to make it yours.

  2. Feel free to edit the sheet to make it your own. There are some instructions on how to use the sheet to make it yours.

  3. Follow the instructions on what to edit. We recommend that the “Initialization WBS” and the “Finalization WBS” stay intact (you can edit them once to your standard flow, then replicate for all projects).

  4. For each project, you will want to copy your template to customize the “Project’s Epics WBS” as per the project. The template has some samples for you to consider.

  5. Once done, export the “Master WBS” sheet to CSV. So you can import to your JIRA project.

  6. Map fields to your JIRA. See sample [image to illustrate mapping]

  7. That’s it!

 

Conclusion

Two things have helped us to standardize our work process when developing a Drupal site, and insure consistency and quality:

  1. Starting a project by finishing up components-first approach, not page-first approach.

  2. Documenting our recurring tasks and processes in a Template that uses this approach. This template makes applying this process easier for you.

Next time you start a Drupal project, consider this approach and let us know how this would help you in the comments section below.

Note: This does not depend on a specific Drupal version, this methodology works with Drupal 6, 7 or 8. It depends on Drupal’s conceptual building approach.

Tags:  Drupal Planet drupal 8 Project Management Drupal Templates Title:  Project Manager’s Guide to Breaking Down a Drupal Site for Incremental Delivery
Catégories: Elsewhere

Drupal Console: Drupal Console alpha1 is now available

Planet Drupal - jeu, 21/04/2016 - 09:24
We are so excited to announce the first Alpha release of Drupal Console. Almost three years of working on this project and after 84 releases, almost 86,000 downloads and the awesome help of 169 contributors we released the 1.0.0-alpha1 version. What is so great about this version  This release provides support for latest Drupal 8.1.x version released on April the 20th. For more information and details about this Drupal release you can visit Drupal 8.1.0 is now available. This release includes minor fixes and improvements and only one new feature. Support for placeholders on chain files, I will elaborate about this on another blog post, but if you are interested to know about this please visit the issue 2055. What is not so great about it Drupal 8.0.x is no longer supported. We are still trying to confirm if the Embedded Composer project can help us with this issue. If this is not doable, we can open a discussion to find a better way to approach this issue.
Catégories: Elsewhere

Pages

Subscribe to jfhovinne agrégateur - Elsewhere