Summary: This post provides a way to develop/debug in a web environment where you don't have control of the calling party and your environment must be available on the global internet.
It can be difficult to debug/develop a handler for an incoming web service when you don't have control of the caller. For example, I was developing a handler for Mandrill's inbound webhooks, where Mandrill calls your site when an email comes in. So in this case
- The receiving website must be accessible on the internet so that Mandrill can call it.
- Mandrill alone is hitting the site. I could debug with an imitation POST, but then that wouldn't be the real thing would it?
There are a couple of ways to approach this:
- Set up a full debugging environment on an internet-connected host (For example, set up a full dev environment possibly with GUI and PHPStorm for example, on an internet host. This is easy enough to do but kind of ugly.
- Debug with the application on the server but your debugger local using dbgp. This is do-able and an alternate approach to what's described here. It's a bit more awkward, but great for when the issues or problems can't be recreated on your local machine. It probably also requires most of the strategies shown here, but tunneling the debugger connection instead of tunneling the actual HTTP request.
- Use printf-style debugging, outputting logs of everything that happens. This works, but you have little control and it's really ugly and you don't have enough info about what's going on.
I use the wonderful PHPStorm IDE with debugger (which they provide free to open-source projects like Drupal and Warmshowers.org!), but this technique should work for any IDE. You do have to already know how to set up your system for debugging.
What you need:
- A host on the internet. Any machine which is reachable directly on the internet. My instructions will explain how to do it with a linux machine. This really can be any machine, it's going to be used only to proxy a port to your local machine. You could even use a production machine without doing any damage. You do have to have control of the sshd configuration (/etc/ssh/sshd_config or equivalent). And you have to make sure that no firewall is blocking the port you choose to use (my example uses port 8888.)
- Your local IDE/project set up for debugging with Xdebug. (I'm sure you can do this with any other IDE and with Zend Debugger as well, but I will only show for PHPStorm and xdebug.)
- Make sure you can do step-debugging on your local machine. Use the classic technique where you create a debug configuration for your project rather than the new PHPStorm "Listen" button. (I imagine there's a way to do this with the listen button.) Test this by setting a breakpoint and running your project and making sure it stops on the breakpoint. If you don't already know how to do step-debugging, this would be a good time to learn. It's a critical skill for any developer. Resources: (PHPStorm instructions, Drupal.org article, PHPStorm debugging article) (Images of my PHPStorm web application configuration are below.)
- On your internet-connected machine, configure sshd to set "GatewayPorts yes". (The default is "no"). This is configured in /etc/ssh/sshd_config on both DebianUbuntu and RHEL/CentOS distributions.). After changing the configuration, restart sshd with "sudo service ssh restart"
- Use ssh to create a tunnel which will bring a port on your internet-available machine to your local machine: ssh -R :8888:localhost:80 example.com will bring traffic on port 888 of the internet-connected "example.com" to port 80 on localhost (where your debugging environment is). Essentially, once you've done this, all traffic hitting http://example.com:8888 will actually land on your local machine on port 80. (This won't work if you're blocking port 8888 on the firewall of your internet-connected host, of course.)
- Try accessing your internet-connected machine with a browser to demonstrate that traffic is routed through. With my example, you would now be able to hit http://example.com:8888 and the dev site on you local machine would appear (slowly, perhaps). If you don't have access to your local dev site now using this technique, you aren't ready to continue.
- Test debugging by accessing your external site. Start debugging on PHPStorm using the configuration you created earlier. The console will show "Waiting for incoming connection with ide key '15247'". Your IDE key will be different of course. With a breakpoint set, visit your external internet-connected URL like this: htttp://example.com:8888?XDEBUG_SESSION_START=15247- you should see your IDE take control and stop at your breakpoint.
- Now configure the external service with the URL it is to hit when some event happens AND with the XDEBUG_SESSION_START query string. I was working with the Mandrill Incoming project, so my inbound URL looked like this: http://example.com:8888/services/rest/mandrill_events?XDEBUG_SESSION_START=15247
- Whatever you have to do to cause an event to happen, make it happen. In my case, this meant sending an email via Mandrill which was to be delivered to my web service.
- Voila, your debugger opens and lets you step through the web service call.
PHPStorm Server Configuration
PHPStorm Web Application Configuration
PHPStorm "Waiting for connection"
Today I learned that the Drupal Behat Extension provides an extremely useful step definition for debugging. Breakpoints!
As the saying goes, “If it ain’t broke, don’t fix it.” A lot of people live by this saying. In the world of IT, this doesn’t appear to be the case. With the ever-changing IT landscape; constant changes, updates, upgrades, and “bigger and better” widgets, it is crucial to update your digital presence and strategy for a successful web presence in the dynamic world of IT.
Businesses are looking for scalable solutions to adapt, remain flexible, and integrate across several applications in order to remain competitive. As part of these improvements, employees can become frustrated with learning something they are completely unfamiliar with. Upgrading end-user Content Management System (CMS) environments can often be a source of frustration for web content administrators. Suddenly, an existing website on a familiar platform becomes redesigned and built on a new CMS, causing frustration and panic. This leaves staff with a loss of ownership and an extremely high learning curve to master.
This scenario was something that I dealt with during the redesign and rebuild of our partner’s web platform, Sciencenews.org and Society for Science and The Public (SSP), a non-profit organization dedicated to public engagement in scientific research and education. Sciencenews.org and SocietyforScience.org are basic sites, with tons of informative and detailed content. SSP wanted a fresh new look and feel for its websites on a Drupal platform. The Drupal platform allowed for additional functionality and complexity that was new to the entire content team. Learning Drupal seemed like an extremely tough challenge, but was something that we were prepared to work on together.
In the end, the website launch was very successful and the training process helped me hone my skills and best practices of how to prepare a content team for a massive CMS overhaul. Below, I have outlined the steps I developed from this project to successfully help teams prepare for a CMS adoption.1). DO: Be Flexible
The client was very flexible with our schedules and worked with us closely before and during the implementation and launch process with regular meetings to keep the staff informed and involved in the launch and CMS developments. Communicating key dates to the client was a crucial success factor here. It ensured everyone remained up to date and prepared for changes and added features and functionality.2). DO: Be prepared to answer questions (with confidence)
The Science News team was very good at coming prepared to meetings with questions about the new CMS. They had some great fundamental questions such as, why the content management system was being implemented? And how such a tool could benefit the SSP team? It is important to work with the client initially to understand their goals and needs in order to explain the reason’s for the CMS changes fully and clearly to the content administrator team. With questions like “how do I edit the homepage?” to “how can I update Ads across blog pages and sponsored content sections?” use these questions to demonstrate the benefits of the new system to the content team. Part of training is selling the new system to its new users!3). DO: Make sure your team takes full advantage of CMS training
Our training sessions varied in detail from Science News to the Society News site. A lot of this had to do with the complexity of each site, but the SSP team was always prepared for onsite training. The teams were excited to learn the new system and everything it had to offer and were fully prepared for the challenge ahead.4). DON’T PANIC
This is great advice for the trainer and the trainees! Employees like the status quo. New technology involves a learning curve that leaves a lot of people uncomfortable. As a trainer, do not panic. Instead, encourage staff to feel excited about the new technology and the opportunity to master it. With a system such as Drupal, changes can be handled and managed by the editorial staff, not developers, which means both teams are empowered to manage their digital presence at their convenience.5). DO: Have Fun!
I know it can be really stressful, especially as lots of users are learning something completely knew, but it can be very helpful to think about a new CMS implementation as a fun and exciting opportunity! Understanding the ins and outs of the system and understanding how this can benefit the company in the end will really help the team in the long term. Our only constant in this world is change, so let’s enjoy it and grow!
Looking for more tips on launching your content management system and website? Check out Kellye Roger’s blog post: “Avoiding the Top 5 Mistakes Web Leaders Make With Their CMS Launch/Relaunch in Drupal CMS“
One of my clients, SproutOnline.com, has a variety of games and mobile apps that submit user-created data via POST to a URL and then email data to a gmail account. Not the most elegant way to do it but it's worked for years and they never had a problem. Last week something broke and we couldn't (quickly) find the cause. Instead of spending more time troubleshooting, I decided to create a resource in Services 3.x that would save each POST as a node. Read more to see how I did it.Tags: DrupalDrupal Planet
In this podcast, I have a chat about Drupal with Josh Waihi, Enterprise Architect for Catalyst IT, and also find out why you should attend Drupal South in Wellington, New Zealand in February 2014 if you possibly can (Submit your session proposals now!)
If you are reading this announcement right now, then we did it! Drupal.org runs on Drupal 7! This was a big and complicated project, which took longer than we expected. But we are finally done!
So, Partyopoly is in alpha2 and I thought it was time to start talking about it. There is a long and well thought out description on the sandbox page, but it’s a bit marketingy and I am more of a fan of look and touch than lets think about it.
So here it is!The Journey
I have been working on this project for a long time on many different sites and iterations, but I wanted to figure out how the ‘package’ as it currently exits works for someone who has never used it before…Installing
Partyopoly is currently still in a sandbox. So the best way to get it on a server is to git clone the repository and then use ‘drush make’ to build it the web root.cd /var/www/git git clone http://git.drupal.org/sandbox/rlmumford/1905260.git partyopoly drush make partyopoly/build-partyopoly.make /var/www/partyopoly
This will build drupal and download a huge amount of modules that partyopoly requires, while checking out the relative commit messages and in some cases applying various patches from the various issue queues.
Don’t you love distributed code?
Now you are able to install drupal as usuall and… Oh dear, it uses omega as a default theme!
Quick, throw Marinelli on there so that it at least has some skin!
Anyone else struggle to find simple clean drupal themes for demo sites?
Now we need to configure it a little.Configuration
First things first, Partyopoly has a wonderful feature that is the Party Dashboard, but before we can take a closer look, we need to configure solr.
No not SOLR!
Don’t worry, you can swap out the Solr Server with a Database Server and the indexes will still work out of the box.
If you have never set up solr before these two posts should help:
Next head over to
Configuration -> Search and Metadata -> Search API (/admin/config/search/search_api) and enable the Solr Server and the Party Index
Next we can start adding Parties to the site. Go to the Party Dashboard (/admin/party). This should still be a bit boring because our index is empty, so click on Add Individual, fill in some details and save.
We may need to clear caches or go back to the Party Index settings and tell it to index the first time, but we should now have our first Party on the Dashboard!
Boom!Lets start building!
Lets add a few more Individuals and also an Organisation. Its starting to look pretty good.
I can even add Individuals to said Organisation and kill off an employee!
Imaginary employee homocide is no joke, so lets not get carried away!
That is basically it! There are many more things that Partyopoly comes with, but as a first taste, I think that suffices.Demo Site:
Feel free to click around the demo site to have a look, but don’t try to add/edit/delete any content… The site will crash! #lazysysadmin
As we mentioned before, in Gizra each project's GitHub repository - code, issues etc - is completely open & transparent to the client from the day we start working together. We've discovered this provides us with some unexpected benefits.
I may be alone in this, but I hate having to list Views handlers in the .info file in order for the class registry to register them.
CTools has a mechanism to autoregister its plugins. Why doesn't Views have this mechanism for Handlers? Because Views doesn't know what handlers actually exist in a meaningful way.
But your module does.BlogDrupal · Views
This week we continue to re-publish lessons from the Pantheon Academy, with Jon Peck. In these free lessons we'll be taking a look at working with Git and MultiDev, the Pantheon cloud-based development environment for teams working on projects together. We'll also take a look at dealing with error codes, and setting up SSL fro your server. We're going to wrap up our current list with a checklist for taking your site live.
On the 22nd October Manchester saw it’s very first Drupal Sprint, this was run in parallel with London Sprints.
Regular sprints are now becoming a great way to help out with Drupal Core and Contrib. In the UK there are now two sprints covering London and Manchester. In India there are 3 user groups all sprinting regularly. No doubt there are some in the US and in other countries.
Sprints unlike regular user groups are an ideal place to flex your code knowledge and get your hands dirty. With initiatives such as the Drupal Ladder it’s proving that everyone can contribute in some fashion whether this be via code, documentation or testing. Within the first 30 minutes of Drupal Sprints Manchester we had our first commit to a contributed project (Omega 4) by a first time contributor.
We communicated with London via Google Hangouts where we updated each other on our progress, I know in the past that London has spoken with India an impressive feat considering the time difference.
It’s not out of the realms of possibility to organise a monthly world-wide sprint. The idea being that user groups all over the world organise a sprint for a specific date with the aim of working on core and contrib. I believe this will be possible with a minimal amount of effort as we can offload a lot of the organisation to each user group. Coordination of communication can be done through IRC. It would be a great publicity stunt as well, something that every country could benefit from especially in the run up to Drupal 8 release.
If anyone is interested in a world wide sprint comment below and lets organise something!drupalDrupal PlanetDrupal Sprints Manchester
Jeff Eaton (eaton), Senior Architect at Lullabot, Drupal core contributor (Token API, Form API), author/maintainer of the Voting API, Entity Views Attachment, and others. Host of the Insert Content Here podcast, and (according to our very unscientific poll) one of the most popular Drupal speakers ever in the history of the world joins Andrew Riley, Ted Bowman, and Mike Anello to talk about what makes a great presentation and (possibly) to retire one of the five questions.
We are happy to inform you that we have put new beautiful photo gallery module on Drupal.org
Project page: https://drupal.org/project/swipe_photo_gallery
This module show the matrix of images on one separate page. User can see only one page in matrix and go to the next with Up, Down, Right or Left key or using swipe in Up, Down, Right or Left direction.
You can see demo on http://carolsachs.com/overview
We have done site of Carol Sachs and understand that there are some custom solutions that we must share with world Drupal community.
The simple instruction how to use this module:
- Turn on module as usual drupal module.
- Check that you use Jquery 1.7 or higher with Jquery Update module!
- Put several nodes with Slide content type. This content type will appear automatically.
- Go to the configuration page admin/config/media/swipe_photo_gallery and set path where you would like to show your slides.
- Go to this page and enjoy your slides.
Feel free to send us any recommendations and ideas that can improve this module!Blog tags:
Greetings! We took a week off last week to attend BADCamp. There was sprinting-o-plenty, as well as tons of sessions on Drupal 8.
Photo credit: Amazee Labs
In the meantime, a lovely new Drupal 8 landing page has launched, which includes a list of major new features, information about the current state of the release, and a nice, big, green, demo button!
And speaking of Drupal.org, don't forget to get your patches downloaded in advance of the Drupal.org D7 launch, when d.o will be down for 24 hours. Please take the opportunity to go outdoors in the sun and otherwise step away from the computer. :)Notable Commits
The best of git log --since "2 weeks ago" --pretty=oneline (121 commits in total):
- Issue #2099363 by tim.plunkett, mtift, swentel: Allow single config files to be imported and exported is another super handy patch. This takes the Views import/export functionality from Drupal 7 (which allows you to easily copy/paste Views definitions between sites) and expands it out to cover any configuration file in the system! An awesome feature, for both site builders and developers.
- Issue #2112807 by tim.plunkett: Move the form builder functions in form.inc to a form service: This snazzy patch moves the guts of includes/form.inc to OO code in the form of FormBuilderInterface, and increases the unit test coverage from 0% to 70% in the process. Awesome work!
- Two big patches went in for improving Drupal 8's accessibility: Issue #1872206 by jessebeach, quicksketch, Wim Leers, mgifford: Fixed Improve CKEditor toolbar configuration accessibility, which provides both an accessible way to create CKEditor toolbars in the admin interface, as well as makes those toolbars output markup accessible to end-users, and Issue #1851414 by nod_, quicksketch, frega, dawehner, damiankloip: Convert Views to use the abstracted dialog modal., which replaced Views's custom JS with the standard dialog library used throughout core. Win!
- In Developer Experience land, Issue #2057259 by alexpott, cweagans, swentel: Fixed Malformed modulename.info.yml file causes fatal error greatly helps to reduce brittleness in error handling, Issue #2107137 by dawehner, damiankloip, tim.plunkett: Fix the DX for declaring custom access checkers provides a much less verbose option for folks who just want to stick an access method on their controllers and use it, Issue #2112711 by tim.plunkett: Provide an easier mechanism for using drupal_get_form() directly nicely cleans up form builder calls, and Issue #2098795 by tim.plunkett, Sean Charles, tstoeckler, dawehner, larowlan, damiankloip: Create Base Class for RouteSubscriber Class makes defining routes much easier, by consolidating commonly-used properties into a base class.
- Meanwhile, in Router System Conversion Ville, Issue #2102125 by dawehner, tim.plunkett, neclimdul, pwolanin, YesCT, damiankloip, Tim Bozeman: Big Local Task Conversion went in, to move all local tasks to the new routing system. Work is still ongoing to replace theme callback and convert contextual links. On the bright side, Issue #2046367 by jhodgdon, amateescu, larowlan, juanolalla: Menu routing docs do not match what the code is doing will help folks figure out the new routing system much better now. :)
- In the ongoing quest to create dream markup, Issue #1982256 by mortendk, rteijeiro, tim.plunkett, LewisNyman, tlattimore, yannickoo, ry5n: Clean up html.html.twig markup removes numerous extraneous divs and classes from Drupal's default markup, which can be added back by themes that need them.
- And finally, a follow-up from an old security fix in Drupal 7, Issue #1934498 by attiks, Jelle_S, David_Rothstein, effulgentsia, tstoeckler: Allow the image style 'itok' token to be suppressed in image derivative URLs cleans things up so derivative images can work once again on CDNs and whatnot.
Blog posts about Drupal 8 and how much it's going to rock your face.
- If you're a book author, trainer, etc. don't miss Survey Results: Drupal 8 Information Needs, which has lots of helpful graphs about what people want to know about Drupal 8. A must-read for documentation writers, trainers, book authors, bloggers, and screencasters! :)
- In Getting Excited for Drupal 8 - The Donkey, Human, Ant, Chameleon CMS, several core contributors chime in on what gets them excited about Drupal 8.
- If you want to stay up to date on the latest happenings with Drupal 8's upcoming Drupal 6 and Drupal 7 => Drupal 8 migration path, you can subscribe to the The IMP initiative: Migrate in Core issue, where ongoing updates will be posted. You can also read a progress report of what's happened at BADCamp and since.
- Why PHP is for Real attempts to contextualize some of the big OO shifts in Drupal 8's architecture with some historical insight about the PHP project.
- Dynamic Routes in Drupal 8 with a RouteSubscriber covers how to convert from old hook_menu()-style routes to the new system in Drupal 8, and Drupal 8 - New comment notifications provides a handy, short module for automating mail sending in D8.
- In Focusing on Contrib for Drupal 8, Chris Brown makes the case for why developers should start focusing on D8 contrib now, from a career development point of view.
- Two sprints are coming up in the next week: another edition of "First Friday" Contrib Sprint Initiative (CSI) (read about the results from last time) and the
Drupal 8 Tour Writing Sprint, to help get additional site builder tutorials into core.
- Drupalhagen in Copenhagen, Denmark will also cover a number of Drupal 8 topics, including Twig, technologies such as Guzzle and Symfony, the new plugin system, and even a talk about the TYPO3 CMS.
- Later on this month, don't miss DrupalCamp Vienna with a series of sprints in addition to Drupal 8 trainings.
Do you follow Drupal Planet with devotion, or keep a close eye on the Drupal event calendar, or git pull origin 8.x every morning without fail before your coffee? We're looking for more contributors to help compile these posts. You could either take a few hours once every six weeks or so to put together a whole post, or help with one section more regularly. Contact xjm if you'd like to help communicate all the interesting happenings in Drupal 8!AttachmentSize Screen Shot 2013-10-29 at 11.19.37 PM.png560.09 KB
Drupal 7 was released with an automatic cron option which is handy for those who do not have shell access or a way to enable a Unix type cron with their web host.
At Classic, we use Jenkins to trigger cron via Drupal's cron URL. Because we have cron scheduled, we have no need for Drupal's built-in automatic cron which defaults to run every 3 hours. Also, we sometimes have to disable the Jenkins cron during deployments to make sure that that cron doesn't execute until all code has been deployed successfully.
There are a number of ways to disable Drupal 7's automatic cron if you decide you do not need it. I have outlined these options below. You can decide which is the best for your situation.Drupal UI
The Drupal UI is always a great way to go for confirmation settings.
Go to the following path admin/config/system/cronand set the "Run cron every" drop-down to "Never". Next, click the "Save Configuration" button.Drush
Many prefer to use Drush and find it faster and easier than using the Drupal UI for configuration tasks.
Run the following Drush command in your terminal. Make sure to change directory to your Drupal install. drush vset cron_safe_threshold 0settings.php
Setting configuration variables in the settings.php is a great way to lock in changes that you don't want someone to change in the Drupal UI.
Add the following to your settings.php: $conf['cron_safe_threshold'] = 0;hook_update_N function
I often find adding configuration changes to hook_update_N functions a great way to make configuration changes during deployments.
Simply add the following line of code inside your hook_update_N function: variable_set('cron_safe_threshold', 0);Drupal 8
Drupal 8's automatic cron setting currently defaults to "Never". According to this issue, the intent is to still have Drupal's automatic cron default to run every 3 hours.
It looks like disabling Drupal 8's automatic cron can be done in the Drupal UI or through the configuration override system. I need to dig deeper to test which options will work best for our environment as we move to Drupal 8.Blog Category:
berdir unified the DatabaseStorageController and the DatabaseStorageController classes. I have been working on migrate, this reports is at https://groups.drupal.org/node/357993
Have you ever wanted to try something new on your website, but you were afraid it might go wrong and leave you with a broken site? Or maybe you want a second website that is is a close match to an existing one? Creating a copy of your live (or "production") site gives you the opportunity to explore changes to its current configuration with less risk. This tutorial will show you how to create a copy (or "development") website using a backup of your production site.
I wasn't sure how to describe this approach so I felt the title above was maybe eye catching enough to get people to check it out. Here are the modules showcased all playing nicely together to produce a unique content contributor workflow for complex elements in a node.
In September, we deployed a survey with the goal of better understanding the kinds of information various audiences need to learn and understand Drupal 8. The survey results are helping to tailor the content on Drupal.org including this new Drupal 8 overview page. The page will evolve over time and will soon include links to deeper information such as technical documentation. The results will also help determine which topics need the most amplification on social media.