Creating a Drupal plugin system

November 13th, 2018

Filed under: Computers and Technology , Software , Work/Professional

Drupal 8 has a great Plugin API that not only allows one to write plugins that modify Drupal core’s behavior, but also to define your own pluggable system that you and others can plug into as well. The Plugin API documentation has a number of examples that cover many common use cases for plugins and configuration entities, but none quite covered my use case, so hopefully this article can help fill some of the gaps. I’m going to use an actual example that I built to help ground the discussion of abstract concepts in something more concrete — the abstractions like Annotations, EntityWithPluginCollectionInterface, PluginFormInterface, Derivatives, and others are powerful, but it can be tough to pin down how and when to actually use one versus another.

The Challenge

My end goal is a module that creates Drupal nodes of a custom “Event” content-type based on a variety of external calendaring systems. My campus uses the Resource25 calendaring system which will be the source of most events, while others may be sourced from Outlook calendars, Google Calendars, or other data sources. What the source systems are and how they are connected to will never be completely unified and will change over time, so I want to create a flexible, pluggable, system that will allow new plugins to be written that just cover how to fetch from new source systems without reinventing the whole thing every time we need to add a data source.

There are two complications that lift this implementation out of the realm covered in a straight-forward manner in the Drupal documentation:

  1. Each plugin has both standard configuration options (such as “Enabled”) as well as plugin-specific configuration options.
  2. For a given Drupal site, there might be multiple instances of each source plugin that are configured to pull from different calendar feeds using the same protocols and methods.
Diagram of the relationship between plugins and config instances.

Relationship between plugins and Configuration Entity instances

 

This system is very similar to Drupal-core’s Block system. The Block system defines an API for “Block Plugins” where each block-plugin has PHP code that drives it. Multiple instances of each block-plugin can be placed in the page, each with their own settings. For example, the System Menu Block Plugin in the system module can have multiple instances added to the block-layout, each configured to display a different menu with different settings for initial menu-level and depth of menu-levels visible as well as the standard settings for title and display of the title.

Add block dialog example

 In fact, the Plugin API documentation specifically recommends using the Block system as an example. Unfortunately this is a complicated example to follow because the Block system is scattered between Drupal-core library files and the Block module, rather than all contained in one place.

 

What is needed?

I’m going to skip over the basics of the module skeleton as these are well documented already in order to focus on the parts that are needed to achieve the system described above.

Here are the pieces that we need to get functioning and which I will dive further into:

  • Defining the plugins themselves with a Plugin Manager and discovery mechanisms so that Drupal knows what plugins are available.
  • Defining the Configuration Entity for our plugin system so that instances can be configured and exported along with the rest of our site configuration.
  • Defining dynamic links so that we can create new instances for each plugin installed.
  • Allowing plugins to define extra configuration fields/properties that are particular to their needs.

A snapshot of the resulting module is available on GitHub at adamfranco/middlebury_event_sync. I’ll be linking to code in the module rather than copying it here.

 

Defining the Plugins

For this system, we are going to define a single Plugin Manager that will provide discovery and loading of our Event Source plugins and a Plugin Interface that all of our Event Source plugins will have to implement.

The Plugin Manager gets a listing in our module.services.yml as well as a class that defines what the alter-hooks are and what the plugin name-space is. More on Services and Dependency Injection can be found on drupal.org.

Next up is an Annotation class at src/AnnotationEventSource.php which defines the parameters that will be used in each plugin class‘s doc-block to identify that plugin class as one of our plugins. This is the key to the discovery mechanism we’re using — the annotation in the doc-block registers the plugin so that it is available through our Plugin Manager.

We also define Plugin Interface which defines common methods for all of our Event Source plugins. This isn’t strictly required for all plugin systems as the PluginInspectionInterface and/or ConfigurablePluginInterface can be implemented by plugins directly, but I wanted all plugins to implement a few common methods for standard settings and actual operations.

The manager, service, annotation, and plugin interface are the things we need to define to allow our plugins in src/Plugin/EventSource/ to be discovered and available to Drupal. Now that we have defined our plugins, we can use \Drupal::service('plugin.manager.event_source')->getDefinitions() to access all of the Event Source plugins installed in our site, both those provided by our module as well as those provided by other modules.

 

Configuration Entities

The next task is to define a configuration entity, the instances of which will hold the configuration for each service we are connecting to and reference the plugin that will be used to do the connecting. Configuration in Drupal 8 is  a great new system that allows configuration to be modified in a development environment, exported, version-controlled, and then imported in a production environment. Configuration Entities allow units of configuration to be created, listed, edited, and deleted like other entities in Drupal (nodes, users, taxonomy-terms, menu items, etc). For this module, I need to use config entities because we don’t have just a single global setting, but rather zero or more configuration sets for each plugin.

Our config entity is a single class that extends ConfigEntityBase and has a number of annotations that point to the Controllers and Forms used to list and edit the instances of the Entity. Our config entity class also implements an interface I created that defines how to access the plugin associated with the config entity, not something available by default.

If we just wanted to have a single config entity for each plugin, this would all be enough. However we want to be able to have multiple config entity instances associated with each plugin. To achieve this our config entity must also implement the confusingly named EntityWithPluginCollectionInterface which provides access an EventSourcePluginCollection that we define. This “Plugin Collection” handles the mapping between config entity instances and the plugin instances that house our actual data-fetching code. Via this Plugin Collection we can pass a config entity (or its settings) off to the plugin instance so that the plugin instance has access to data in addition to its methods.

After defining the config entity and getting it working, it is a good idea to define a Configuration Schema in yml. The primary purpose of this is to allow translation of your configuration — it doesn’t seem to affect the actual export of the configuration.

Our config entity is pretty light-weight — it stores its id (a.k.a. machine-name), label, which plugin-id it is associated with, which module is providing that plugin, and finally an arbitrary set of plugin-settings defined by the plugin. These plugin settings include some common ones such as enabled, ttl, and time-shift, as well as plugin-specific settings like URIs, username, and password. There are other ways that the plugin settings could be injected into the config entity, but I felt that putting them in a single container called “settings” was flexible and simple and avoided potential conflicts between the keys needed by the config entity and the plugin.

 

Dynamic Links to add instances

A missing piece with what we have so far are the links to create new config entity instances. Because new plugins may be provided by other installed modules, we can’t hard-code our links to add new instances in our action links. Instead we need to implement a Derivative plugin that generates a list of links based on the available plugins and point to this Deriver from our module’s module.links.action.yml.

Screen shot of config UI with "add" links.

Configuration Entity “add” links provided by our Deriver plugin.

 

Extra configuration fields per plugin

To assist with common settings and functionality, I created an abstract base class that all of my plugins can inherit. This base class implements the PluginFormInterface to provide form fields on the config entity’s editing screens. Each plugin can override these methods to provide and handle additional fields while leaving the common fields to the base class. These fields are handled as a sub-form loaded in our config-entity form class.

Event-source edit form

Event Source editing form showing fields provided by the configuration entity itself (label & machine name), common fields from our abstract base class (TTL, time-shift, enabled) and plugin-specific fields (URI, username, password).

 

 

The Result

Once the items above were in place I now have a system that lets me create as many instances of each plugin as needed. The buttons to add a new instance are provided by the Deriver and the listing itself by a controller.

Event Source admin UI.

Admin UI for adding Event sources. Note that only two plugins are currently installed.

 

Hopefully this overview can point you enough in the right direction to implement your own Drupal plugin systems for other purposes.

Resources

Below are links to the documentation and resources that I used in writing this pluggable system and this article:

Street orientations in Vermont

September 20th, 2018

Filed under: Life and Everything Else

By now many folks have seen Geoff Boeing’s really neat charts of street orientation that highlight how “gridded” (or not) the layout of city streets are:

Since Geoff was kind enough to provide a full open-source tool-chain, I decided to tweak it to chart the orientations of the streets of some cities and villages in my home state of Vermont:

Vermont street network orientation

Unfortunately, until I get around to importing Vermont town boundaries, not all towns will be super easy to include in this analysis.

New Curvature site

November 17th, 2016

Filed under: Computers and Technology , Software

Tags: , ,

Several years after creating Curvature –my program that analyzes road-geometry and builds maps of twisty roads– it now has a dedicated site of its own:

roadcurvature.com

The new site is written to help non-techies understand how to use the curvature files with step-by-step instructions and a lot less jargon than in my original post about the program.

I also took this as an opportunity to learn how to do vector graphics in Inkscape and design myself a new logo — with significant design help from Alison.

Regex from the dark lagoon

November 13th, 2013

Filed under: Computers and Technology , Work/Professional

Tags: , , ,

As a software developer or system admin have you ever encountered regular expressions that are just a bit too hard to understand? Kind of frustrating, right? As a rule, regular expressions are often relatively easy to write, but pretty hard to read even if you know what they are supposed to do. Then there is this bugger:

/^(?:([0-9]{4})[\-\/:]?(?:((?:0[1-9])|(?:1[0-2]))[\-\/:]?(?:((?:0[1-9])|(?:(?:1|2)[0-9])
|(?:3[0-1]))[\sT]?(?:((?:[0-1][0-9])|(?:2[0-4]))(?::?([0-5][0-9])?(?::?([0-5][0-9](?:\.[0-9]+)?)?
(Z|(?:([+\-])((?:[0-1][0-9])|(?:2[0-4])):?([0-5][0-9])?))?)?)?)?)?)?)?$/x

This is the most complex regex I’ve ever had need to write and I just had to share. Can you guess what it might do? 😉

Continue Reading »

Further north and farther east — an exploration of Nova Scotia

September 11th, 2013

Filed under: Life and Everything Else

Tags: , , , , ,

This entry was original published as a Ride Report on the Adventure Rider (ADVrider) forums. The only changes are reformatting to flow as a single article.

Prologue

I’m continually thankful that I get to live in one of the more scenic corners of the planet. Central Vermont boasts numerous twisty roads that tie together charming villages over rolling farmland and steep mountainsides. While my back-yard riding options are nothing to take for granted, it was traveling by motorcycle that attracted me to riding in the first place. I have the travel-bug and long to explore exotic places. Reading the fabulous trip reports at ADVrider has only fueled a greater hunger to step out of my day-to-day environment and explore new places. Since the Rocky Mountains are too far to fit into my vacation schedule this summer, I settled on maritime Canada (and Nova Scotia in particular) as a suitable destination from my two-week end-of-summer trip. As an inlander, the sea-coast provides a novel and ever changing landscape to feast my eyes upon and the population density seems about right to allow me to get a bit of wilderness fix while never being too far from the next town. I also wanted to try my hand at locating camping spots on deserted beaches and former logging cuts as a way to get away from the RV crowd at public campgrounds. While I wasn’t quite sure what to expect on the ground, zooming around in Google Earth seemed to indicate many likely-suitable spots in Canada and Maine where no one would notice or mind a tent for the night.

My best guess at a planned route:

(The actual route can be seen below at the end of this post)

While the vacation itself is certainly the goal, at least 25% of the fun is thinking about and preparing for the adventure. In the weeks leading-up to the trip I took care of various maintenance on the bike, changing the oil and tires as well as adding a top-case and a few other bits and bobs. This trip also gave me an excuse to refresh my camp stove and a few other pieces of gear I haven’t needed in a while. One of the things I thought I’d try out was this brand new super-hydrophobic-and-oleophobic coating called “NeverWet”. After watching their YouTube videos I thought, “this would be perfect to keep water/mud off my boots/riding-pants”. I’ll come back to this later, but the moral of the story: don’t. Preparations began in earnest a week out and by the night before I had the house clean, the bike packed, and was ready to go.

Continue Reading »

curvature.py — find the most twisty-turny roads around

December 5th, 2012

Filed under: Computers and Technology , Software

Tags: , , , , , , , , ,

Update November, 2016: New dedicated Curvature site — roadcurvature.com

Update October, 2013: Google Earth KML files generated by curvature.py are now available covering the entire world.

In the process of taking up motorcycling this summer I also gained an additional hobby: scouring maps and travel guides to find the roads that would be most fun to ride. While I’ve had great times on dirt roads through farmland and wide open highways, there just isn’t anything that compares to the thrill of leaning through the corners on a winding road.

While I’ve had some good successes in locating roads by map (such as Tracy Road), one of the shortcomings of a map is the tight curves you can really lean into tend to be below the resolution for many maps. Atlases and electronic maps like Google Earth allow you to zoom in, but then there is the problem of finding the gems in the sea of data. What I realized I needed was a way to highlight just the most curvy roads so that I would know where to explore next.
Continue Reading »

The Middlebury Town Plan

September 24th, 2012

Filed under: Life and Everything Else

Tags: , , ,

Today I found unexpectedly good reading in Middlebury’s new 2012 Town Plan. The document is really well done and its quality highlights the vigorous engagement between the town, its citizenry, and businesses that makes this a fabulous place to live.

While all 226 pages are worthwhile, I found Section 2.13  “Land Use – Conservation and Development Plan” to be especially moving. I mean “moving” in a totally serious, non-ironic way. Coming from a town (Carlisle, PA) with rampant “Miracle Mile” commercial development and suburban sprawl, this forward-looking vision of how the town should be developed and improved in coming decades reassures me that 30 years from now Middlebury will be an even better place to live than it is today.

My favorite quote is from Section 2.13, page 151:

A fundamental objective of this Town Plan is to maintain Middlebury as a traditional Vermont town and to prevent incremental change to “anywhere USA”. This is not merely an aesthetic notion, it is a recognized economic development strategy for Middlebury and Vermont. This Plan supports architecture that is designed to fit its context in Middlebury and does not support standardized trade-marked or corporate prototypes.

In 2005 James Howard Kunstler (author of The Geography of Nowhere among other titles) gave a great talk at Middlebury College on human-scale urban development and the lack thereof in much of American urban planning. In his talk he justly derided our own little “Miracle [1/2] mile” by the Hannaford Shopping center. As an avid bicycle commuter who lived south of the village for many years I felt the effects of this poor zoning and development planning on a daily basis as I tried to safely navigate the no-shoulder/no-sidewalk turning-lane and fore-court infested section of road without getting killed. I am very pleased to see that slowly remedying these past lapses is part of the town’s plan for the future.

In addition to spending several hours reading the 2012 Middlebury Town Plan, I heartily recommend taking another hour to watch William H. Whyte’s The Social Life of Small Urban Spaces – The Street Corner. My favorite part is at 12:00: “People tend to sit where there are places to sit.”

Vermont to Michigan on a motorcycle

July 13th, 2012

Filed under: Life and Everything Else

Tags: ,

In 2005 I took a 3-week trip around Turkey with my parents and brother. Mid-way through this fabulous trip we met the affable Roland Pfitzenmaier, a German man touring the middle east on a Triumph motorcycle. While I am someone who usually travels with a full load of gear — bicycles, kayaks, and all the rest — the minimalism of Roland’s trek was intriguing. This summer my father was kind enough to lend me the use of his motorcycle (a 1993 BMW R100R) and I figured that there was no time like the present to try a long-distance motorcycle tour. Now, I’m well aware that for serious iron butts a 2,000 mile round-trip isn’t all that far — but for someone who is just getting into riding and has only done day-trips, four straight days on the road each way would be a significant journey.

My family has a small cabin on a lake in northern Michigan where I every summer growing up. Since moving to Vermont 14 years ago my attendance has slipped somewhat as the trip lengthened to a driving time of 15-17 mind-numbing hours along the flats of the New York Throughway and various mid-western highways. That said, I still try to make it to the lake at least once every few years. Since I took the full month of July off from work I figured I’d make the trip via motorcycle this time and learn if this sort of travel was for me.

To keep things interesting I planned a route out that would avoid expressways as much as possible and give me a chance to see the landscape of central Ontario — a region I haven’t seen before.

Total distance: 1077
Total moving time: 23 hours, 17 minutes
Average speed: 46 mph


View Larger Map
Key: blue line – planned route, red line – actual route

Continue Reading »

Black Bean Crostini

June 15th, 2011

Filed under: Life and Everything Else

Tags: , ,

252/365: Tomatos in SacksOne of the joys of participating in a CSA is exploring new vegetables and foods that I otherwise wouldn’t have known about or thought to eat. Last year we received husk cherries, red carrots, bok choy, several types of kale, purple potatoes, and at least 4 varieties of beets in addition to many more standard vegetable varieties. I don’t consider myself sheltered in terms of food, but many of these were simply things I never would have thought to look for even if they are available in a grocery store.

241/365: Red CarrotsFor the past two weeks our CSA share from the Gildrien Farm has included several cups of dried black beans, a food I’ve eaten many times but never really cooked with. In their weekly letter Jeremy and Caitlin helpfully included a recipe for Puerto Rican Black Beans, a tasty-sounding launching pad for the evening’s dinner.

Since I didn’t have any bacon grease on hand I figured I would just fry up several large pieces of bacon and use both the meat and the grease. I had planned to make a fritata as the main course for the evening, but after sampling the beans, decided to add some more veggies and put them on bread as our main course. Unfortunately, the result was so delicious that the crostini never made it out of the kitchen for a photo shoot.

Black Bean Crostini

  • 1 cup dry black beans, soaked overnight
  • 1 large onion, diced as small as possible
  • 1/2 a red pepper, diced
  • 1/4 lb of bacon (4-5 pieces)
  • 1 cup of cherry tomatoes, quartered
  • salt
  • 1 baguette
  1. Soak the beans overnight to soften, then simmer over medium heat for 45 minutes until tender.
  2. While the beans are cooking, fry the bacon in a skillet over medium heat until it is crispy and most of the fat has melted off. Pull the bacon strips out of the pan and let cool, trying to keep as much of the grease in the skillet as possible.
  3. Turn down the heat on the skillet to low. Add the diced onion and some salt to the bacon grease in the skillet and cook for 15 minutes, slowly letting the onion turn clear and caramelize.
  4. Drain the majority of the water from the beans (leaving about a half cup) and add the beans and their water to the skillet with the onion and bacon grease. Stir together with the red pepper. Raise the heat to medium and stew for another 15 minutes or so, until the beans begin to fall apart.
  5. Mash the beans in the skillet with a utensil of some sort until you have chunky bean paste interspersed with red-pepper and bean husks. Crumble the bacon and stir it into the beans. Salt to taste.
  6. Cut the baguette into thin slices. Pile a large dollop of beans on each slice and top with diced cherry tomatoes.

River Levels Widget v.1.2.2 available

March 23rd, 2011

Filed under: Computers and Technology , Software

Tags: , ,

RiverLevels 1.0 Screen Shot

The RiverLevels widget provides an easy way to monitor the amount of water flowing in your favorite streams and rivers right from your Dashboard. The RiverLevels widget is of particular interest to whitewater kayakers and canoeists.

Once any United States Geological Survey (USGS) stream-gauge station is selected, it is automatically refreshed to always provide you with the latest graph of the water-level. As of version 1.2 you can choose between two graph styles: discharge in cubic feet per second (CFS) and water-height in feet.

This widget is Free software, licensed under the GNU General Public License (GPL) version 3 or later.

Requirements:

  • OS X – 10.4 “Tiger” or later

Change Log:
1.2.2 (2011-03-23)

  • Fix for image URL change in USGS site.

1.2.1 (2008-02-10)

  • New zip archive includes the ‘library’ directory missing in the 1.2 release.

1.2 (2008-02-06)

  • Fixed Leopard (10.5) compatability bug.
  • Added the ability to choose Gauge Height (ft) in addition to discharge (CFS).

1.1 (2007-01-08)

  • Fixed graphs extending off bottom of widget
  • Fixed invisibility of front refresh icon

Next »