Drupal 8 offers big improvements to the front-end development experience. Twig improves working with markup; requiring libraries to declare dependencies allows us to deliver only necessary CSS and JS; and there are now two flavors of markup in core, provided by Classy and Stable. Yet we can still do more to improve front-end development with Drupal. At MidCamp in Chicago, a number of people who work with Drupal discussed some big potential improvements we could work towards, including:

• Component-based theming in core
• Object-oriented render system rather than render arrays
• Targeted libraries to improve core JS rather than a JS framework

We also worked on developing some initial personas to help better understand the variety of needs for those working with Drupal’s theme system.

These discussions helped point out areas we can explore, research and discuss further. While we had numerous members of the Twig team involved in the discussion, by no means are these decisions. There will be plenty of opportunities for others to provide input and shape the future direction of Drupal’s front-end.

Component-based theming

Drupal’s superpower? Content modeling.

Few content management frameworks do as good a job at modeling complex content relationships as Drupal. That’s great! However, much of our theming is then tied very, very tightly to those data structures.

Our render system typically begins with an individual piece of content, a node. That content can have multiple view modes, where fields can be displayed in different orders, and with different field formatters. Generally we begin by rendering the content within a field, then the formatter for that field, then the node as a whole, then we ask every block on the site whether it should be rendered in conjunction with this node, and finally we render the wrappers around the node to construct the final HTML.

Views allows for less tight coupling of rendering from Drupal’s data structures. In contrib, Panels also does a solid job of moving away from data structures as the primary way to render content.

Approaching a design with components

For front-end developers implementing a design, we typically begin by breaking down a design into components. We might begin with large chunks of the page, like the header and the footer.

Within the header, we might identify more fine-grained bits: the site branding elements like the logo and site name, a primary menu, a secondary menu, a search box, social icons. A hamburger icon might trigger an offscreen menu to appear that could have several identifiable sections within it.

Even the main body area of an article likely can be broken down further: the byline at the top of the article, the comments section at the bottom, a share bar.

Ideally we break down the design into a series of components. It’s entirely possible those components might be used in multiple places in a site. For example a landing page for a particular topic might have a list of articles: the styles for those article teasers might be the same as article teasers on the home page.

A component-based approach to that design might use BEM class names for each component. The root div for a particular component might have a class of component-name, while each part of the component might have a class of component-name__element-name. Slight variations in components can be handled with a class name like component-name--modifier. Within the CSS, you’d apply all of the styles for each element in a component with a selector consisting solely of that BEM class name (at least in an ideal world).

If you’re using Sass, styles shared by multiple components can be applied with a mixin. I typically use a separate Sass partial file for each component. Those partials can then be pulled together into a larger aggregate CSS file, particularly for global styles. If you’re not using Sass, you can still use a single CSS file per component, and let Drupal take care of aggregation through careful use of libraries.

This has some big benefits, particularly when working in larger teams. Different team members can work on different components and minimize the risk that their work creates conflicts with each other. By scoping CSS to a particular component, this also minimizes the risk that style changes in one part of the site will have inadvertent effects on other parts of the site. Using a singular BEM class to apply styles also helps to avoid specificity wars caused by chained selectors. Once you go down that path, forever will it dominate your destiny. So overall a component-based approach to markup and styles can reduce risk on a project.

Current approaches to components in Drupal

While you can apply all the styles for a component in a single Sass file, it’s much harder to get all your markup in the same template file. Because Drupal’s templates are tied tightly to data structures, you often need to create several template files for each component, often with corresponding preprocess functions. That can be pretty clunky. When you need to start chaining together multiple theme suggestions in order to tightly target a particular bit of markup, the risk matches that caused by overly complex CSS selectors.

There are ways within Drupal to set up component-like structures. A display mode for a content type could be a component. Display Suite allows for more complex layouts on a display mode. Views can help create components. Blocks, particularly when they are fieldable entities. Paragraphs help break a big body field into smaller components. And perhaps most interestingly, custom panel panes can look remarkably like an ideal component.

Steve Persch demonstrated a technique he uses where he creates a custom panel pane or mini-panel for a component. He creates a custom layout plugin for that component, only instead of defining regions in that layout, he defines component variables. For example, an event listing might have a component for each event that consists of the event name, a thumbnail image and the event date. Steve then creates a layout “region” for each of those pieces of data. Within the panel configuration, he can select the necessary data sources and assign the exact pieces of data to the region. The layout template then has the exact variables it needs and can construct the precise markup needed for that component. These component layouts can be placed into categories, so that when you create a new custom panel pane, you can select a particular layout to create a certain kind of component.

That could provide a really powerful model for component-based theming in Drupal. A component/layout plugin defines the variables needed for a component template. In the UI, you select the data sources, connect to related data sources if necessary, and assign the exact data to the particular variables defined by the component template. That allows front-end developers to have all the markup for a component in one template. The library system could attach CSS and JS to that particular component to only load those assets when a page uses that component.

Combining components

Components also need to be combined with other components into a component section. For example, multiple components might be combined to create a header section. In principle, the same technique could be used. A component/layout plugin defines where each component in the section should go, and those components can then be placed in the UI with the appropriate data/context sent to the child component.

The same principles would apply for full page templates, with a layout where large page sections or even individual pages could be placed.

Components often combine with each other through content lists as well, e.g. a river of news items. So imagine if parts of Views were used to construct an individual component, while other parts were focused solely on the construction of lists of component objects. A component builder and a list builder, if you will. That might be conceptually a lot simpler than how Views works now.

Entity queues are also an important way to construct lists: assigning a particular component type to each item in a queue seems doable.

Component lists, either those assembled automatically or manually, might also need additional pieces of data assigned before and after the component list itself, or even as attributes on the wrapper for the list itself. Making this flexible without being overly complicated could prove challenging.

Components for site builders

Defining design components and page layouts so specifically could work great for front-end developers, but what about site builders? Drupal excels in part because of how it enables the assembled web. Would such a model work for them too?

Sure! This maps a lot closer to how site builders think of putting together a site.

• “I’d like to put a search bar right here in the layout.”
• “Could I have some social icons over here?”
• “That looks great, but I’d like a photo with a caption in the footer.”

This component-based approach fits that mental model on how to assemble a site, so that part’s good.

Where this probably gets trickiest is when you get down to the level of an individual component. Components with regions for multiple pieces of data might work well for this. Ultimately those pieces of data would likely be fields. If each field could be automatically wrapped in an individual component, then the entire system starts to fit together.

Building components

A component builder tool could provide some tools to select the data sources you’d need for a component. Perhaps from a content type, there would be an easy way to automatically create some components in a way similar to view modes. So if you went to an article content type, you could easily generate an article teaser component that would already have an article defined as its data source, and the fields for articles set up on the component. However, you could then select a different layout for a component, and place field components in each region, or decide not to place them all. You could still select formatters for each field. Maybe these default components are automatically created when you set up a new content type.

However, while that might be where you start, you could also connect up related data sources beyond just articles, like you can with Views and Relationships, through an entity reference field on the article for example. That would allow you to pull in the fields you need for a component, even if they are on a different entity type.

The component builder could also allow you to select a more specific design component layout, where each “region” stood for a particular variable used in the markup of that component. Those components might be somewhat less flexible for assemblers, but could serve front-end developers very well. Steve also discussed an interesting technique where multiple fields could be assigned to a variable region, but only the first one with results rendered, which served as an interesting way to provide fallbacks if there were multiple ways to get data for a particular component. In any case, the result would be templates with tight control over where data is placed in markup, as opposed to templates with more flexible regions that might contain multiple field components whose order could be changed around in the UI.

Some pieces of data might need to be assigned to the component as a whole: that data might not be automatically displayed, but could be used to construct attributes on the wrapper for a component, for example. That could be true for both small components or for larger template-level component layouts.

Component rendering

A model like this could greatly simplify page rendering. Rather than needing to ask every block on the site whether it should render for a particular route, you’d define at the top level which component blocks should be placed in various parts of the page. Ideally you might be able to set up some master layout templates that could be extended to child templates, so that some default component blocks could be placed on the master template and automatically show up on the child template. That would reduce the hassle of placing a header component on every page of the site.

Anyhow, the top-level template would identify all the child component sections, which could in turn define any child component sub-sections and/or individual components, which would consist of field components. At each level, only the necessary amount of data is passed from top to bottom, rather than passing around giant object and array structures that may or may not be needed. The data source for each component could still pass on any cache context information necessary for that component, so that we still would have the ability to cache individual sections of the page more aggressively than those which are updated more frequently.

Components and modules

Modules might end up defining default components that could be placed. Quick Edit could still pass in data through component attributes to allow for front-facing admin tools. How modules would need to work with components is definitely an area we could explore further. Hearing from back-end devs on typical tasks they need to do which affect the front end could be very useful.

In general, I think we should find ways to allow components to be created both from code and through the UI. Balancing those two methods could be tricky.

Implementing components with panels and blocks

If you’re thinking this looks a lot like the way Panels works, I’d agree. Panels offers the closest model for how something like this works. Mini-panels lines up very well with how components could be combined with each other into a component section. (Fun fact: mini panels can even contain other mini panels!) Panel pages are essentially templates for how to place various types of components on the page. Custom panel panes often line up very well with components.

Larry Garfield also thought blocks could form the basis for how components could work. Since Drupal 8’s panels is built on top of the block system, that sounds about right. I do think it might be wise to create a new thing called a component, simply because it might be easier to do so during the D8 cycle, when backwards compatibility cannot be broken. Using an existing concept like blocks could be risky if we find we need to make a change to improve the implementation, and that change would break BC.

Components and core

So while in theory it might be possible to do everything that’s been discussed so far once Panels is fully up and running in Drupal 8, I think it’s worth thinking about whether a model like this could be embedded in core in the work that leads up to Drupal 9. Solid UX work would be needed, along with renaming items to better match the sort of terms people tend to use when discussing components. I think ideally we could improve UX by reducing the number of different ways components can be created. A well-designed component builder could take some elements from views, block creation and entity display mode config pages. A list builder would probably take up other elements from views. This could be a spur to clean up our UX, use more familiar terms and in general make Drupal more approachable.

I believe this sort of approach would have some big wins by separating content structure from content presentation.

Two other items we discussed would also help support this work:

• A more object-oriented approach to rendering
• More robust JS in core

Object-oriented rendering

As much as Drupal 8 has moved to a more object-oriented back end, the render system still relies upon giant arrays with arbitrary array keys. That’s true for general render arrays and for the form API.

Using giant arrays for site data can be pretty problematic. Objects work a lot better with IDEs: it’s pretty hard for an IDE to provide code suggestions when array keys are so arbitrary. Mixing arrays with objects in Twig can also result in really awkward drillability with mixes of dots and hashes. Objects simplify unit testing, particularly with well-designed objects. Making the render system object-oriented might also make it easier to expose site data through REST. There’s little argument that one of the big goals for Drupal 9 should be converting our render system to objects.

If we’re looking to do this overhaul, we have an opportunity to make our render objects less tightly coupled with Drupal’s data model. In the discussion above, it might be possible to envision the render objects as a series of nested components, from large template level components to individual design components and even components for individual field formatters. I’m tempted to call these data components and data formatters to further separate how we structure content into fields to the various ways we present that data.

We discussed having one template per component as an initial goal: render objects might help with that. The individual field/data components on a design components might sometimes need a separate template. You wouldn’t necessarily want to recreate the template logic necessary to construct an image element in every single component. However, for some text data, being able to more easily drill into the field data and loop through that directly within the component could help reduce the number of templates. For field data that still requires separate templates, we discussed how the Template Mapper module allows you to rename templates from really long template names with suggestion modifiers into more approachable template names. We also liked how Fences allows you to outright remove field wrapper markup when it isn’t necessary. Using these approaches to keep all templates necessary for a component in the same folder could help minimize the hassle of managing multiple templates for a component.

Overall, we could have big reductions in render time with this new model. We’d have less need of passing around giant array structures, instead relying upon sending only necessary data objects from one component to the next, and only passing necessary variables from component objects to templates rather than huge array structures that can make debugging difficult. And again, we could still pass cache contexts from source data into render objects so that complex rendering strategies are still doable. With our library system, we can attach only the necessary CSS and JS to each component so that the final render has only the assets we need, an even more solid strategy once http/2 becomes prevalent.

If elements of our site are broken down into tightly defined components, what else might we do? Why, transform our admin system of course!

Improving site admin with targeted JS libraries

Another large part of our discussion revolved around what to do with the efforts to improve the JS in core.

There’s been a lot of talk about putting one of the currently popular JS frameworks into core: Angular, Ember and React have all been suggested. The two big reasons put forth for this: attract front-end developers who want to work with modern JS, and improve UX with optimistic feedback and fewer page refreshes.

There’s been a lot of consternation around that discussion, and I won’t go into all the details of that here. A few quick points on challenges with that proposal:

• Getting a JS framework into core could take multiple years, and given the pace of change within the JS world, any framework popular now will likely be outdated by the time we have integrated it with Drupal, thus negating any attraction that framework might have for outside developers. We’ve already seen that happen with Backbone.
• Rendering the same components on both the server and the client has become an increasingly important feature. However, the server-side rendering for Angular, Ember and React is geared towards NodeJS servers rather than PHP. Rendering server-side on PHP might in theory might be possible with a good amount of hijinks, but doing so when the community for these frameworks isn’t focused on PHP-based rendering might not be maintainable.
• Progressive decoupling looks at rendering part of the page on the server with traditional Drupal PHP and part of the page with a JS framework client side. While that reduces the footprint of how much of the page becomes decoupled from Drupal’s rendering, some chunks of the page still have all the disadvantages that decoupled sites do without any of the possible benefits that fully decoupled sites do, like server-side JS rendering. Having only certain sections of the page rendered through JS can make it challenging to make changes to the page as a whole.

We agreed on the importance of several key tasks that would be key for the success of a framework:

• Managing state. Avoiding multiple AJAX requests from crashing into each other and locking up the page. In other words, playing traffic cop and getting various bits of JS to play nice with each other.
• Updating the DOM quickly, at least more quickly than what jQuery typically offers.
• Changing the URL to account for changes to the state of the application to allow for easy bookmarking.
• Flexible templating that allows for shared Twig templates across server and client (e.g. Twig.js)

There was broad agreement that we don’t want to convert Drupal to render with Node JS. Drupal is a PHP-based content management system, so we should explore ways to improve our JS capabilities without going against that grain.

While existing JS frameworks might not be geared towards working on PHP, perhaps we can leverage more targeted JS libraries that combined together could handle the tasks we would ask of a monolithic framework. For example, we could look at using TwigJS to reuse our server-side templates for client-side rendering. Diff HTML might allow us to make virtual DOM updates. Other JS libraries like Vue and Riot were also mentioned.

Do we have a solution for which libraries should be used, and how or if they could be combined? No, not yet. However, we had broad agreement that such an approach should be further investigated.

If we find a solid combination of modern JS libraries that allow for better state control, optimistic feedback, and fewer page refreshes, we could provide better UX while also providing a better environment for JS developers.

An approach like this could also allow us to avoid any decoupling at all. We could maintain Drupal’s accessibility, multilingual capabilities and friendly Twig templating system. We still may find it useful to expose configuration entities through REST to power this system. That could end up empowering other admin solutions like a framework-based Electron app, for example. That’s great! We keep core PHP-friendly while allowing for great decoupled experiences too. Everybody wins!

Pulling this together to go outside-in

What’s more, when you pair this with component-based theming you can envision an even more outside-in approach to working with a Drupal site. Start by creating a page template, choose a layout, then begin assigning components to various regions. Need to combine multiple components together within a page section? No problem. Start building a new combo component without leaving the page thanks to more powerful JS. Add individual components to that combo. Either pre-existing components provided by a module, or by building a custom component. Select data sources, place individual data components into regions within the custom layout. Perhaps you could do that all with one continual workflow. If these individual parts of the admin side are themselves defined as components, it could be easier to pull forward related admin components where necessary without leaving the page, with state maintained during the entire process.

To me, this seems like a much more compelling direction to take Drupal’s front end than simply adding a JS framework. The fact that numerous people agreed on this approach—from OO back-end architects to Panels fans to JS devs to markup nerds—tells me we should at least explore this further.

As a front-end developer, I find this exciting for both Drupal’s back-end and its front-end. We can create a better admin experience, which rocks. We can also create a much better system for implementing robust site designs. The improved JS that would be geared towards the back end might end up being really useful as optional tools on the front end too and could help delivery really compelling user experiences without the overhead necessary for a decoupled front end.

Future possibilities

Imagine some of the other possibilities that could result from these changes:

• With true components in Drupal, we could create a list of components from small design components to component sections to full page component templates. These components could be further grouped together by category. If components could define default data to use if real data isn’t available, you could have… a pattern library. I love Pattern Lab, which provides a way to view component patterns either on their own or combined together. Getting it working together with Drupal without at least duplicating markup can be challenging. If you had real components in Drupal, though, you could display them in the UI with default data, add on the open source responsive toolbar Ish, and you have a handy way to preview your components and see how they behave at various viewport widths. That could mean a living style guide that uses Drupal’s real markup, CSS and JS: chasing that is one of the holy grails for front-end developers. How exciting would it be if Drupal could provide that out of the box?
• Such a pattern library could also provide a straightforward way for site builders to see what components they can use on their site. That has big benefits in reducing the number of one-off solutions used on a site by encouraging reuse of existing site patterns. Visualizing site components like this could be very appealing to site builders, designers and front-end developers alike.
• The approach described here matches up well with Atomic Design, an approach advocated by Brad Frost and used by Pattern Lab. The design components I describe above match up well with molecules, combo components with organisms and component page templates with… templates. Individual field data components most likely map to atoms, with a few more complicated data components fitting into the realm of molecules. In any case, this serves as an example of one prominent front-end architecture pattern matching up well with the component-based theming described here.
• Front-end development in general is moving towards the use of encapsulated components. The Web Components API, which scopes CSS, JS and markup within a custom HTML element, may well become a key tool for web development in the next few years. A component-based theme system would be well-positioned to make use of web components if they become the way of the future.
• Having large design components pass data to smaller components also lines up really well with the mental model used by frameworks like React. Using similar mental models makes it easier for outside JS developers to work in Drupal, and also makes it easier for Drupal developers to work with JS outside of Drupal. That’s very similar to the benefits of shifting Drupal’s mental model from primarily procedural code to primarily object oriented code. More tangibly, that shift in mental model might make it easier to reuse components for both server-side and client-side rendering.
• Conceptually we’re defining components as on one side a template with certain variables (or more flexibly regions), while on the other side certain pieces of data are being selected from related data sources that can be passed into those component variables or regions. That’s starting to sound like an area where GraphQL could come in very handy. GraphQL allows for more targeted data requests without creating a custom REST endpoint. Particularly with client-side rendering of components, GraphQL could be a key tool for making this system work well, solving a lot of performance problems from this separation of concerns for data structures vs how we present that information.

If Drupal 8 overhauled Drupal’s back-end, with some really nice boosts to our front-end with Twig, perhaps Drupal 9’s focus can be big leaps forward for UX with better JS and component-based theming.

While the initial impetus for this discussion came from some blue sky thinking before getting on a plane to MidCamp, it became clear from the discussion that many others have had similar thoughts over the years. The ideas might have been articulated in different ways, but this is not the first time ideas like this have been bandied about. This time hopefully we can make this a reality.

So let’s get talking! Do these approaches make sense? Are we missing important gaps in such a strategy? Are there particular JS libraries we should investigate for specific purposes? Let’s get the conversation started!