On this page

Share

Tags

License

This blog post is licensed Creative Common Attribution Required v4.0.

HTML imports are the best web component

3
Official Construct Team Post
Ashley's avatar
Ashley
  • 11 Jun, 2017
  • 2,585 words
  • ~10-17 mins
  • 8,135 visits
  • 1 favourites

We recently launched Construct 3, an incredibly ambitious port of an entire game development IDE to the browser. This involved porting a Windows desktop app with around 250,000 lines of C++ code to the web platform with hand-written Javascript using the latest ES2015+ features. There are many fascinating aspects of this project which will probably produce several blog posts, but the most interesting — and pressing — aspect is our use of Web Components, particularly HTML Imports.

Hopefully projects like Construct 3 are exactly what browser makers like Google are hoping to see: ambitious Progressive Web Apps (PWAs) that are fully-featured cross-platform replacements for their native counterparts. We've had great feedback on it too, with some users even saying they forget they're in a browser and not a native app. Ports like this still appear to be uncommon as the web platform is still developing and maturing the necessary APIs. Few other companies are willing to take on such a vast and potentially risky project, or if they do, they appear to downplay their PWA as a "Lite" version. We make no such apology; our PWA is the real deal. Therefore I think that our web development experience is particularly notable, especially since our approach to Web Components is pretty much the reverse of what everyone else does from what I gather reading around the web.

Web Components in an ambitious PWA

Web Components are made up of four key technologies: Custom Elements, HTML Templates, Shadow DOM and HTML Imports. Across thousands of lines of code, style and markup, Construct 3 makes minimal use of the first three. They just don't seem to be particularly important to this kind of app with our development approach. However Construct 3 is entirely architected around HTML Imports. They're the glue that holds the whole app together; we use around 300 separate HTML imports. Construct 3 is a huge app and imports work beautifully to break it down in to manageable components that are genuinely a joy to work with.

Guess which Web Component Google wants to remove?

That's right! HTML imports are down for possible removal, or later replacement by something different. We'll be fine — we've successfully polyfilled HTML imports, so even if they're removed, Construct 3 will continue on regardless. But like most polyfills it has some pitfalls, particularly for performance. I'm stunned that such a key part of the future web platform is being disregarded, but I can sort of understand how it's come to this. So part of the reason I'm writing this post is to try and push back against that.

What's so great about HTML imports?

Construct 3 is made up of over 1000 JavaScript files, nearly 200 CSS files (not including decorative themes — most of that is for functional layout), and as mentioned around 300 HTML imports with markup for various parts of the app. (Don't worry, our build system massively reduces this so end users don't need to make that many requests. It is however what we work with as developers.)

HTML Imports allow all three aspects of a web app to be componentised — markup, style and script. The alternative is either no components (just throw thousands of tags in to one HTML file!) or, in the near future, JavaScript Modules. JavaScript Modules make no effort whatsoever to componentise markup or style. They only deal with one out of the three aspects: script. So that doesn't help you organise hundreds of CSS files, or thousands of lines of markup. Hopefully you can see that neither of these options solves the problem of nicely architecting a huge web app.

The best demonstration of how beautifully HTML imports can work is when defining a dialog. Construct 3 has over 50 dialogs ranging from a simple OK dialog to a fully-featured image and animations editor implemented in a <dialog>. So this alone is a significant part of the product covering a whole spectrum of features. Let's look at our typical approach to developing a new dialog. We:

  1. Create a new import for it
  2. Write the dialog markup in the HTML
  3. Add dialog-specific styles to its own CSS file, and link to that from the import
  4. Add dialog-specific logic in its own JavaScript file, and reference that from the import

Here's a simplified version of what the import looks like: 

&lt!-- Dialog styles -->
<link rel="stylesheet" href="settingsDialog.css">

&lt!-- Dialog markup -->
<dialog id="settingsDialog">
	<dialog-caption>Settings</dialog-caption>
	
	<dialog-body>
		<label>
			<input id="option1" type="checkbox">
			Option 1
		</label>
		<label>
			<input id="option2" type="checkbox">
			Option 2
		</label>
	</dialog-body>
	
	<dialog-footer>
		<button class="okButton">OK</button>
	</dialog-footer>
</dialog>

&lt!-- Dialog logic -->
<script src="settingsDialog.js"></script>

This allows us to define a new piece of functionality in our web app in a simple, modular and isolated way. It even allows a sort of poor-man's scoping for CSS — just prefix every selector with #settingsDialog and you know it will only apply within that dialog. We also have a mini dialog framework that handles moving the <dialog> element to the main document, displaying it, running transitions, handling OK/Cancel and so on.

This approach is rolled out over the entire app. It's so effective, we use it everywhere. Each separate pane in the main view has its own import. The main menu has its own import. The account component lives in an import. Each kind of object in the game development IDE (which we call a Plugin) is defined in its own import. It covers everything, because it's so simple, intuitive and effective.

Surprisingly, it even covers pure-script components like the data model as well. HTML imports have built-in deduplication, preserve order of execution, can load asynchronously, and are easy to parse and concatenate in a build process. If we started today we'd probably choose JavaScript Modules for these components, but mainly just to avoid having to share top-level names in the global namespace. Other than that, HTML imports actually provide very good script componentisation too.

What's the alternative? Even if you use JavaScript Modules, where do you define a new dialog? Are you supposed to wedge it in the middle of thousands of lines of markup in a single HTML file that covers everything? Are you meant to have hundreds of link tags for CSS in your <head> that take forever to scroll through? Some languages let you wedge bits of markup inside JavaScript itself, such as with JSX, but I strongly dislike this development pattern. Such languages are non-standard, require an extra compile step, and moving large amounts of markup in to script bloats script files, slows down parsing and startup time, mixes instead of separates concerns, and is harder to write tooling for since you have a script-markup mish-mash. Shouldn't markup stay in HTML files?

What about other Web Components?

We make minimal use of all other components. Perhaps it might make our code a bit cleaner in some cases, or be more academically correct/modular, but we get by easily enough. Here's a quick run-down of what else we use:

  • Custom Elements: we don't use this beyond custom tag names, like <dialog-caption> in the previous example. I think our architecture is best described as Model-View-ViewModel (MVVM). So they are probably not particularly relevant to us since we tend to have JavaScript classes owning DOM elements. Custom Elements appears to use the reverse — DOM elements owning a JavaScript class. I guess both ways can work; MVVM works fine for us though.
  • HTML Templates: these are handy for stamping out a chunk of DOM repeatedly. I counted, and we use the template element exactly four times in our entire app. So a nice thing to have around, but hardly critical infrastructure in our case.
  • Shadow DOM: we don't use this at all. I'd guess this is much more applicable if you actually use Custom Elements, or if you're developing isolated components intended for third-party use in a library. We developed our own UI library because none existed that did what we needed (open Construct 3 and you should see what I mean), so this again this kind of isolation isn't particularly important since it's generally all our own code and markup. If we rewrote our whole UI library, I might experiment with this though.

Part of developing Construct 3 involved designing our own comprehensive UI library. This covers a windowing system (of which dialogs are a subset), tabs, toolbars, icons, menus, notifications and tips, tree controls, icon view controls, table controls, property grids and more. Custom Elements and Shadow DOM could potentially make these more modular, but we've come this far and it's worked fine, so these do not seem to be critical components for a large web app. That contrasts with HTML Imports, which are fundamental to the overall architecture.

I must add that I don't at all assume that everyone develops web apps like us. I am sure that for other kinds of app, these other web components will be critical. That's fine, and this is not meant to dismiss these as unnecessary technologies. My point is to emphasise that in at least some cases, HTML imports are by far the most important of the set.

What went wrong for HTML imports?

Basically, I think HTML imports are ahead of their time.

We started development of Construct 3 about three years ago. Before that we actually made a prototype even further back, using the traditional block layout, used jQuery, and so on. The prototype was so obviously going to be a huge mess with that approach that we decided we'd have to bet on all the modern web platform features for it to be feasible. Construct 3 uses to name but a few: CSS Grid, CSS variables (aka custom properties), CSS Containment (also critical for layout performance), the Dialog element, Service Worker, WebGL and WebGL 2, Web Animations, and more. Notice many of these features only recently became available even in just Chrome. In other words, web apps on the scale of Construct 3 have only just become feasible to release. HTML Imports were first released in Chrome 36 in mid-2014. That was just too soon for many web apps of Construct 3's scale to be around.

Meanwhile, Google appears to have been putting the most resources in to browser development of all the browser makers. Other browsers are generally playing catch-up, and have a wide array of APIs to consider implementing with limited resources. Understandably they are conservative. Ideally they want to see developer demand and widespread usage of an API so they know it's important it's implemented, but this creates something of a chicken-and-egg situation if they hesitate. Other vendors take a "wait-and-see" approach. In particular some browsers wanted to see how JavaScript Modules play out and how they will interoperate with imports — and Modules are still only just on the cusp of being supported. By now all this postponing from other browser makers has probably put off developers who want to see features with cross-browser support.

Finally, as outlined above, I think one of the best use cases for HTML imports is with dialogs. However the <dialog> element itself is still currently only supported in Chrome. So this feature which is a particularly compelling case for imports has no cross-browser support either. This probably also reduces the perceived utility of imports.

However it's now been so long, the Chrome team (and some other browser makers) appear to be taking the view that it's a failed feature. I hope this blog post helps counter that perception, and demonstrate the real utility of the feature.

Why not polyfill it?

We can, and do. However the main reason is performance. Browsers are good at parsing HTML and can start resource fetches ahead of the time they're actually needed. Consider this case: 

<link rel="import" href="sub-import.html">

<script src="script.js"></script>

Ideally we can fetch and even start parsing and pre-compiling script.js while sub-import is still fetching. This ensures maximum performance since as soon as sub-import.html finishes parsing, we can immediately execute a fully parsed copy of script.js. This feature is very difficult to polyfill. The script can be pre-fetched as text, but it has to use ugly blob URLs and still isn't parsed or compiled until it is added to the DOM. Experiments with link preload tags made things slightly worse, not better.

It seems only the browser has the power to control the precise scheduling of fetch, parse and execute for scripts. Our polyfill ends up having to wait until sub-import finishes loading in its entirety before even requesting script.js.

What other options are there?

I've heard of HTML Modules as a possible replacement for HTML Imports, but I can't find much information on what's different about them or how they'd work for a web developer. I do also worry a bit that it's just an exercise in tweaking and renaming it in order to present other browser vendors with something new to consider. In my view, HTML imports in their original form are already very effective.

One major concern I've seen is that JavaScript Modules and HTML Imports define two separate dependency systems. Ideally the web will only have one common module system. This is understandable. JavaScript Modules look like the more likely one to pick between the two, so perhaps it's worth considering implementing HTML Imports in terms of JavaScript Modules. I imagine it working like this. Suppose you can import HTML from JavaScript, like a new kind of module: 

import doc from "./import.html"

This would be the same as fetching "import.html" as type "document", and assigning the resulting Document to doc. Now you can access the import document from the script, and perform typical calls like doc.getElementById(...). This actually looks to me like a pretty elegant way to pull in DOM content to a module script. It's also straightforward to statically analyse.

Going one step further though, we can actually re-define <link rel="import"> in terms of JS modules. We could say that this: 

<link rel="import" href="import.html">

is equivalent to this: 

<script type="module">
import doc from "./import.html"
</script>

In this case 'doc' is not used, but it causes the fetch and propagates through the import's own dependencies. Now your dependencies can also propagate through HTML — but it's entirely based on the JS module system. That means only one module system is used, only one set of deduplication applies, and so on. Hopefully this is an idea worth consideration from browser vendors.

Conclusion

HTML imports are a compelling way to structure modern web apps. They arrived too early for their own good, but that doesn't make them less useful. I say this not as a casual developer who has played around with some code demos, but as the lead developer of a commercial PWA that was successfully ported to the browser from a large native Windows app. There is considerable real-world development experience behind this view. I would encourage other developers to experiment with using HTML imports to architect large web apps, and I hope browser vendors can appreciate the great utility of them. If they must be removed or replaced, I hope my suggestion of implementing them in terms of JavaScript modules is useful. If they are completely removed with no replacement, we'll get by fine with our polyfill — but I'd feel that we missed a great opportunity for the future of web development.

You can also find our polyfill for HTML Imports on GitHub, which is robust enough to work with Construct 3.

Subscribe

Get emailed when there are new posts!