Delicious Reverie

2018 Review / 2019 Aims

Thu, 03 Jan 2019 00:00:00 GMT

Twenty Eighteen was a whirlwind of changes for me professionally. I went from working in an agency to working for a FinTech, and the difference in those two roles and the tech stacks involved have brought me many new skills as a developer. But I still suffer anxiety with all the things I want to learn.

What I Learned #

1. Moved to JavaScript #

I've already blogged about this before, but I moved from a PHP / JS role to a fullstack JS role. That was great, but it was very hard initially. I had to put in many hours outside of work to bring myself up to speed on certain core JS concepts I'd missed before, as well as learn some of the key React APIs. But I went from not knowing how to pass props down to rebuilding a fairly complex calculator. That felt like a big achievement.

2. Learned The Value of QA Engineers #

The QA process in my current workplace is much more rigid. Every ticket is defined, has parameters, and is approved / rejected by a QA engineer before it goes into production. For someone who is obsessive about quality, and takes it hard when I get things wrong, having a QA Engineer has meant very few issues are raised once the code is in production. Those I answer to in the organisation have much more confidence in the development process in its entirety because of this.

3. Learned about CI Tools #

I'd used Netlify to build my static sites before, but in this new role I was able to get visibility of the pipelines that were set up by our SRE (site reliability engineering) team that built the site in a Docker container, and also ran a feature CI on our PRs. This way, we could see straight away whether code was good enough for production. I'm still fascinated by this complex world and want to learn more.

4. Learned the Value of a Good Code review #

I've had some very smart people review my code, and I've learned a lot. For example, why destructuring is a good thing, why you wouldn't use the index as the key value in a React map function, and loads more.

5. Learned how to Use GitHub Properly #

I made a grand mistake of merging to master at my new job ... but I only made it once. I broke loads of stuff and caused a world of pain for my team. Now even with personal projects it goes against the grain to commit anything to Master, even if it's a tiny change.

6. Learned the value of JIRA #

I used to believe JIRA was this unwieldy tool that only crackpots would use. But it's so useful. We have strict definitions of where the responsibility lies for each step of the process, we can't release code into production unless is the process has been properly followed, and we also have an "expedite" lane so developers can focus exactly on what they need to do. Honestly it's transformed the way I work and made shipping code so much faster.

What I Want to Learn #

TypeScript #

I had never seen the reason for type checking before, but now realise that a lot of the issues I've been having all along with JavaScript can be resolved by using TypeScript. The verbosity puts me off, but I can't wait to TypeScript all my projects.

Unit & Integration Testing #

I've never written tests before, but I want to get to the stage where I could say I would be happy to undertake a role that involved Test-Driven Development.

Docker #

Still don't understand that thing. I want to learn this dark art so that I can set up feature CI tools and production pipelines myself.

More Logic Based Development #

I enjoy being the Fronted Developer, but I'd also like to build a few microservices and become more comfortable with the thought process of building for stuff that isn't going to be displayed in a browser.

Advocate for Younger Developers #

As I see how skewed the whole development industry has become, I want to use my privilege to help others from different backgrounds to get into writing code."

2019 Review / 2020 Aims

Fri, 11 Jan 2019 00:00:00 GMT

I opened last year's post by stating what a rollercoaster it had been for me. Well, this year has been no different; I started out the year becoming a father for the third time. That has been a wonderful, exhausting experience!

Professionally too, my team went from a band of three (a developer, QA and project manager), to one of five to seven people, including another full-time developer, my personal friend and former colleague David.

David and the other members of the team have taught me a great deal. Here are some of the main learns I think I've achieved:

What I think I achieved last year #

1. TypeScript #

TypeScript was going to be one of the biggest accelarators for me in terms of the quality of my output, and I have really enjoyed using it. I started out by implementing interfaces and ended up configuring my environments with much more type safe settings by using noImplicitAny and similar.

2. Testing #

Not just unit and integration, but I've also started to plan and develop end-to-end tests as well. I've become a lot more aware of the value of tools like Cypress and Percy, and have even started to write some components using test-driven development.

3. Docker, Kubernetes, CI tools #

I hoped to continue my journey into DevOps / Site Reliability Engineering by learning these tools, but I've hit a bit of a philisophical roadblock.

I think I know enough about Docker to use it fairly well, and I'm getting to grips with the basics of Kubernetes and CI tools (I've been involved in writing build scripts for Jenkins and CircleCI).

4. Do more than UI Development #

I started the year by saying I wanted to do more "logic-based development", however I now think those words were ill chosen. The logic of what you're building is so tied to the requirements of what you're working with that applying it to a UI or an API (or something of that nature) is equivalent.

I've been able to improve my coding chops this year so that JavaScript no longer daunts or scares me. I would no longer feel inadequate if I looked up on MDN even simple functions like map or reduce in order to achieve the task at hand.

I still consider myself a Frontend Developer, but I now think that's more to do with circumstances than it is ability.

What I Want to Achieve This Year #

Only this week I was screensharing with our tech lead, watching him whizz around Kubernetes pods like that scene in Valerian, killing and restarting pods, chatting with the SRE team, sshing into a container and using a bash script to figure out what was going on there.

I'd love to have the confidence to do that. So that's the two things I'm going to add first:

1. Bash scripting #

It's becoming clear to me there's a huge benefit to learning this. If I'd known Bash, I would be able to customise my environments far beyond what I can achieve currently.

2. Continue getting to grips with Kubernetes #

Kubernetes is an awesome tool, and has so many applications outside web development. I'd love to at least be able to understand it's lifecycle methods and helper functions well enough to see what's going on in my stack and fix minor issues.

3. Serverless! #

Serverless really makes sense to me. That separation of concerns with the frontend, backend and satellite services, lambda functions, and other functionality, just seems more natural and less of a cognitive overload.

I'd love to think that by the end of 2020 I will be able to write a Terraform script and get a system comprised of a few key pieces not only up and running but fully deployed, monitored and testable too.

4. Writing #

One thing that has surprised me is that I really love writing about web development. This way I can help others by utilising another skill I think I have: being able to write moderately well.

My relationship with Net magazine is very strong, and I'd love to see if I can write a bit more of the kinds of articles that have been well received by their audience so far. This might be more difficult to achieve though because of the lack of personal time I currently have."

2020 Review / 2021 Aims

Fri, 04 Dec 2020 00:00:00 GMT

2020 has been ... a lot. But I've also learned a ton of stuff. Though I can't ever say I know enough of anything, my objectives for 2021 are going to be quite different from previous years...

I've definitely not been the smartest person in the room over the last year. I've worked with some incredible people, such as David Hewitt, David Fernandez, Romina Moya, Kate Beavis, Gerard Blanes, and many other great folk who have been good enough to explain stuff, pair code with me, and bear with my failures.

What I set out to achieve this year #

1. Bash scripting #

Erm, didn't really get to this a huge amount. Though I'm less concerned with remembering the syntax of specific languages these days (more about that later...)

2. Continue getting to grips with Kubernetes #

Oh wow ... I can't believe I added this lol. I spent only as much time with K8s this year as I had to. I know it's got it's value but the point of an abstraction is generally so you don't have to worry about it unless something goes wrong. Nothing has gone horribly wrong with our Kubernetes instances this year.

3. Serverless! #

Serverless still really makes sense to me. I spend a bit of time during furlough following along with Seed's excellent tutorial at serverless-stack, and messing around with the serverless application framework Webiny, which were great eye openers.

"I'd love to think that by the end of 2020 I will be able to write a Terraform script and get a system comprised of a few key pieces not only up and running but fully deployed, monitored and testable too."

Ha ha ... nope. Not that bothered by Terraform any more. It's a good enterprise system that I'd gladly pick up if my job required it ... but if not, I'm sticking with the Serverless framework. As with Kubernetes, it has more abstractions but less to have to worry about yourself.

4. Writing #

Ah you know, that didn't go at all the way I'd hoped! Netmag folded just when I'd finished writing my first feature article for them. How sad for everyone who worked there. I'm still scoping out how I might be able to continue writing, but haven't figured anything out yet.

What I Want to Achieve This Year #

As I mentioned near the start of this post, my journey this year has involved less about deepening my understanding of tools, and more about the coreskills of a software engineer. So that's what I'm aiming for primarily this year.

Advocacy and mentoring #

I already really enjoy advocating for others, so my role leading Zopa's Frontend Guild will continue. I hope that I can chair the group fairly and focus on the needs of our most junior and least outspoken members.

Next year will also bring a new opportunity to help someone grow their career, and I relish the opportunity to be involved in that.

Follow through #

One piece of feedback I received this year was that I should pursue issues that arise with third parties and other internal teams, instead of waiting for others to ask me what the next steps could be.

If there's an issue I should pick it up and progress it as far as I can on my own, not leaning on others in the team to do so. Although, keeping them informed about what's happening and allowing them visibility over key events and decisions is a necessity.

Stand up for whats right #

During a few exercised I wasn't able to explain clearly why my preferred approach should be adopted, over another approach that was favoured by someone who wasn't as experienced as I.

I found that I struggled to make a case why it was better. If I'd been more insistent I think I would have been able to politely convince the other person why my approach was the best.

I still need to introspect about this a little, hopefully I can find out why I do that!

Not giving up #

I recently paired with David on a project where we were both unfamiliar with the codebase. What impressed me is that David didn't give up, he kept pursuing different avenues to figure out what the code did and how it worked.

This persistence ultimately led to us refactoring the code to achieve our objective instead of reaching out to colleagues who would have been able to help.

The tricks he used to progress through the code were ones I use often, but on an unfamiliar codebase I would more than likely reach out to others earlier instead of applying those skills in this setting. I'm going to try to change that perspective and persist with puzzles that I'm not familiar with.

Conclusion #

A lot of these items are here because I want to become a Senior developer, and a lot of that isn't about code but about other skills which you need to exercise.

There's a lot of churn in our industry, and I'm still really keen to learn things like Svelte, spend more time with Webiny, and more with fullstack apps (instead of static frontends). However that's proved to be really difficult due to the time my children need from me at this stage in their lives."

2021 Review / 2022 Aims

Wed, 29 Dec 2021 00:00:00 GMT

2021 continued the trend of being more challenging than any period before it, in terms of maintaining some sort of normality to our lives. However there have been some really good things too. Here's a review of what I set out to achieve and what I want to achieve this year

When 2021 dawned I was still a Mid Level Software Engineer at Zopa. That changed in April as I took a new role as Senior JavaScript Engineer at Purple Bricks. It was a great boost for my self confidence and a fantastic team to work with. But 2022 finds me at another turning point: going from that to a split between DevRel and Fullstack Engineer at Webiny. With that change comes some reassessment of where I want my career to go in the next year.

What I wanted to achieve #

I set out to improve how I was doing in these following areas:

Advocacy and mentoring #

At the start of the year I was trying to build better bridges between disparate teams at Zopa. I enjoyed that a lot. My desire to do more of that got a massive boost when I joined Purple Bricks as one of their 3 Seniors.

It was fantastic to not only mentor various members of the team, but also help set the direction of the whole chapter. I also enjoyed delivering a training course to the business on GraphQL, and building what was for me the most complex system I've built: a messaging app.

But what I remember most fondly was the moment I told my colleague Helen that this was my first Senior role. "Wh ... what?!" was her response! I've been keen to demonstrate that I don't always have the answers (or even the best answers), and that I often don't know what I'm doing, and I think that has actually boosted the confidence others have in me, rather than the opposite.

Follow through #

Yep, I definitely have had to give this some attention. But as most of the follow through has been on making sure more junior teams aren't still stuck after I've left them with a solution. I think I've got better at this, at least I hope I have!

Not giving up #

I'm still bad at this! I'm more persistent definitely, and I'm a lot more comfortable at diving through code I don't understand and finding ways around problems. But I also realise my personal style is a lot different to my friend David's, and I've become more comfortable with my own aptitudes.

Aside from these, I've really got used to digging into pipeline problems, environment issues, using Kubernetes and Helm, as well as getting better at Javascript in general, routing, graphql, and different rendering options.

What I want to achieve this year #

There wasn't much technological focus in last years' ambitions, but I know for certain that there are going to be some strong tech challenges this year!

Fullstack #

I've been working a lot with NextJS applications at Purple Bricks. That has helped me build a better mental model about static, server and client rendering, and the benefits and hazards of each. Also, I have come to really enjoy the flexibility of NextJS with each of these methods.

But I still have been predominantly working on the client. I'm really looking forward to working with lambdas at Webiny, and I hope it'll really help me build my understanding of server side.

Serverless #

Now that I understand the principles and benefits of fullstack applications, I can't wait to discover more about serverless, and how it solves some of these problems. I anticipate it'll have it's own set of caveats too, and I can't wait to understand these more. I'm also looking forward to understanding more about AWS service and APIs.

State management #

One of Webiny's applications is a page builder, with a lot of heavy state management using Recoil. It's going to be really great to get more into a state-heavy application and learn more about that world.

Content creation #

As of the time of writing, I have never made a TikTok video. I think this is about to change with my taking on a part DevRel role at Webiny. That seems really scary to me at the minute since I know nothing about video editing let alone how to tell a story in less than 1 minute.

Community building #

This is the one I'm really looking forward to. Webiny has so much potential. I can't wait to see where the team take this product. There's already a small community with some really great contributors and users, I'm hoping that I can help blow that up to a much larger audience in the next year."

2023 Site Rebuild

Wed, 27 Sep 2023 00:00:00 GMT

Sorry to those people who subscribe to my RSS feed, I had a bug which I couldn't fix before I re-launched deliciousreverie.co.uk, but which I've resolved now.

Yes, I've rebuilt my site again. This is perhaps the fifth iteration of it ... let me see, yes:

Perch
Hugo
Gatsby
Astro (Nx monorepo, Webiny CMS backend)
Astro (Markdown)

The main driver this time is that I broke my production instance of Webiny CMS by trying to deploy a bunch of upgrades which I should have made gradually.

But I also was struggling to manage the complexity of the CMS plus the monorepo in my spare time. It's no joke when you've got 3 kids.

Also, for all the goodness that Nx brings to a project, I was held back a little; I wanted to adopt Astro 3.0 early because of the view transitions API, but I couldn't because the Nx plugin for that was pinned to version 2.

This time, the site is super simple. And it also means that I'm ditching the plan I had to keep building blogging sites. Instead I'm going to let the domains lapse or leave them as they are.

Again it's the time thing, but also as I've moved into a new role I've been playing with some more serious JavaScript stuff and want to focus on doing some more experimentation with the things I've learned, as well as starting up another class to teach kids how to code.

I've got a few articles planned, including a write up of how I made a media server, and another one about web workers and cache storage APIs.

Hopefully I'll get some time to write those soon!

20MinJS: Serverless JavaScript: What, Where and Why

Tue, 20 Sep 2022 00:00:00 GMT

I appeared on the podcast 20 Min JS with Fernando Doglio to talk about Serverless as a concept, and how developers can maximize their time and ship deliverables faster by adopting serverless."

4 things that I learned from counselling training

Sat, 02 Jan 2021 00:00:00 GMT

Most of us have moments in life where we question ourselves, often forced into being by some personal crisis or difficult situation. During one of these I enrolled on a course to learn Counselling, a form of Cognitive Behavioural therapy that is popular in the NHS and privately in the UK. I studied for about two years, and afterwards took a course in abnormal psychology. Even though I never qualified to become a counsellor, it's one skill I use often, even in web development.

Here are some of the main takeaways I gleaned from my course:

1. Drink more water #

It amazed me how much of our psychology changes with our eating and drinking habits. I'm not talking about drugs or stimulants here: my tutor noted that we often compensate for tiredness by drinking coffee, however a good glass of water would often have a much bigger impact.

Similarly with getting sufficient sleep: if we get 7.5 hours or less, our thinking is impaired. The less we sleep, the more irrational our thinking becomes. I realized the truth of this statement most clearly one occasion when I hadn't slept for three nights, drove my car, and crashed it into a truck.

If you're having an issue with some code, take a break. Have a nap. Drink some more water. It amazes me at how often this helps me to solve coding problems.

2. Everyone has different perspectives #

You're on the way to a close friend or family members' wedding, driving down the motorway, and running late. You see a puppy on the side of the road, in distress, clearly malnourished.

What would you do?

Some people would drive past: taking care of the dog would make them late for the wedding; something they've been planning for a while. Others wouldn't hesitate to stop to help the stranded animal. In fact, they might only attend the wedding once the dog has been safely cared for, no matter the time that might take.

Everyone has a reason for what they do; and these perspectives often drive our decisions. The person who stopped to pick up the dog wasn't being inconsiderate to their friend who was getting married; they simply judged that another had a greater need at that moment.

Conversely, the person who ignored the dog wasn't being heartless: they merely have different drivers for their decisions.

Some people have a preference to write with the right-hand, others the left. We're all inherently have these preferences for our personalities too. We can train ourselves another way, like a left-handed person can train themselves to write with the right hand, but our overriding preference doesn't really go away.

This taught me that we should respect others' choices, and stop judging them, they have good reasons for making their decisions. They are also their decisions, not ours.

We should take into consideration these perspectives when we're writing software. We should also be writing software with other people - not necessarily developers, but other people invested in the outcome. Because if you're writing software in isolation, how can you take into consideration other people's ways of thinking, or even be aware of things like their skin colour or facial features? What about their physical or cognitive disabilities?

3. Psychology theory is just theory #

One of the most famous psychologists, Sigmund Freud, was obsessed by evolution, and therefore the base instincts a person has. This led to some pretty interesting ideas about the activity of the subconscious, but also some pretty horrific viewpoints on racism.

Carl Jung was a natural successor, and I often think of him as ... kinder ... to human nature. He's the one who came up with cognitive behavioural theory (CBT), that a person can change if they have enough awareness and motivation.

Isabel Briggs-Myers and her mother Katherine Cook-Briggs subsequently came up with the Myers-Briggs Type Indicator (MBTI), which suggests there are four differentiators (like a preference for writing with the left-hand or right) that combine to form a persons' personality. They tested loads of people on this, and quite often the results were a remarkably good fit. Their ideas form the basis for a lot of personality tests you see today. However, because nobody knows what is really going on in our heads, it's still just a theory. We can't say it'll fit every individual. For example, by suggesting a person has certain personality traits, are they more prone to adapting to show those traits? Or to not show them over time? I've taken the same MBTI test with a 10 year gap. My personality type the second time was significantly different to the first.

As developers, we can be tempted to strictly categorise and compartmentalise everything - specifically people, but other things too. But these will ultimately break down and become less relevant. Sometimes you just have to go with what works.

4. Just listen #

Most of the time, the most powerful thing you can do to help someone is to listen to them. As much as it gets repeated on social media, memes and other places, actually, really, attentively listening to someone is a skill we often lack.

The main thing that gets in the way is ego: we strongly desire to share our experiences, our solutions, to validate ourselves, or to compete with another person.

But once we let that go, instead focusing on the needs of the other person, we put ourselves in a place to really help them.

What I mean by really listening is active listening: hearing what they say and then validating it, either by reflecting it back to them (using different words to say what they just said), or asking pertinent questions. If you didn't understand what someone said, don't be afraid to ask them, "what did you mean by that?", or check your understanding by reflecting back and welcoming their explanations.

This can help us in many aspects of web development: at the planning stage when a feature is being explained to us, when listening to feedback on a feature we've just written, or when doing user interaction research with customers.

How does any of this help me in my web development career? #

Some people take up software engineering to get away from people. However, if you're tempted to do that I think you'd probably be disappointed. We primarily engineer software to help people, and people are intrinsically involved in deciding what we build, and how we build it."

2016 Review / 2017 Aims

Fri, 12 May 2017 00:00:00 GMT

2016 as a year was as unconventional as they come. Globally there have been some massive shifts politically, socially and in other ways. My life too has taken some pretty interesting turns. I'm following suit here by posting a quick review of my year and what I hope I can achieve in 2017 from a professional perspective.

When I look back at the start of 2016 it is with a huge amount of mixed feelings. Hannah and I were still getting used to the sleepless nights with Morgan, who was just then a year old. We'd also recently got told to move out of our house with 4 weeks notice. That proved to be incredibly stressful.

The Summit Media project #

From a professional perspective, I was doing some of the best work of my career for Ech Design on the Summit website, a monster Wordpress build with multiple custom post types, and some pretty involved animations. The deployment process was also a bit of a minefield for a team that had only just started using git.

That build nearly killed the 4 of us, especially Neil and Phil, the project leads, but it's something we are all incredibly proud of even today. It was nice to have the confidence and gratitude of the Summit team after a very intense 4 / 5 month design and build period.

It was also my first introduction to using a slew of new technologies, including Composer, Atlassian JIRA, a range of new animation techniques and git branching & deployment strategies.

Moving on from Ech #

Until late in October I was still contracting at Ech, a great team with some exciting ambitions. I'm looking forward to hearing what they achieve in the future. I was sad to say farewell after almost a year there to go and work at an agency, a move which was conducive to a better work / life balance (because it's a much shorter commute).

Looking back, I certainly was pushed to move my skills forwards, at times much further than I believed I could. I also was able to leave Ech having imparted to them some useful processes that I had acquired already (kanban boards for project management, git for version control, and working locally instead of via ftp).

I also moved to an agency for the chance to work with the current team and their excellent in-house Wordpress theme. This theme has many fantastic features, including the fact that it uses OO PHP and has command-line interface similar to Laravel's Artisan tool for making meta fields, widgets and custom settings area.

$ origin make:metabox mymetabox

The one thing that really got to me about it's nearest competitor, Elliot Condon's Advanced Custom Fields, is that you can create everything in the theme in code. This speeds up my development time exponentially, to the point that I can create an entire theme in as few as 3 days.

Aims for 2017 #

A few things I had in my goals list which are underway already include the following:

1. Build a home web server #

After a few false starts (I initially used Debian, but struggled without the support as a beginner at this), my home server is up and running . The main reasons for wanting to do this is because we currently maxed out on iCloud storage and don't want to pay increasing amounts ad infinitum to store & share our stuff.

It was also a good task to help me discover more about servers and the LAMP stack in general, which will help me understand my role and how it connects to other roles in the workplace.

2. Get to grips with SVGs and SVG animations #

I really believe SVGs and animations are the future of the web. There's an insane amount going on with these at the moment, and I've built sites using some rudimentary animations. Unfortunately, none of them have gone live yet. I would really like to showcase some of what I've done and find out new ways of using animations to tell stories on websites.

3. Graduate to Laravel #

I've already had the opportunity at an agency to start learning Laravel, and already love it. I think this could quickly develop into a standalone blog post (umm, yep, it probably will) so I won't say anything more about it, other than I know I'm going to love the framework. A colleague has already told me "learning laravel will make you a better Wordpress developer". I can already see the truth of that statement.

4. Design More #

This one is a bit more of a vague idea than a concrete goal:- I only designed one website in 2016 and think I'm losing my sensitivity to design a bit. I want to get that back a little so that I can continue to break down the siloed approach to building websites which is sadly so prevalent still."

6 reasons why we chose Nx as our monorepo management tool

Thu, 23 Sep 2021 00:00:00 GMT

Sharing code across multiple disparate projects, at scale, can be a problem for many organisations.

The challenge is to eliminate code duplication and unify user interface design patterns, whilst at the same time providing (hopefully) speedier development for engineers.

There are a few ways to tackle this issue. In 2017, we settled on a monorepo pattern that facilitated Yarn workspaces. We had hoped this would provide the benefits above, but in reality we found that not a lot of code sharing was happening, especially across our frontend projects.

This was because each project would have its own package.json, its own unique pipeline, and, due to its own custom React implementation, it’s own UI library.

So, in 2019, we decided to rethink the problems we were having. We took a look at the tools available and found that a relatively new tool, Nx from Nrwl, already had support for some of our common tools.

We started a new monorepo using this tool that we hoped would eventually facilitate all of our initial requirements.

First of all, what problems is Nx solving for us?

What problems does Nx Solve?

Nx is a smart and extensible build framework to help you architect, test, and build at any scale

Nx is a modular build framework for architecting and maintaining code projects. Using it means you configure once, then just focus on build your tools and apps. Nx can effectively manage the configuration around APIs, micro frontends and libraries of tools, so you don’t have to consciously think of this step each time it comes to building a new project.

It facilitates a consistency of output that is difficult to achieve otherwise. With Nx, we have gained the following benefits:

1. Standardised libraries

Each of our microfrontends use the same version of NextJS, React, TypeScript (with one established ruleset), Jest, and Cypress. Each of our UI libraries utilises the same Storybook.

This means that instead of having to make decisions about what tools you’re going to use, Nx already provides them for you. As well as avoiding possibly lengthy and heated discussions about which tools developers might want to use, there’s already a standard in place for others to follow.

The other benefit of standardising libraries like this is that it’s much easier for developers to dip into different apps and libraries, because they’re all much the same.

2. Code sharing

Having a monorepo also helps with onboarding. As soon as a new developer pulls down the repo, they have access to a wealth of information about our coding standards and they can immediately start to learn our coding style and what our approach to certain problems might be.

New developers also have better discoverability around existing libraries that we have built, so it’s easier for them to utilise these already built tools instead of having to ask whether something they need already exists in another repo somewhere.

3. Code standards

Do you want to use spaces or tabs? Should we enable noImplicitAny? Should we use rem, em or px? None of these discussions happen, or they happen less often, because we can enforce the same code standard across all of our projects, and automatically alert developers and block PRs that don’t follow these standards.

It makes life a lot easier.

4. Deployments

We can also standardise deployments using Nx. We have one single pipeline for all of our apps, which means we have less overhead in maintaining that pipeline.

When deploying, we check which applications have beenaffected by the code changes being made by utilising the command nx affected. This allows us to run the test suites for these applications. If all of these pass, we know we have a valid build and can have relative confidence that releasing code that alters several applications at once won’t result in any breaking changes.

5. Versioning

Working at previous organisations, I have encountered issues with different versions of libraries being used by different applications within the organisation’s overall codebase. This means that the UI can be significantly different across different areas of the business.

This can also result in deprecated functions still being used months or years after they have been retired.

With Nx, all of our apps consume a single version of a library, and that’s updated every time there’s a merge to master.

This means there’s less risk of unexpected breaking changes, and no sprints wasted bumping library versions: it’s all done once, at the same time the library is modified. This could also help changes to the API of a library to be more robust, because it’s being consumed immediately, and not after some time has already passed and other teams have begun to implement the change.

6. Development Experience

In previous iterations of our development workspaces, it could take up to 3 days to deploy a new application. With Nx, it can take a few hours. All you need is to run one command:

nx g @nrwl/next:app becky-quotes && yarn generate

We have configured this to build a new application that has helm charts for the deployment, as well as a NextJS application and a Cypress e2e test suite.

This also helps with onboarding as well as assists us with automated dependency management.

We use Renovate to raise nightly PRs to update dependencies. We have a requirement of 100% unit test code coverage, which is instrumental in ensuring the changes this automated system makes don’t break anything.

These benefits have clearly saved us a lot of time and we’re extremely happy with the results. But it hasn’t all been plain sailing. Aside from all the benefits, let’s look at one of the main challenges we’ve faced…

Poor library strategy

Here’s our dependency graph. As you can see, all of the apps depend on a single library: our shared UI components.

This means that when we update something in this library, or when writing a new component here, Nx sees that all of these apps have the dependency of that single library, and it initiates tests for all of those applications.

We have 14 currently, so this task can take a great deal of time. This is especially true in our pipelines, which as a consequence sometimes experience timeout issues.

Our solution to this is to have much smaller libraries. Instead of one UI library, we are going to have one library per component. This, at first, might seem counterintuitive, but it means that changing components, and creating new ones, can be done much more quickly. All of the components can still be consumed with a single Storybook instance. And some of the libraries will of course consume some of the other libraries (the base theme for example).

Conclusion

Nx has been a terrific investment for Purplebricks. It hasn’t all been straightforward, and we have faced challenges, but I would definitely recommend Nx as a great way of managing projects — as it makes the jobs we do far easier by comparison to the alternatives available."

2022 Website Rebuild

Fri, 25 Nov 2022 00:00:00 GMT

Around a year ago I rebuilt this website using Eleventy, the static site generator. I did this because my existing site was using Gatsby but was [using a plugin which removed the JavaScript in a way that caused errors on the site](https://www.gatsbyjs.com/plugins/gatsby-plugin-no-javascript/). I didn't want to put the JavaScript-heavy build back, but the broken links were causing errors in the console and I knew it wasn't the best approach to a JavaScript-less Gatsby site, but at the time there was no alternative.

Eleventy is quite a unique tool. Whilst I can see it's appeal for some, I found the way it consumes data and then injects it back in at build time a bit of a black box. At the time I rebuilt it, I was hoping to use the revolutionary plugin Slinkity to provide client-side JavaScript so I build build back features such as animations and search. However, by the time I launched my site, the author of Slinkity had moved to Astro, and development had slowed.

Rebuilding in Astro

I had determined that I would rebuild the site a second time in Astro, but it was early days for that tool as well, so I've sat on it for the time being. However, in recent weeks I've really needed to make some serious changes to the site so I could showcase some of my recent work at Webiny, and Astro had reached v1, so it seemed a good time to switch.

Therefore, the frontend is built in Astro. As yet there's no JavaScript, but I'm planning to use Svelte because of it's developer experience and unique approach to providing interactivity. However, I could very easily add components that are built in other frontend frameworks. This agnostic use of frontend rendering libraries is a powerful enabler, and one I want to leverage.

Styling with Tailwind (and others)

I've used Tailwind with DaisyUI for the most part because of the way it compiles away unused CSS. At most I've used 4 Tailwind classes on an element. My reasoning for this was that if I'm using a lot of classes, I'm probably doing Tailwind wrong. I've got more to say on this subject so I'll save it for a separate article. But suffice to say for now that using Tailwind with DaisyUI is a great combination because you have access to a library without being tied to semantic elements and which are also tree-shakeable. DaisyUI is a Tailwind plugin, so elements are only compiled into the CSS bundle when you use them, same as Tailwind.

I haven't exclusively used Tailwind and Daisy, because I think every system is at some point going to fall short of what you need; in my case it was the ability to underline links that are inline in the content.

Big shoutout to Salma (@whitep4nth3r) for "roasting" my site on her Twitch stream which identified loads of layout, typography and spacing issues.

Webiny for the Content Store

I'm pleased to use Webiny to store content. I manually moved all of my articles over to my new Webiny instance which is deployed on AWS using Webiny's Pulumi integration. Webiny is an eminently hackable CMS which is why I like it. Because it uses EditorJS in the backend, I was able to customize it by adding a plugin for code blocks. Unfortunately there seems to be a problem using some of the other plugins (embeds and inline code blocks) which I want to investigate.

It was tricky to figure out how to render this content in Astro. Rather than using Prism, I opted for Shiki because it's native to Astro and would mean adding one less dependency.

Actually I really like Shiki's API:

import shiki from 'shiki'

const { code } = Astro.props

let codeOutput

await shiki.getHighlighter({
  theme: 'monokai',
}).then(highlighter => {
  codeOutput = highlighter.codeToHtml(code, { lang: 'jsx' })
})

This is using Astro's top-level await. Unfortunately the EditorJS plugin doesn't allow you to store a language by default. This is something I might try to change soon.

Although Webiny's editing interface isn't as nice to use as others, it's a great tool that can scale with the content I produce. Since all of the blogs I look after only have around 100-200 records, I'm planning to use one Webiny instance for all of them. I think that will ultimately mean I need to migrate from the current implementation (DynamoDB only) to one which uses ElasticSearch to index and retrieve records. I also may need to pay for a license to use the multisite feature, but I'll worry about that later.

Also, I've been very lazy about retrieving records in my frontends. Instead of utilizing the built-in cursor based pagination from the GraphQL API, I've just increased the limit of retrieved articles to 200 and grabbed all of them in one API call. I realize I shouldn't really do this so I've added something to the codebase that will case builds to fail if there are over 200 records.

I have found that writing in Markdown isn't fun anymore either. I'm hoping using this interface will therefore encourage me to write more often.

Code Management

One of my objectives with this rebuild was to consolidate all of my content-heavy websites. I manage and write content for https://freebabylon5.com and https://www.discovermikeoldfield.info, and I want to build some more sites soon. However, each of these sites is built on different technologies. I was a hoping that I could consolidate all of these into one monorepo using Nx so I could re-use the setup and different shared elements, like the header, footer and rich text renderer. I followed this tutorial to set up Netlify deploys from the monorepo, but as yet I haven't tested it with a seconds site, so I'm not sure how that might go.

This means there will be one monorepo for frontend code, one for the backend code, and I can iterate on the two as I go along. This setup is pleasing from a code re-use perspective as well as saving me time that I can spend with my family, whilst at the same time allowing me to enjoy my hobby of writing.

The future

I'm hoping to add some features in the near future

Search with Lunr.JS
Add back the wistful animations
Inline embeds and Code blocks
Fetching data from GitHub
Fetching and rendering the latest video from my YouTube channel
Rendering previous & next links on articles
Using a self-hosted alternative to Calendly
Using self-hosted analytics

In addition to this, as I mentioned, I want to scale out this stack to provide unique experiences for each of my websites.

Also I've just discovered that creating a new build on Netlify is cached by Nx, so when I want to publish a new article (which doesn't involve any code changes), I need to alter the build settings to include the skip Nx Cache command. that's something I need to dig into as well as setting up a CI-CD pipeline for the backend.

Both the frontend and the backend repos are public, so feel free to follow along as I add more features.

Another shoutout goes to Swapnil (@swapnilmmane) for waking me up by saying "I don’t know what your site is about, it’s a black box" which prompted me to redesign my home page to have more focus on what I do."

A Journey With Dylan: Halfway Through

Tue, 06 Dec 2016 00:00:00 GMT

A Few weeks ago, I wrote a post about how I was taking a journey through the studio albums of Bob Dylan, starting with his first eponymous album, and discovering new albums and tracks along the way which resonated with me. I'm now about half way through the catalogue, and have encountered a few surprises.

(Here's the previous post, in case you haven't read it.)

I wrote a few weeks ago that Dylan is an artist that demands constant re-assessment from his fans. And indeed, some of the assumptions and opinions which I had, and which I'd heard, about his back catalogue have proved ... interesting, if not completely false.

1. Dylan's Early Work is Hard Going #

I had already struggled with Dylan's earliest work. I can appreciate that he was an artist absorbing, emulating and building on those around him at the time, but I really didn't identify at all with Bob Dylan, The Times They Are A-Changin' and only just got through Another Side . Maybe I have been tainted a bit by listening to a lot of what came afterwards, but I just can't see the appeal.

The Freewheeling' remains my favourite album of this period, and my favourite track of that album has become my favourite duet of all time: "Girl from the North Country" with Johnny Cash is an absolute gem of a record. Cash's perfectly toned pathos is a great contrast to Dylan's lighter, more introverted approach.

2. Self Portrait Is a Masterpiece #

From what I'd heard, the world was aghast at what Dylan had released in 1970. Even the artist himself claimed that the album was "something of a joke". So I took the CD out of it's case ... and didn't listen to it for a few weeks, fearing the worst.

Although I've discovered a distaste for Dylan covering other artists' work (I hated Dylan — and don't get me started on the Sinatra albums) there are plenty of absolute diamonds in there. I could listen to "Copper Kettle" and "Early Morning Rain" for hours.

Taking a rounded out view of the first half of this journey, I'd have to add "Nashville Skyline" as one of my top albums.

Oh, and listening to John Wesley Harding was going well, until "All Along the Watchtower" started playing, and for the first time I got the context of that song. Whoa.. that changes things. Just having that track amongst some instrumental movie soundtrack music makes it stand out as genius these 50 years later."

A Journey with Dylan

Fri, 02 Dec 2016 00:00:00 GMT

I've been in & out of love with Bob Dylan for many years. I started listening in my early 20's, bought a compilation album, but I have to say that I just didn't get it. Now, 16 years later, I'm listening to every album he's produced so far.

I guess I might be a glutton for punishment, some of the albums aren't that easy to listen to. But I want to just chart this journey because Dylan is someone who demands constant reassessment. Every time you think you've got the guy sussed, he does something you don't expect.

My favourite albums up to this point are:

Time Out of Mind
The Freewheelin'
Love and Theft
Blood on the Tracks
Desire

Time out of Mind is because that's the album that got me into Dylan initially; when I heard it I was in a very sultry space and every track suited that atmosphere like it was made for me.

The Freewheelin' is full of uniqueness and perspective that you just don't expect from someone that young, especially being his second album.

Love and Theft was the third album I bought and I really didn't expect it's timelessness, antiquity and dry humour. I also love the F Scott Fitzgerald quotes peppered throughout the album.

Blood on the Tracks. Such raw loss gets my attention every time. This is a very personal album from Dylan, and it's isolation and pain comes through loud and clear. And then there's "Lily, Rosemary and the Jack of Hearts" which is a blindside.

The duets with Emmylou Harris on Desire work so well, but for me this album is more about atmosphere. I love the Southern American feel to what Dylan is doing here. "One More Cup of Coffee" is a great track, even though I'm not a fan of the Spaghetti Western, I can't help being drawn into that atmosphere listening to this track."

Alliteration in Poetry

Fri, 15 Mar 2024 00:00:00 GMT

I've slowly begun to realise something about my taste in poetry, and I wonder how much this is because of my Welsh heritage. As a result, I'm diving more into traditional Welsh poetry, and really enjoying what I find.

The very fact that poetry is somewhat inaccessible drew me to it. I'm one of those oddballs who revels in the company of unusual people, and who seeks out friendships with people who others find difficult to understand. So poetry was a good candidate for me from the start.

Due largely to F Scott Fitzgerald (who I got into very young), and another writer I liked (science fiction author Dan Simmons), I got heavily into Keats, and through him John Milton, Byron and to a much lesser extent Tennyson and Wordsworth.

The descriptions in the works of these poets is delicious, even sumptuous. Keats especially wants us to swoon and even expire because of the beauty of the physical sights around him. And the way he conveys the power of what he sees is amazingly beautiful.

But I've recently discovered other authors that for me at least equal if not surpass Keats in my personal taste. And this is because of the words they write, not the scenes they paint.

Philip Nikolayev

I have no idea how I stumbled on the Moldovan / Russian / English poet Philip Nikolayev. I know I joined a Facebook group and that was where I first found his translations of Osip Mandelshtam's own poems from Russian.

But I loved his keen sense of alliteration. Within each line he places such delightfully similar-sounding words that for me began to exceed even the profound pleasure I got from the deeply meaningful poems I'd read so far.

<div class="[&>pre]:ml-0! [&>pre]:mr-0! [&>pre]:bg-transparent!">

Insomnia, Homer, taunt sails: my lips have lisped
Down to the middle the detailed list of ships
That long brood and angular train of cranes
That rose above Hellas once on wings of waves.

</div>

In the first line, the alliteration is in the sound at the start of "lips" and "lisped", and is carried on in the second line's "list" and "ships". It's very playful.

I don't know how much of this poem has been written by Nikolayev, but I imagine it's only the ideas that have come from Osip Mandelshtam: each language has it's own forms of poetry that are I think unique to that language, and you can't translate them directly.

Which is now why I lament that there isn't more of Raymond Garlick's poems online that I can share with you.

Raymond Garlick and Anglo-Welsh poetry

It was quite by chance that my wife gifted me a volume of Raymond Garlick's poems last year. But during his lifetime last century he was regarded as one of the principal Anglo-Welsh poets.

That's a bit of an odd term. Let me explain what is meant by "Anglo-Welsh". When you hear that expression, it means poetry and literature that's written in English by a poet that lives in Wales or who has Welsh cultural influences.

His poems didn't so much paint pictures of the landscapes of South Wales that he was familiar with as make you think about his personal connection with them. And he used alliteration extensively to convey that.

<div class="[&>pre]:ml-0! [&>pre]:mr-0! [&>pre]:bg-transparent!">

To you I give a golden garden, Wales
To play in: half a hundred voels and vales
Shall be your toys, and you shall sing and laugh 
amid a noise of flights of nightingales

</div>

To me this poem sounds very rich (as in chocolate cake) because of the frequency of alliterations, "golden garden" and "half a hundred", "Wales", "voels" and "vales" ... he didn't happen upon these as if they were coincidences: alliteration is woven in the very fabric of his verse.

This brings me to the most well known Anglo-Welsh poets: Dylan Thomas. Which is funny, because I used to really hate Dylan Thomas.

Dylan Thomas

When I was younger and listened to recordings of Dylan Thomas reciting his own poetry, I was completely unimpressed. As well as verging on the nonsensical (this was "Under Milk Wood" instead of the far more poignant "Fern Gully") it sounded like he was in some sort of trance, as if words were tumbling out of him in what I considered to be a monotonous tone.

I didn't appreciate that he was tapping into many hundreds of years of Welsh tradition. And Raymond Garlick was my doorway to understanding how that works.

Welsh Medieval Poetry

The forms poetry takes is unique to each language it's spoken in. To translate a poem is to take it's ideas and try to convey them in another language; or else try to convey the way it sounds. One or the other must be sacrificed.

But let me try to help you understand why that isn't always necessary:

<div class="[&>pre]:ml-0! [&>pre]:mr-0! [&>pre]:bg-transparent!">

Gwae'r gwan dan oedran nid edrych, ni chwardd,
ni cherdda led y rhych 
Gwae ni wyl yn gynilwych
Gwae ni chlyw organ a chlych

</div>

This is a form of medieval Welsh poetry called "cynghanedd sain", and to me it seems remarkably similar to the alliteration (a term we use for poetry written in English) of Raymond Garlick's poems and the cascading rhythym of Dylan Thomas's as well. The words tumble and fall, trip, skip and scatter in interesting ways across the page.

I've been reading about this effect in an article by Eurys Rowlands in the 4-volume "A guide to welsh literature". In that essay the author notes that English poetry is created to be more appealing to the eye. You see that in a lot of Romantic poetry: they're describing a scene or event before them, much like painting with words.

Welsh poetry is entirely different: it's written to appeal to the ears. I can't imagine you'd ever get awful things like trying to rhyme "eye" with "symmetry" just because it looks right. Apologies William Blake but I just can't forgive that one.

So you see, I don't need to translate it for you to be able to enjoy it. As long as you enjoy the way it sounds, that is the real power of the verse. It's meaning can be an enhancement.

And for me, that means I have found an interesting way for me to learn the language, and that's something I've really struggled with over the past few months: I don't easily succumb to things like "gamification" so Duolingo isn't a good fit, and I don't have time for a regular scheduled course.

However, I do have time to translate the odd bit of poetry. I make the time, because it's something I really enjoy.

Alternatives to Heroku’s Free Tier

Fri, 26 Aug 2022 00:00:00 GMT

Heroku announced yesterday that it's extremely popular free tier plans are being phased out starting on November 28 this year (2022). It's the end of an era for a lot of people, particularly independent bloggers who have hobby sites, developers evaluating software for use at their organizations, and non-profits who host their Headless CMS backend. But is there an alternative to Heroku?

The Heroku Story

Heroku started as a platform offering hosting to exclusively Ruby-based websites. It's since expanded that option and now a host of "build packs" for other languages exist which can deploy infrastructure to Heroku in one click. This was likely the inspiration for similar "one click deploy" buttons that have been adopted by Netlify, Vercel and others.

The unique thing about the way Heroku was built is that it leveraged AWS' ability to hibernate EC2 instances. So when you deploy your infrastructure to one of the free-tier "dynos" (a dyno is an environment in which your application can run, similar to containers), after a while of inactivity the dyno would hibernate: it would turn off, or go to sleep. The effect of this was that you couldn't contact the application straight away in that state. It would take around 30 seconds for it to activate, or wake up. This was enough time for Heroku to spin up your dyno again so that it can start accepting requests.

This method did come with some complications like ephermeral file storage, meaning you couldn't store things like SQLLite databases or upload images alongside your application. But there were other Heroku services and 3rd party integrations to allow you to use databases and store images and other files.

This was a great help to many people who's main concern was with small blogs built on an open source blogging platform such as Ghost or Strapi. With Heroku you could deploy your application and build a separate, static frontend (which would be hosted on a separate service, such as Netlify or Vercel). That way, you could write an article, rebuild your static site, and afterwards allow the application to hibernate. And it would be free to use.

Alternatives to Heroku

Looking for an alternative to Heroku depends on what you were using for. Many people had used it to host backend services for small hobby apps that didn't make any money, or for proof-of-concept ideas so they could evaluate tools for their organizations to later use in production.

If this is what you're looking for, there are a plethora of services which deploy containerized applications to a managed environment and that offer free tiers with different pricing models depending on what databases persistent storage and other services you require.

Depending on what you were using, it's feasible that you could move your application stack there without too many code changes. In fact, both Fly and Render offer guides for migrating from Heroku:

Speed up a Heroku App with Fly.io
Migrate from Heroku to Render

But some other popular alternatives include

Linode Managed Hosting
Digital Ocean Managed Cloud Hosting
Firebase Build

You could also get quite far by using the serverless functions that now come with (formerly) static hosting providers like Vercel, Netlify, and Gatsby Cloud. There are even complex frameworks like RedwoodJs that have been built on top of these providers.

If you were using Heroku for a Headless CMS for a static frontend, most of the open-source Headless CMS platforms offer their own cloud-based infrastructure which will avoid having to re-platform to a different CMS altogether.

There is however potentially another problem looming...

Stay in Control of Your Application Stack

If Heroku can suddenly turn around and kill it's free-for-starters offering, who's to say it won't happen for these other services? Even Vercel and Netlify leverage the raw computing power of cloud providers such as AWS, Azure and Google Cloud, repackaging those services and selling them at a profitable margin.

For that reason, we think the best alternative is to embrace these foundational computing platforms, instead of standing again on the unsteady ground of a reseller.

AWS and other providers have a free tier which applies to most of its services. These are specific allowances for each of their services that you can use before you start incurring fees.

With this and similar offerings from other cloud providers like Google Cloud Platform and Microsoft Azure, you could set up an EC2 instance (similar to a virtual machine), configure it to hibernate in a similar fashion to what Heroku was doing, and move your application to it manually.

This would take a bit of work, either to SSH in and upload your application, or set up a deployment pipeline.

But there's also another opportunity here that we would like to highlight.

Embrace Serverless

If you notice, the limits on EC2 instances for containerized applications are time-based. You get a certain number of free hours in traditional "always-on" computing models. After that you need to pay for them. Even though this is quite a generous offering, you could still leave yourself with a substantial bill if you're not careful.

However, other services are based on the number of invocations. Why not take advantage of these?

Functions as a service with AWS Lambda: 1 million free requests per month
File storage You get 5GB of S3 static file storage (think images and static website hosting)
AWS' no-sql Database, DynamoDB, comes at 25GB of storage for free

The next time you need to build a proof of concept, or hobby app, you could build your solution using a combination of these and host it indefinitely. It wouldn't need special configuration to hibernate since that is it's natural state.

Why not try serverless in your next proof of concept or hobby project?

To do so, you could leverage a framework which allows you to write infrastructure-as-code, such as AWS Amplify, Serverless Stack, the Serverless framework, or Pulumi. There is a bit of an overhead to learning these tools so you will need to research which one is best for your needs.

But there's an alternative...

Webiny: Leverage the Power of Serverless

For a little over two years, we have been building Webiny. It's a Headless CMS you can use for your personal blog ... but it's not only a Headless CMS: it's a fully-featured application framework that you can use to scaffold unique applications that will have access to the existing setup we have built via plugins.

You don't need to reinvent the wheel of how to store files, or how to build a GraphQL API, or how to build an admin interface because we've already done that for you.

And what is more, you get all of that for free when you choose the DynamoDB-only option in your setup. You will eventually pay for the amount of requests, storage and database usage ... but only if you exceed the free tier on AWS.

Once your proof-of-concept has satisfied it's requirements, you can destroy your Webiny instance ... or leave it there for future tinkering!

And if you want a place to write blog articles, Webiny is built to do that out-of-the-box."

Automated accessibility testing is great, but ...

Wed, 20 Jan 2021 00:00:00 GMT

We recently had a situation using jest-axe in tests, where the landmarks feature wasn't work as expected. This helped me verify why we shouldn't rely solely on automated tools, never mind how good they are.

In unit tests for our react component library at Zopa, we have utilised jest-axe to test whether our components adhere to a number of accessibility rules.

The funny thing was, when we adhered to the rules (specifically the way components either use semantic HTML or aria roles), further testing in Google's Lighthouse tool revealed that there were accessibility issues with our components:

Some ARIA child roles must be contained by specific parent roles to properly perform their intended accessibility functions.

The trouble is that our library contains a lot of components that do not require the use of landmarks. We build our UI pages up from these component pieces, and would utilise landmarks on the page typically.

This rule was forcing our developers to add aria roles where they weren't needed.

Thankfully, You can disable this rule. However, this means that there are far less checks for semantic HTML in our codebase.

I don't think this kind of issue is something that's easily solved by the axe team. It does highlight the flaws in tools like jest-axe: we can't rely solely on them. In fact, right on the Github repo the team identify clearly (in large writing!) that " This project does not guarantee what you build is accessible", and that "The GDS Accessibility team found that only ~30% of issues are found by automated testing."

We ultimately decided that ~30% is better than nothing. And <div> soup, whilst undesirable, is better than using roles where they're misleading. So we disabled this rule and we're going to continue to use jest-axe in our project.

Other ways forward #

To try to keep our focus on giving all of our users the best experience possible, we have recently created the role of "feature champions" among the frontend community.

These individuals take on a subject that they are passionate about, such as TypeScript, unit testing and, yes, accessibility too, and will occasionally give a quick eye to projects to kindly advise whether we are following best practices."

Book Review: The City & The City, China Mieville

Wed, 15 Jun 2016 00:00:00 GMT

Time is precious for me having a kid and a demanding job, so I'm pleased I've managed to find the time to read The City & The City, China Mieville's (very successful) attempt at the crime thriller genre.

I'm not typically a reader of crime fiction, however I've really enjoyed some of Mieville's urban fantasy work in the past, and know he's a smart, well-researched writer who is prone to a bit of genre-hopping ... his plots often take unexpected twists, for which this work is no exception.

The City #

Melville, writing as he often does from the first person perspective, describes an inspector sent to investigate a grimy murder in Beszel, a city "on the outskirts of Europe", which we hear is jostling for economic position with it's more successful neighbour, Ul Quoma.

But it's pretty soon we realise that this other city isn't in a separate location from Beszel, but in the same place. It's quite difficult to describe, but you get parts of the topography which are located in Beszel, but some are in Ul Quoma. So you could live "gross topically" (a word coined to describe a physical location which nevertheless is in a different political locality — that of the other city) next to someone who lives in a totally different city. A city for which, if you wanted to visit them, you needed to go to a consulate and obtain a visa. This is complicated by parts of the territory that are "crosshatched", or located in both cities, but through which you have to navigate residents of the other cities ("foreigners" to you) without interacting, or even noticing them.

Make sense? No? Stop there, go get the book, because it does get weirder.

Borders #

The borders are policed by a seemingly omnipotent force which is above the law called Breach. If you, even inadvertently, step over in to the other city, or speak to a "foreigner", they will immediately arrest you. You might not ever come back once they do.

When our detective finds out that the murder may have been committed in Ul Quoma, the foreign city, he needs to cross over to uncover what happened to her, and in the process discovers that something more sinister is happening.

Conclusion #

I really enjoyed this book, but always feared it would step too far away from the crime genre to be accessible. However, it doesn't do that. Even though the plot takes some pretty intense turns that leave you reeling, its still a crime thriller through and through."

Atomic Development

Thu, 04 Oct 2018 00:00:00 GMT

Structuring components in a reusable, discoverable way has been a challenge for some recent projects I've worked on. I've seen different approaches used, but now I think I've found a method that's basically intuitive, avoids duplication of effort and facilitates easy discovery of components.

I didn't come up with this approach. Brad Frost published the book "Atomic Design" 2 years ago, and I was fascinated with his idea, that you break down your components to the smallest reusable unit, then build them up again piece by piece. Just as living organisms are made up of atoms, then molecules, then larger entities until you get the complete creature.

I tried this approach on a recent React project, and want to outline some of it's benefits. It's important to note that I don't believe it's 100% the best approach, since I don't think there is one. What I do want to show is where it's useful, where it isn't, and hopefully what those limitations are.

The Atomic Design Approach #

Brad Frost's book outlined a way to structure the design of a project based on building things up from the smallest component part.

For example, this could be an input field. Input fields are never used on their own, so the next item up the scale would be an input field and a label. This would be your atom and molecule respectively.

You could keep building upwards in this way so that you eventually had a fully designed page using a template which contained your complete form (an organism) which is made up of your separately designed molecules.

This approach could help designers think more about the patterns they are building into a project, instead of treating every item as separate compositions, which would then require a lot more work from the development team and time and money from the project owner.

But I believe we can utilise this approach in development so that there's more cross-functional collaboration between design and development. And it can also help developers structure their projects well.

Atomic Development #

On a recent project I decided to investigate whether this approach could be uilised in development. In my React project I typically use Styled Components to write my CSS styles. I therefore structured my project like this:

- components/
  - Atoms
  - Molecules/
  - Organisms/
  - Templates/
  - Pages/

Atoms #

Atoms I designated as constants which could receive props. For example, a paragraph that had a prop for the text colour.

export const Paragraph = styled.p`
    ${props =>
      props.textColor
        ? `color: ${props.textColor};`
        : null }
  font-size: ${variable.REGULAR};
  margin-bottom: ${variable.SINGLE};
  & small {
    color: ${variable.BRAND_HILIGHT};
  }
  > a {
    ${props =>
      props.textColor
        ? `color: ${props.textColor};`
        : null }
    text-decoration: underline;
    &:hover,
    &:active,
    &:focus {
      color: ${variable.BRAND_HILIGHT};
    }
  }
`

I kept all of these in a single file because I initially believe there wouldn't be too many small components, and they were so tiny they didn't justify having a separate file for each.

I now believe that approach wasn't the best one. As I got further involved in development, and I had built up a larger library of components, I struggled to remember what I had designated as an Atom, and what was a Molecule. In future, I'm going to separate these components into individual files to facilitate easier discovery.

Molecules #

Molecules consist of stateless functional components, often comprising of several Atomic elements and usually some custom Atoms that relate to that specific Molecule.

This is a "popout" section, which has a custom MainContainer element that's only used in this Molecule. It can accept multiple props and can wrap other components.

const MainContainer = styled.div`
  width: 100%;
  background-color: ${props => props.backgroundColor ? props.backgroundColor : variable.BRAND_PRIMARY };
  height: 100%;
  padding: ${variable.SINGLEplusHALF};
  text-align:center;
  color: ${props => props.textColor ? props.textColor : variable.BRAND_HILIGHT };
  border-radius:${props => props.corner ? props.corner : variable.RADIUS_CORNER };
  position: relative;
  background-size:cover;
  background-position: 50% 50%;

  @media (min-width: ${variable.BREAK_TABLET}) {
    padding: ${props => props.NarrowView ? `${variable.QUAD} ${variable.SEXTUPLE};` : `${variable.QUAD};` };
  }
`


const PopOut = props => (
  <Container>
    <MainContainer {...props} style={ props.background ? {backgroundImage : `url(${props.background})`}: null } >
    {props.children}
    </MainContainer>
  </Container>
)

export default PopOut

Organisms #

As we get to organisms, we start to see Classes being used, bringing in several Molecules and atoms, often with components that render on the frontend as well as server.

This is a carousel component. It had several smaller components and so many unique styles that they were collocated within a folder.

class CardsCarousel extends React.Component {
  constructor(props) {
    super(props)
  }

  render() {
    return (
      <Main
        style={
          this.props.backgroundimg
            ? {
                backgroundImage: `url(${this.props.backgroundimg})`,
                backgroundSize: `cover`,
              }
            : null
        }
      >
        <Container {...this.props}>
          <MainTextContainer>
            <HeadingSecondary textColor={this.props.textColor}>
              {this.props.heading || 'Recent News'}
            </HeadingSecondary>
            <Paragraph textColor={this.props.textColor}>
              {this.props.content || 'Content.'}
            </Paragraph>
            <Button to="/news/" white>
              Read more articles
            </Button>
          </MainTextContainer>
          <CardsContainer>
              <Carousel lightBubbles {...CAROUSEL_SETTINGS_LARGE}>
                {this.props.posts
                  ? this.props.posts.map(item => (
                      <Card key={item.node.id} data={item.node} />
                    ))
                  : null}
              </Carousel>
          </CardsContainer>
        </Container>
      </Main>
    )
  }
}

export default CardsCarousel

Partials #

As I developed this project, I realised there was an extra item which I didn't feel fitted into either Organisms or Templates. These were the site's headers, footer, and a parent component called Layout which would wrap all of the pages and provide some base CSS (fonts and a minimal reset).

So I created a Partials folder for these 3 components. I'm still not certain they shouldn't have been Organisms. But easy discovery for me meant that I wanted to keep them separate from other, re-usable Organisms that would be used in different contexts.

Templates #

Does what it says on the tin really. These are where the layout for pages that were programmatically created from other data sources are kept.

Pages #

Fully formed JSX pages lived here, comprising of a component that would wrap every page (Partials/Layout.js), and many Organisms, Molecules and Atoms to build up entire pages.

Here's a short example of a page so you can see the final use of this approach:

import React from 'react'
import Helmet from 'react-helmet'
import * as variable from '../components/variables'
import Layout from '../components/Templates/Layout'
import Link from '../components/Molecules/Link'
import * as atom from '../components/Atoms'

import Masthead from '../components/Organisms/Masthead'

export const frontmatter = {
  ---
title: "NOT FOUND",
  path: "404",
  description: "Page not found",
  inMainNav: false
}

const NotFoundPage = props => (
  <Layout>
    <Helmet>
      <title>NOT FOUND | { props.data.site.siteMetadata.title }</title>
      <meta name="description" content="Page not found." />
    </Helmet>
    <Masthead
        title="NOT FOUND"
        intro="Page not found"
        textColor={variable.BRAND_SECONDARY}
    />
    <atom.Band>
      <atom.Container>
        <atom.Paragraph>You just hit a route that doesn&#39;t exist... the sadness.</atom.Paragraph>
        <atom.Paragraph>You might want to check out the <Link to="/sitemap">sitemap</Link>, or go back to the <Link to="/"> homepage</Link>.</atom.Paragraph>
      </atom.Container>
    </atom.Band>
  </Layout>
)

export default NotFoundPage

export const query = graphql`
query notFoundPageQuery {
  site {
    siteMetadata {
      title
    }
  }
}
`

As you can see from this code, an approach I started to adopt included importing everything from atoms and using <atom.band> to identify clearly what grouping it came under. It also made imports much shorter and somewhat tidier.

Atomic Could Be the Future #

The title of this section is a bit of a misnomer. The Atomic approach might not be best for your project. It might not fit your development team workflow. Or it might not match your particular mental model of how you see your project's code. That's OK, I document this in the hopes that it might be useful to some.

If you do end up adopting this approach I'd really like to know about it."

Better Type Safety with JSDoc

Thu, 18 Jul 2024 00:00:00 GMT

This is a follow-up article to one I wrote a few months ago, and which quickly became the most popular article on my blog. I've learned a few more tricks, including how to ensure that types in events are respected.

Here's a link to the first article

One thing I struggled with when I started using JSDoc was how to type event handlers without TypeScript.

Take a look at the following code:

const button = document.querySelector<HTMLButtonElement>("button#my-button");

button.addEventListener("click", (event) => {
  const toggleThis = event.target.closest(".toggle-container"); // TypeError: closest does not exist on type EventTarget
  sendToApi(JSON.stringify(event.detail)); // TypeError: detail does not exist on Element
});

With TypeScript, you can type the selector or event directly:

const button = document.querySelector<HTMLButtonElement>("button#my-button");

button.addEventListener("click", ((
  // you don't need both of these, this is just an example
  event: MouseEvent
  ) => {
  const toggleThis = event.target.closest(".toggle-container");
  sendToApi(JSON.stringify(event.detail));
}));

We have to think a little bit differently with JSDoc. However, we can have more type safety with JSDoc:

const button = document.querySelector("button#my-button");

if(!button || button instanceof HMTLButtonElement) {
  return;
}

button.addEventListener("click", (event) => {
  const toggleThis = event.target.closest(".toggle-container");
  sendToApi(JSON.stringify(event.detail));
})

Why do I say more type safety?

Because TypeScript is compiled away at run time. It doesn't make it into code that runs in a browser, because it's not valid JavaScript.

What would happen if the button wasn't in the DOM? Where's your static analysis now?

This is something that's very easy to miss if you're writing TypeScript because (a) you can easily force the compiler to recognise the button is there (via a dirty ! after the selector), resulting in (b) a possible error in production if the button isn't there for some reason (for example, some code that produces it has failed, some developer in another squad removed it without telling you, or somebody replaced it with a <span>).

It could be relatively easy to get into one of these situations ... JSDoc ensures that type safety is baked into your production environments, not just your development code.

I've been using mostly JSDoc for the last 18 months, and I'm definitely preferring it to TypeScript. I would lean on TS for React still but other than that it's very robust. It's really sad that there isn't more documentation for it, and I think that causes a lot of people to steer towards TypeScript.

But if you have the luxury to choose, definitely consider using JSDoc.

Borges and I

Sat, 30 Sep 2017 00:00:00 GMT

As a teenager, and on into my twenties, I developed a real affinity for existential philosophy, and really enjoyed reading the works of Kafka, Alain Robbe-Gruillet, Satre, Camus, Samuel Beckett and others. Though these writings aren't what you might call "easy reading", they appealed to this somewhat narcissistic, isolated youth who already felt as if he didn't really belong anywhere.

One author that I never encountered as part of that group was the Argentinian writer and translator Jorge Luis Borges, and it wasn't until I stumbled across a collection of his work that I even heard of him.

Borges the Translator #

Borges was a translator into Argentinian of popular Western works. I get the sense that this work must not have challenged him very much, since he didn't exactly stick to the material he had been given. I am told that he sometimes introduces new text, plot lines and even entirely new characters.

I know what you're thinking, "that is a betrayal of what it means to translate" ... and perhaps you would be right.

But perhaps there's another view.

I've often thought that the work of a translator is an act of co-creation, not merely one of replacing one word for another. What got me thinking along these lines is when I heard that people signing for the deaf are called "interpreters", not "translators", since they are interpreting words into something else - thoughts or phrases that can be understood in a different medium.

Was Borges trying to force this point? Or was he just being facetious, taking liberties because he knew his audience would be none the wiser. I guess we may never know.

Borges the Writer #

When he came into his own as a writer, Borges' tales can verge on the kind of creepiness HP Lovecraft fans would adore; tales of vast, hidden cultures with special knowledge in dense jungle lands, of unintelligible worlds existing just beyond our own.

As an example of this type of story, "The Garden of Forking Paths", takes a somewhat paranoid narrator trying to escape a determined killer by finding some hidden garden through which he can escape, only to find that the killer knows the maze better than he knows it himself, and so the narrator cannot escape the inevitable.

But what I really love is Borges' ability to get me to question my own world view: more often than not, his mostly short, single page or two-chapter stories end with a plot twist sure to cut you to the quick; quite often playing tricks with the role of narrator to spin your interpretation of the story completely on its heels.

My absolute favourite of these stories is "Borges and I", the last story in the compendium I own. I'll reproduce the entire delicious text here by way of a conclusion.

I argue that Borges should not so readily be ignored, and be much less eagerly forgotten as he seems to be today. Instead I think he should be considered as among the core group of existentialist writers that I mentioned at the start of this article. I'd love to see more people recognise this talented, mischevious and thought-provoking artist for what he was, a true original among his contemporary 20th century authors.

Borges and I #

It’s to that other one, to Borges, that things happen. I walk through Buenos Aires and I pause, one could say mechanically, to gaze at a vestibule’s arch and its inner door; of Borges I receive news in the mail and I see his name in a list of professors or in some biographical dictionary. I like hourglasses, maps, eighteenth-century typefaces, etymologies, the taste of coffee and the prose of Stevenson; the other shares these preferences, but in a vain kind of way that turns them into an actor’s attributes. It would be an exaggeration to claim that our relationship is hostile; I live, I let myself live so that Borges may write his literature, and this literature justifies me. It poses no great difficulty for me to admit that he has put together some decent passages, yet these passages cannot save me, perhaps because whatsoever is good does not belong to anyone, not even to the other, but to language and tradition. In any case, I am destined to lose all that I am, definitively, and only fleeting moments of myself will be able to live on in the other. Little by little, I continue ceding to him everything, even though I am aware of his perverse tendency to falsify and magnify.

Spinoza understood that all things strive to persevere being; the stone wishes to be eternally a stone and the tiger a tiger. I will endure in Borges, not in myself (if it is that I am someone), but I recognise myself less in his books than in those of many others, or in the well-worn strum of a guitar. Years ago I tried to free myself from him by moving on from the mythologies of the slums to games with time and infinity, but those games are now Borges’ and I will have to conceive of other things. Thus my life is a running away and I lose everything and everything is turned over to oblivion, or to the other.

I do not know which of the two is writing this piece."

Both Sides, Now

Tue, 12 Mar 2019 00:00:00 GMT

CSS isn't a programming language. CSS is a programming language. JavaScript shouldn't be used to render HTML. JavaScript is the only way of writing HTML. CSS is overly complicated, difficult to learn and hopelessly outdated. All of these opinions provoke a response in me. It's just perhaps not the one the authors intended.

When I see comments like the ones above I can't help but wonder if they're written out of a genuine frustration or concern a person has, whether they're meant to provoke an argument, or even whether they are using others' frustration to raise their own profile.

Whatever the case, there's another way of looking at the situation that I hope others can benefit from:

Both Sides, Now #

I was surprised on checking Rolling Stone Magazine's top 500 songs that Joni Mitchell's song "Both Sides, Now" wasn't in the top 50. For me, this song is even more mesmerising than any of Bob Dylan's incredible output ... and I'm saying that as a firm fan of Dylan.

The song suggests for me that there are different ways of looking at everything in life ... even conceptually simple things such as clouds can appear totally different to two observers.

The lyrics seem to convey the idea that eventually we can learn to look at things from other perspectives, not just our own.

That invokes a more mature and happier approach to life: with hindsight the things we argued about once can seem somewhat trivial compared to the way we look back on them.

I imagine the author of this song looking back on an experience and thinking, I wish I hadn't argued that point, now I'm older I can see it from a different perspective. All I achieved was conflict. And that conflict stopped me from enjoying and appreciating the differences I encountered.

Things that Aren't Conflicts #

When I see arguments like I mentioned at the outset, I think, that's merely one way of looking at things.

Is CSS-in-JS a mistake? For some people, it is. Totally unnecessary. For others, it's absolutely the best way to achieve the results they seek. Having worked in both camps, I can see there's no conflict in building for the web either way.

Is JavaScript destroying the web? I'll say this again (though I am starting to sound like a stuck record!) that there's a lack of HTML and CSS skills among our peers, and that will need to be addressed at some point. But look at the things a dynamic language like JavaScript has enabled us to achieve. Without it, the web would be a much poorer place.

Looking Back #

Conflict is paralysing. It stops anything useful from being achieved. It's far better to acknowledge the difference, respectfully share our opinion, and leave it at that.

Starting now, and in years to come, if there are to be any, we ought to try to do the same. If we see someone voice an opinion that riles you, try to think, can I see it from both sides, now?

https://open.spotify.com/track/1pjATX7sbd6Y4jMVqIvzHk"

Building a media player with Web Components

Mon, 23 Dec 2024 00:00:00 GMT

This past year I've been involved in replatforming our media player to use media-chrome from Mux instead of JWPlayer. Here's how that went.

I hate JWPlayer with a passion. Everything that should be easy about it causes us as a development team headaches every day. Here's an example:

let numberOfTimesChecked = 0;
function doSomething() {
  setTimeout(() => {
    if(typeof jwplayer === 'function' && jwplayer().hasOwnProperty("play")) {
      doSomething();
    } else {
     numberOfTimesChecked += 1; 
     return;
    }
  }, 500)
}

If you're any kind of javascript developer you're screaming internally now, aren't you?

This function uses recursion to fire the function again continually every half a second because there's no way to verify that JWPlayer is ready before you call it.

JWPlayer makes API calls to their CDN (another reason we're moving away: we have corporate clients who proactively block code delivery CDNs). These calls check whether the token we provide is valid and load extra code that's not included in the bundle (hls.js is loaded from their domain using this method. Don't ask me whether the hls.js code has been modified, I have no idea).

Yet, jwplayer() provides no asynchronous API. Not only that, there's a significant delay between jwplayer() being ready, and its child functions being ready, as you can see above, jwplayer().play() might not be defined when we call it.

This is stupid and is causing us to write spaghetti code to mitigate it's ... let's call them idiosyncrasies ... and I am still not sure how many users we're letting down because of that.

So, good riddance JWPlayer.

A better way

If you're having similar struggles, I heartily suggest you check out Media Chrome from the Mux team. This comprehensive suite of Web Components provides a lot more functionality which you can build on top of the native <video> element. They even have elements for streaming video. We use their Mux Player for decoding audio and video streams served from their platform.

I really love that you can write a simple lines of JavaScript once you've done npm install media-chrome:

import 'media-chrome';

And you can do the rest in HTML:

<media-controller>
  <video
    slot="media"
    src="https://stream.mux.com/DS00Spx1CV902MCtPj5WknGlR102V5HFkDe/high.mp4"
  ></video>
</media-controller>

The slot directive tells the <media-controller> element you want to render this <video> element in the media slot.

There are other slots for things like icons, poster images, and many other fancy things.

If you're looking for a quick implementation you can run with, I'd recommend taking a look at their website https://player.style, where you can copy and paste a theme straight into your HTML.

Of course, our situation is a bit different in that we have to provide an API for our clients to use as well as integrations for our trackers to ensure qualifications can be achieved, and settings that are publicly available to customise the player.

We also have live events as well as podcast content to think about.

This code had so many moving parts and integrations that it's honestly a bit dizzying thinking about it all.

But honestly we started with the basic idea that we wanted it to be super easy for people to use. We wanted them to load the script and add as few lines to the page as possible.

I think we've achieved that, the basic implementation is

<my-media-player
  token="player-token"
>
</my-media-player>

Along with the script tag, that's the basic API.

I love the fact that this is native HTML. It hides so much complexity that the average user just doesn't need to think about.

In the first instance we didn't want the player to fail silently when that's in our control. As long as they have included the JS, We have to provide some UI for occasions where the video can't be resolved.

This card shows when the provided token can't be resolved to a specific video.

There are other notable visual differences when a livestream is playing. Specifically this changes whether they can do things like seek or otherwise skip segments of the video:

Otherwise the chrome looks like this for a video:

And this for audio:

There's a huge amount of customisation available, both in what the user can provide when they instantiate the player, and in what they set up on the backend interface for options etc.

For example, they might want to show some custom branding for their channel, so the player controls will adopt their custom colour. They can add a custom button into the player. They can allow or block downloads.

Mux elements are very well thought out

I really admire how thoughtful the team has been about making this player as adaptable as it can be.

Just about all of their individual elements that make up this suite have a <slot name="icon"> so that you can override their default icons with a simple SVG element.

They also provide a huge range of CSS variables, which allow you to customise a lot of the internal CSS:

:host {
  --media-font-size: 24rem;
  --media-range-thumb-background: red;
}

These cascade down, so as here if you do this in your :host {} block, all elements will adopt them.

Honestly, this is fantastic. A lot of the CSS is hidden in the shadow DOM, but honestly I don't want access to all of the internals. I've build this complex player without being restricted by that.

Events

We also provide an API so that users can send events into the player. For example, once you've added the HTML code you can do:

const player = document.querySelector('media-manager-player')

player?.addEventListener('play', () => {
  player.dispatchEvent(new CustomEvent('pause'))
})

This is a totally contrived example but when a play even is detected we can pause it, or do any one of a number of events.

But there was one I found particularly tricky: chapters.

The use case here is that we need to expose the content of the chapters to users. We use this internally to inform users of how many chapters they need to watch before they can take a quiz which will enable them to get points towards their accreditation.

But the Mux stuff is so straightforward that it handled the chapters content for me.

<my-video-player>
  <mux-video>
    <track type="chapters" url="https://path/to/my/chapters.VTT">
  </mux-video>
</my-video-player>

Once you pass a VTT file to mux-video it fetches it and parses it to provide a chapters menu. However it doesn't expose that content in a way I can further transform or pass back outside of the player.

So I implemented our own solution:

myPlayer.dispatchEvent(new CustomEvent('getchapters'))

myPlayer.addEventListener('chaptersdataupdated', (event) => {
  console.log(event.detail) // JSON string of chapters
})

Internally, this fetches the chapters, parses the VTT file and fires off the event. This means average users aren't burdened with the extra wait time of fetching the chapters a second time as the player instantiates.

This project hasn't been without it's pitfalls and struggles especially since it's been in development for around a year, so significant (and very positive) upgrades needed to me merged in.

We also found it a challenge to ensure that the code was as well tested as it can be. This is hard because most test runners assume components are instantiated in JS after the real DOM has loaded, and in this case that isn't true. I've come up with a solution, and Storybook's support has got a lot better in it's latest release. But there are still significant gaps.

You can find a full write up on the testing situation here.

However I'm so pleased to have stumbled across the media-chrome project, and to have in that time also got to grips with native Web Components.

And yes, I did a case study for integration with Vue, React 17 and 18, and with Svelte, and the player rendered fine in all of them.

Augmenting the details element

Wed, 23 Jul 2025 00:00:00 GMT

I've found the details element superbly useful lately. I added only one or two things to make this useful element suit two very different use cases

I recently rebuilt some dropdowns with a very minimal amount of JavaScript, and most of that is to support Safari's lack of support for the :marker pseudoelement.

Previously, we had a structure similar to this:

<nav>
  <ul>
    <li>
      <button type="button" aria-controls="mega-menu-one-dropdown" aria-expanded="false" id="mega-menu-one-button">
        link parent
        <svg class="caret"></svg>
      </a>
      <div id="mega-menu-one-dropdown" aria-controlledby="mega-menu-one-button" aria-expanded="false">
        // sub-nav content, which is huge
        <ul>
          <li>
            <a href=#>
              link child
            </a>
          </li>
        </ul>
      </div>
    </li>
  </ul>
</nav>

As you might imagine this was difficult to deal with and since it was generated dynamically, very hard to reason about.

Next to that of course we had a bunch of JS that would do the following when one of the buttons were clicked:

Update the aria-role on the button
Rotate the SVG caret on the button
Update the aria-role on the dropdown element
Reset the aria-role on other buttons
Reset the aria-role on other dropdown elements

The HTML for something like this would be

<div
  id="mega-menu-one-dropdown"
  aria-controlledby="mega-menu-one-button"
  aria-expanded="false"
  class="tw-hidden aria-expanded:tw-block"
></div>

And we didn't actually just do this. In my first iteration of JavaScript I neglected to notice that you can add modifier Tailwind classes to the HTML so that when aria-roles change, different styling can be applied:

Instead I was doing the following:

button.addEventHandler("click", () => {
  ["tw-hidden", "tw-block"].forEach((classListItem) =>
    dropdown.classList.toggle(classListItem)
  );
});

Yeah, probably don't do that.

Since it received a design refresh, I took the opportunity to simplify it significantly.

The Refresh

Now with the details element the markup looks a lot more sane, although to be honest it's always going to be quite complicated:

<nav>
  <details>
    <summary>
      Link parent
      <svg class="caret"></svg>
    </summary>
    <div>
      <ul>
        <li>
          <a href="#"> link child </a>
        </li>
      </ul>
    </div>
  </details>
</nav>

This is a lot easier to reason about, aria roles are built in, and the functionality is identical to the user.

Safari

I'm not going to belittle Safari here. It's a solid browser and Jen Simmons and the team have worked hard to get it to feature parity with other browsers. Their release cycle might be a bit slower but it's a formidable tool.

I did find that I needed some extra JS to support just one thing: updating the <svg> caret style, which needs to rotate 180 degrees to indicate the collapsible section is open.

I elected to scope this to Safari only. This is because I want the developer who comes to this file in 2 years time to instantly see which lines they can now delete:

/**
 * @see https://caniuse.com/css-marker-pseudo
 * To polyfill the css ::marker pseudo element
 * so that SVG carets can rotate when <details> elements
 * are expanded. Can be deleted when this is supported in Safari
 */
const isSafari =
  navigator.userAgent.indexOf("Safari") > -1 &&
  navigator.userAgent.indexOf("Chrome") === -1;

detailsElement.addEventListener("toggle", () => {
  isSafari &&
    detailsElement.classList.toggle("details-open", detailsElement.open);
});

Yes, I'm a fan of clear signposting.

Now that we have this class, we can use it to toggle the rotation on the SVG.

Styling

Unfortunately Tailwind v3 doesn't seem to support the open attribute natively, and we don't have capacity to upgrade to v4 just now. So I had to bake this into the CSS like so:

details:open > summary svg.caret {
  @apply tw-rotate-180;
}
/* Safari workaround: 
 * use .details-open class via JS instead of :open pseudo-class 
 * @see https://caniuse.com/css-marker-pseudo
 * Remove if safari now supports this
*/
.details-open > summary svg.caret {
  @apply tw-rotate-180;
}

Closing other details

The one thing I did add was that when you open one menu item, the others should close.

document.addEventListener("click", (event) => {
  detailsElements.forEach((detailsElement) => {
    if (event.target.closest("details") === detailsElement) {
      return;
    }
    detailsElement.open = false;
    isSafari && detailsElement.classList.remove("details-open");
  });
});

Instead of handling this when a <details> element is clicked however, I put it on a document listener. Reason being that I wanted to also close the other elements when someone clicks elsewhere on the document:

Details is great!

When we roll this out to our sites it'll achieve the following:

It'll require less custom JS
The code will be easier to follow at a glance
Browsers will handle aria
There's less chance it'll break when changed
The elements have more semantic meaning

I'm so looking forward to more elements like this, including spicy sections, and especially carousels.

Building a Frontend With GatsbyJS

Sun, 02 Jan 2022 00:00:00 GMT

In this article we’ll learn how to integrate a GatsbyJS application that uses Webiny as a Headless CMS. We’ll go through the following:

Getting started with Webiny
Setting up a GatsbyJS project
Fetching data
Rendering data using Gatsby’s data layer
Rendering Webiny’s rich text field

Read the full article"

Building a Frontend With NextJS

Mon, 03 Jan 2022 00:00:00 GMT

In this article we’ll learn how to integrate a NextJS application that uses Webiny as a Headless CMS. We’ll go through the following:

Getting started with Webiny
Setting up a NextJS project
Fetching data
Rendering Webiny’s rich text field
Using NextJS preview mode

Read the full article"

Building an Accordion Component with React Hooks

Sat, 15 Jun 2019 00:00:00 GMT

In this article, we takes a deep dive into creating an accordion module for React that renders on the server, and that works in Internet Explorer 11.

On a recent project I was involved in creating this Accordion component with a few key requirements:

It should provide collapsible section functionality for our users, a significant portion of whom are still on Internet Explorer 11
It should be as accessible as possible, especially for assistive technology users
It should show content even if the user hasn't been able to download JavaScript, or has it turned off
It can be used multiple times on a page

Let's get started!

Basic Setup #

Here's the basic layout for the module:

import React from "react"
const Accordion = (props) => {
  return (
    <li>
      <h3>
        <button>
          {question}
        </button>
      </h3>
      <div>
      {answer}
      </div>
    </li>
  )
}
export default Accordion

This will render a basic section and can accept our content labelled question and answer. Outside the accordion, I will define an unordered list, <ul>, to wrap around the items.

The elements used here are one of the suggestions made by Sara Souiedan in her blog post "How do you mark up an accordion?". It's well worth a read because it dives deep into the semantic meaning of these elements.

You can find her excellent article here: https://www.sarasoueidan.com/blog/accordion-markup/

IE Support Not Needed? Use <details> #

On Sara's blog post above, using the <details> and <summary> elements here for more semantic markup is discussed. This is a HTML-native accordion element!

If you don't need to support Internet Explorer 11, use these and skip most of this tutorial: you'll see that the accordion collapses and opens without any css or javascript.

Functionality #

Next, I'm going to import our first hook. Hooks allow us to write React lifecyle methods without classes. This means our code can be more functional, or at least not a mix of OO and functional programming.

First, I need to import the useState hook along with React from the React library:

import React, { useState } from "react"

Next, I set my state items. There are 2 because setContentVisible will be a function to update isContentVisible.

  let [isContentVisible, setContentVisible] = useState(true)

I'm setting a default of true here because I'm using server-side-rendering, so that users without JavaScript (and search engines) can see this content. Now I can use this state directly on my button:

<button onClick={() => setContentVisible(isContentVisible = !isContentVisible)}>
{question}
</button>

Doesn't this look a lot more straightforward than writing a function before the render method?

Screen Reader Assistance #

Next, I'm going to add some aria roles to the accordion to indicate to users when the component's content area is visible or not:

<li>
      <h3>
        <button
            onClick={() => setContentVisible(isContentVisible = !isContentVisible)}
            aria-controls="accordion-1"
            aria-expanded={isContentVisible}
        >
          {question}
        </button>
      </h3>
      <div>
      <ContentContainer
          aria-hidden={!isContentVisible}
          id="accordion-1"
        />
      {answer}
      </div>
    </li>
    )

This gives the screen reader ability to hear which section is open or closed, and the ability to open or close the accordion.

But that id isn't going to cut it for more than one use on a page.

To do that, we need to set a unique id of the accordion item so we can show the relationship between the control and content elements.

const elementId = (((1 + Math.random()) * 0x10000) | 0)
    .toString(16)
    .substring(1)

Once I've replaced id="accordion-1" with id={elementId} I should have a fairly unique ID that links up the content and the button.

The Slings & Arrows of Outrageous Smooth Scrolling #

One of the requirements you might have with an accordion of this kind is that when the accordion opens / closes, the user is scrolled to the content area or the title.

You can define a separate toggleElement function to get the ID of the current element, and scroll to the content area.

const domElementTop = elementId.offsetTop - 150
  const toggleVisibility = () => {
    window.setTimeout(function() {
      window.scrollTo({ top: domElementTop, behavior: "smooth" })
    }, 0)
  }

There's an IE11 caveat with this though. IE calculates scrollTo differently, with a px value instead of a string or ID. You can use scrollTop instead, but this doesn't allow you to have smooth scrolling behaviour which is a nicer effect for users. If you want to support IE11 with smooth scrolling, you can use smoothscroll-polyfill and call that before window.scrollTo.

There's a bug in iOS Safari too! If you want to support Safari on iOS (particularly some iPad versions) it's advisable to set a timeout when you start scrolling:

    window.setTimeout(() => {
      // call the polyfill (to support IE11)
      smoothscroll.polyfill()
      window.scrollTo({ top: domElementTop, behavior: 'smooth' })
    }, 0)

Showing & Hiding Content #

The last item we need to take care of is showing / hiding the content area. Now, we could do it this way:

{isContentVisible && (
    <div
       aria-hidden={!isContentVisible}
       id={`content-${AccordionIdentifier}`}
    >
      {answer}
    </div>
)}

Note the use of the double-ampersand. If the isContentVisible is true, the following block (inside brackets) will show.

You can also do the show/hide with a class attributes, using the display property to show & hide them using CSS.

Lastly, I want to make sure that when my user has got the JavaScript bundle and the page has been rehydrated (see box, "What is Rehydration"), the content is collapsed, ready for them to interact with. I can do that with the useEffect hook, but I need to make sure I call useEffect only once, similar to how componentDidMount was used.

First, import that from React the same way as the useState hook:

import React, { useState, useEffect } from "react"

Then reverse the value of setContentVisible:

  useEffect(() => {
    setContentVisible(isContentVisible = !isContentVisible)
  }, [])

Did you see the extra square brackets ([]) at the end? By passing an empty array to the useEffect hook we can call it only once. If we didn't do that, the state would be updated every time the component updated in some way.

Conclusion #

I think using React Hooks is a simpler and tidier workflow that helps our code to be quicker to understand and more flexible. There are some significant changes to the way we use React, but understanding the differences in my opinion is only going to improve the way you write your code.

You can find this example on Code Sandbox: https://codesandbox.io/s/wqnmql8l78

Boxout: What is Rehydration? #

Think of a carton of orange juice. You'll often see on the side of the packet "from concentrate" or "rehydrated".

This means they've extracted the water from the juice to condense it. In this way it can be transported more easily, in greater quantities. The water is added back in at the destination.

Rehydration in JavaScript can be compared with this method. With some frameworks, notably GatsbyJS, our page is built in HTML and rendered in the browser. Then, once the JavaScript is loaded, it replaces the HTML with it's JavaScript version of the page.

The advantage is a fully interactive application that is better for SEO and for users.

There are still some key puzzles to solve with rehydration. I'm looking forward to seeing a time when only parts of a page that need interactivity are rehydrated, instead of having to replace the entire page. But it's a great way of making faster and more resilient interactive experiences.

Boxout: MVT, Object-Oriented or Functional Programming? #

Managing the code of large applications where several people are contributing over the course of longer periods of time tends to be difficult. In fact any sufficiently complicated system tends to break down over time.

To combat this tendency in code, people have tried several different approaches. The ones I'm familiar with are MVC, OO (object-oriented) and Functional.

As far as I know, Angular 1 introduced the MVC principle to the frontend. It tried to keep separate the views (or templates), models (interpretations of data) and controllers (the functional parts) separate.

This proved in many cases to be a burden on the overall code that was sent down to the client. A few later frameworks experimented with Object Oriented programming, which works great for languages like Ruby and PHP, and can be successfully implemented with JavaScript too.

However, JavaScript doesn't really work in an object-oriented way under the hood. The class keyword was a wrapper around some of the prototypes of the language.

In an effort to work with the language instead of imposing an artificial structure on top of it, functional programming has been growing in popularity in recent years."

How to Cache and De-duplicate Fetch Requests

Tue, 18 Nov 2025 00:00:00 GMT

If you have an application which makes many requests to the same API you can use a key to cache and de-duplicate requests.

Why might this happen?

Ideally you shouldn't encounter this situation. Your data layer or state should be a separate entity which components can subscribe to.

This is what React Context and similar solutions do: they hoist the state to a separate layer which can then be passed to each component.

But you know that already.

If you're using vanilla JavaScript you have some other options. You can use TanStack Store or some other third party.

But where's the fun in that?

The implementation

Imagine we have a component that renders a chart. It should fetch data from our API to populate that data.

import retrieveData from "@api/retrieve-data";

async function myChart() {
  const totals = await retrieveData("getTotals");
}

Simple enough.

This is where it gets complicated. This is one component in a complex dashboard that renders 10-20 different charts.

There are inputs on this chart too: users chan change date ranges, apply filters and all that good stuff.

Add to that there are several different pages to this dashboard, each of which involves a server re-render. There's no possibility of using a frontend router.

What would you do?

I created a CustomEvent to enable my components to re-render when a filter changed.

Dispatching the event from filter:

filterElement.addEventListener("change", () => {
  // ... (other event stuff)
  window.dispatchEvent(new CustomEvent("refreshState"))
});

Reacting to it in the components:

import retrieveData from "@api/retrieve-data";

async function myChart() {
  const totals = await retrieveData("getTotals");
}
window.addEventListener("refreshState", () => myChart());

This calls the chart function again when the filter changes. It allows me to keep my component focused on the UI and abstract state to the retrieveData component.

Let's take a look at that now.

Retrieving Data

This is the basic implementation I ended up with.

I've leant on the URL query params as a central location to store updated state. That way I can de-couple the user input filters from the fetch request and still have them easily accessible:

async function retrieveData(requestType, args) {
  const urlParams = new URLSearchParams(window.location.search);

  const dateFromFilter = urlParams.get("datefrom");
  const dateToFilter = urlParams.get("datefrom");
  const user = getUser();

  const cacheKey = `dashboard-action=${requestType}-user=${user}-from=${dateToFilter}-to=${dateToFilter}`;

  const cachedData = localStorage.getItem(cacheKey);

  if(cachedData) {
    return cachedData;
  }
  const freshData = await callApi(requestType, { user, from: dateFromFilter to: dateToFilter });

  localStorage.setItem(
    cacheKey,
    JSON.stringify(freshData)
  )

  return freshData;
}

So you can see we're setting a cache and returning that cached data if it still exists, otherwise fetching from the API and storing that data.

This is a good start, but it's going to get more complex.

Why?

Because it takes time for localStorage to save items. Meaning if we initiate 15 requests concurrently, each one could make an API request when it doesn't need to.

Let's dive into how we de-duplicate those requests.

De-duplicate requests

If we have a dozen or so components on a page, and they sometimes use the same data from the API, how do we reduce load on the API?

We can use the cacheKey above to store a request, and then create a queue for the requests:

const inFlightRequests = new Map();

async function retrieveData(requestType, args) {
  const urlParams = new URLSearchParams(window.location.search);

  // ...

  const cacheKey = `dashboard-action=${requestType}-user=${user}-from=${dateToFilter}-to=${dateToFilter}`;

  // 1. Request is in progress, return that data
  if (inFlightRequests.has(cacheKey)) {
    return inFlightRequests.get(cacheKey);
  }

  // 2. If we have cached data, return that
  const cachedData = localStorage.getItem(cacheKey);

  if(cachedData) {
    return cachedData;
  }

  // 3. IF not, get fresh data
  const freshData = await callApi(requestType, { user, from: dateFromFilter to: dateToFilter });

  // Then set everything else up
  inFlightRequests.set(cacheKey, retrievedData);

  localStorage.setItem(
    cacheKey,
    JSON.stringify(freshData)
  )

  inFlightRequests.delete(cacheKey);

  return freshData;
}

The inFlightRequests lives in the module scope which means it persists when the function is called.

It's also a Map, in which each item is stored by key-value pairs in which keys are unique. You can't have two keys the same in a Map.

More details here.

Expiring the Cache

Last bit is how to make sure we're not returning stale data. This is based on a few assumptions and might be different in your case. But I've gone for 30 seconds.

For me, thirty seconds is enough time for users to load a page, realise they're on the wrong page, and navigate to a different one, or to change their filters a few times before they figure out what data they actually want to see.

Also if the data is updated, I'm guessing it'll take more than 30 seconds to do that.

Let's see how we can expire the cache:

function retrieveData() {
  // ...

  // Not using Temporal? You should look into that.
  const now = Temporal.Now.instant();

  localStorage.setItem(
    cacheKey,
    JSON.stringify({
      timestamp: now,
      data,
    })
  );
  if (cachedData) {
    const parsedCachedData = JSON.parse(cachedData);
    const thirtySecondsAgo = now.subtract({ seconds: 30 });

    if (cachedTime <= thirtySecondsAgo) {
      localStorage.removeItem(cacheKey);
      return;
    }
    return parsedCachedData.data;
  }
}

So now we have a timestamp which we can use to decide if the localStorage item is stale and discard it.

A word on Temporal. I'm using a polyfill alongside the browser implementation just now. It'll be implemented soon now it's at Stage 3.

Conclusion

Being able to cache and de-duplicate requests has reduced API calls on certain pages of my application from 13 to 3.

Like I said at the start, hopefully you won't ever have to implement this yourself. But for those of us who are implementing complex applications in vanilla JS, I hope this helps.

Capture and Minimise Errors in your React Applications

Fri, 15 Mar 2019 00:00:00 GMT

We've all done it. Whether it's by forgetting to close a function, misspelling a variable or just plain old doing it wrong...

If we we've spent any time developing, specifically with JavaScript, we've seen the White Screen of Death. Nothing renders to the screen, and the only way of investigating the issue we caused is by opening the console and looking at the stack trace.

But what if that error occurs in production? On someone elses' machine? If this is happening (chances are it is) you might be losing a large segment of potential customers (How many? Nobody knows!), not to mention the fact that you have absolutely no visibility about the nature of the error, so debugging and fixing it becomes very difficult.

In this tutorial we're going to dive into React's ErrorBoundary API to see how it can be used effectively to (1) stop the app from crashing, (2) provide a fallback, and (3) capture data for analysis later.

This article was originally published in Net Magazine issue 316.

Let's Get Started! #

To save this tutorial from being too long, I've included the code for the above on GitHub at https://github.com/endymion1818/netmag-react-error-reporting. It's a basic create-react-app setup with axios hooked up to get some data from the Star Wars API. However, there's a deliberately placed issue in the render() method that's causing the app to crash.

To see this code in action, download or clone the repo. Open this directory in your terminal and install dependencies by running npm i.

You can run the app in development mode by typing npm start. It's time to take a closer look at what's happening here.

Error Boundary API #

The ErrorBoundary API allows us to create a component that wraps around our whole app, an individual component or both. Then, if anything in that component fails, we can define an error message - or a replacement UI component - to appear instead of our broken one.

We're going to create a new file, FormErrorBoundary.js and add some code there. This does the following:

Defines some state so that we can decide what to do if there is an exception
Use a built-in React function to set state if there is an error
Send the error message somewhere we can examine and track it

export default class ErrorBoundary extends React.Component {
  constructor(props) {
    super(props)
    this.state = { hasError: false }
  }
  static getDerivedStateFromError(error) {
    return { hasError: true }
  }
  componentDidCatch(error, errorInfo) {
  }
  // continued below ...

Next, let's provide our fallback UI component:


  // ... continued from above

  render() {
    const { children } = this.props

    if (this.state.hasError) {
      return (
        <div>
          <h2>Something went wrong with our app!</h2>
          <p>We're aware of the problem and we're working hard to fix it.</p>
          <p>In the meantime, you can reach us by phone or email.</p>
        </div>
      )
    }
    return children
  }
}

Now let's import that into our form, and wrap it around the rest of the component:

import ErrorBoundary from './ErrorBoundary'

Now if an exception occurs in that component, the visitor will see this message instead. You can trigger it on your machine easily by leaving a tag unclosed or doing something else you know would ordinarily break your app.

But, how do we capture what has caused our error so we can fix it?

Grab, Bag and Tag #

This is where a logging service comes in. Now, inside the componentDidCatch() method of our ErrorBoundary component, we can send the exception data to a 3rd party service.

For Enterprise level applications, I've enjoyed using Splunk. Splunk is built for more than error reporting and I've seen people use it as a conversion metric tool as well. Splunk's strength is in the vast volumes of data it can collect. It also allows you to visualise your data in charts so you can track trends and analyse data more easily.

However, for smaller applications Splunk might be a bit overkill. I've recently been introduced to Sentry, which offers a simpler set of tools that allow you to collect, categorise, monitor and track your issues.

Sentry has a free tier for developers and is really easy to get started with.

Error Reporting with Sentry #

Now, let's assume that - even though you've tested your app thoroughly - you suspect there are errors happening in production. Or perhaps you want to find out for certain if there are any.

Let's add Sentry to this project.

npm install @sentry/browser

This gets the Sentry package which we can now import into our ErrorBoundary, and initialise with our credentials to enable Sentry to recieve our errors:

import * as Sentry from "@sentry/browser"

Sentry.init({
  dsn: "your-dsn-here"
})

Update the componentDidCatch() function with the following:

  componentDidCatch(error, errorInfo) {
    Sentry.withScope(scope => {
      Object.keys(errorInfo).forEach(key => {
        scope.setExtra(key, errorInfo[key])
      })
      Sentry.captureException(error)
    })
  }

Now each time an error occurs, the data will be sent to Sentry, and you can even recieve an email alert!

Let's see it In Action! #

Are you ready to see what we've been doing? To do that, we need to build our application, and serve it somewhere - ideally your local computer. This step involves installing a local javaScript web server to your computer.

First, in your terminal, run npm install --global sentry. Next, from your project directory, run npm build. This builds the production-ready app. Now we can serve this folder locally by running serve -s build. Opening up a browser and typing https://localhost:5000 and you should be able to see our beautiful error message!

Congratulations! You are no longer in the dark about your JavaScript errors. Now you can increase the usefulness of your app and improve your stability by not only providing a fallback state for your app, but also by reporting and squashing all of your remaining bugs.

React Testing Caveats #

Catching and reporting errors in the way described in this article does have one caveat: the ErrorBoundary API is only capable of capturing errors that happen in your app. So errors could occur before your app is loaded that aren't captured.

Sentry specifically mentions this in their document, stating that their error reporting tool should be initialised as soon as possible in your application.

On some projects this is harder to achieve than others without some serious hacking about. On projects with server-side-rendering I've been able to write the script into Helmet. That way, the error can be captured by using the window.onerror browser API instead of React's built-in API that we're discussing in the article:

import * as Sentry from "@sentry/browser"
// ...
<Helmet
	    <script type="text/javascript">{`
        {
           Sentry.init({
  				dsn: "your-dsn-here"
			})
            window.onerror = Sentry.withScope(scope => {
      			Object.keys(errorInfo).forEach(key => {
        			scope.setExtra(key, errorInfo[key])
      			})
      			Sentry.captureException(error)
    		})
        }
    `}</script>
/>

Beware of Non-Standard Error Reporting #

It may not surprise you to know that the way browsers report errors differs across different rendering engines and implementations. Sometimes, the errorObject object isn't defined, so you'll likely get some errors that aren't that helpful. Most will show you a really helpful level of detail, down to the exact line of code where the error occurred. But don't expect the same level of reporting from every browser. This doesn't mean Internet Explorer either. Safari only recently introduced some better error reporting practices. There's a comprehensive table on Sentry's blog to identify the inconsistencies, as well as implementing a workaround: https://blog.sentry.io/2016/01/04/client-javascript-reporting-window-onerror.html#browser-compatibility

Errors or Exceptions? #

You may have heard situations where your application fail termed Exceptions instead of errors. The reason for that is that they're not errors insofar as a problem with the language but instead exceptional results from the way that language has been used. This means the problem has originated with something that is not driven by by pure logic ... that means you or I. Therefore they are exceptional results from the way you have applied the language."

Implementing a carousel with classes only.

Thu, 15 Aug 2024 00:00:00 GMT

Last year I built a carousel project and showcased it here. I've just updated the code I used so that you can implement a carousel by only adding the class name to the parent element. Here's a breakdown.

Here's the previous article where I implemented a carousel.

The thing with that project is that although I was 90% of the way there, I didn't realise that there were certain limitations with the large monolith it would be implemented in. I'm working on a Drupal site, and we need to sometimes initialise a carousel on a queue of items. When you create the queue you only have access to the class names of the wrapper. I recently got the chance to revisit this code and implement it so it could work in this setting.

Objective

The basic objective is to be able to have markup like this:

<ul class="carousel">
  <li>
    <img src="my-image.png">
    <div>
      <h4>Title</h4>
      <p>Some text</p>
    </div>
  </li>
  <li>
    <img src="my-image-2.png">
    <div>
      <h4>Another Title</h4>
      <p>Some more text</p>
    </div>
  </li>
</ul>

I'm not arguing semantics here, this is Drupal: the basic premise is that I don't have control of the HTML structure; and in fact this is vastly simplified because Drupal seems to inject a <div> into the DOM every time someone on the planet says "React".

I also wanted to have some customisations for whoever is implementing a carousel, so they could choose to have navigation previous / next arrows, and also indicators (the little dots that typically show at the bottom of the carousel to indicate which item in the list is currently active), and also be able to customise the rotation speed.

<div class="carousel carousel-speed-4000 carousel-show-indicators carousel-show-navigation-buttons">

Let's get to it!

Instantiation

Since this specifically for a Drupal project, I needed to query the DOM element using Drupal's once() method, so there's an IIFE at the bottom of the file that provides the integration:

(function (Drupal, once) {
  Drupal.behaviors.carousel = {
    attach() {

      once("create-carousel", ".carousel").forEach(element => {
        createCarousel(element);
      });
    }
  }
})(Drupal, once);

If you're not using Drupal, then you can of course just use document.querySelectorAll() as normal.

There's another trick to this: Drupal loads in sections of the DOM separately, so the carousel items might not be present in the DOM initially. This is the main case for using once(), but it also means that we have to start with the DOM elements hidden to begin with.

(function (Drupal, once) {
  Drupal.behaviors.carousel = {
    attach() {

      once("create-carousel", ".carousel").forEach(element => {
        createCarousel(element);
        element.classList.remove('tw-hidden') // or element.style.display = 'block'
      });
    }
  }
})(Drupal, once);

This ensures that the elements are hidden until the carousel is initialised, so we don't get a paint where all the elements appear stacked one on top of another.

Setting heights

I'm a big believer that carousels should be uniform height. Nobody wants to be reading some copy below a carousel only for it to shift up and down every 3 seconds. But since the items are being created by users who have no regard for image sizes and length of content, we can't guarantee the height will be uniform.

I've got a little trick for that!

// 1. get the carousel items
const carouselItems = carouselElement.children;

// 2. Set holding variables for new height of the carousel
let proposedCarouselHeight = 100;

// 3. use height of the tallest item
carouselElement.style.display = 'block';

carouselItemsArray.map(item => {
  if(item.offsetHeight > proposedCarouselHeight) {
    // set the proposed height to the height of the tallest image
    proposedCarouselHeight = getHiddenElementHeight(item);
  }
});
carouselElement.style.height = `${proposedCarouselHeight}px`;

// 4. set the carousel to be visible
carouselElement.classList.add('tw-block'); // or carouselElement.style.display = 'block'

The getHiddenElementHeight() function is a bit sneaky. If you do element.offsetHeight on a hidden element, the value is going to be 0. So we need to first show the item, then grab the offsetHeight, then hide it again.

Whilst it seems a little redundant, it does mean we again avoid that flash where images are moving around as the JavaScript initialises. This function will likely run so quickly that users don't see it happen.

function getHiddenElementHeight(element) {
  // Save the original display style
  const originalDisplay = element.style.display;

  // Temporarily show the element
  element.style.display = 'block';

  // Get the height
  const height = element.offsetHeight;

  // Revert to the original display style
  element.style.display = originalDisplay;

  return height;
}

Now we have a carousel with a specific pixel height, and all items inside it can be set to position: absolute with inset: 0 to make sure they all take up the height of the container.

Autorotation Interval

The last tricky little bit was to facilitate custom autoplay speeds with only class names available.

The api for this is that the user would add carousel-autoplayspeed-<number> to the class list, we can then check if there's a class beginning with carousel-autoplay-speed, and parse the digits at the end to get the desired speed:

let autoRotationInterval = 3000;

const speedClasses = carouselElement.className.match(/carousel-autoplayspeed-\d+/g);

if (speedClasses) {
  const speedClass = speedClasses[0].split("-")[2]
  autoRotationInterval = parseInt(speedClass);
}

Now users can specify a millisecond interval of their choosing.

Here's the full implementation, including indicators and navigation buttons. Happy copypasta!

function getHiddenElementHeight(element) {
  // Save the original display style
  const originalDisplay = element.style.display;

  // Temporarily show the element
  element.style.display = 'block';

  // Get the height
  const height = element.offsetHeight;

  // Revert to the original display style
  element.style.display = originalDisplay;

  return height;
}
/**
 * 
 * @param {HTMLElement} carouselElement 
 * @returns a ✨ new carousel ✨
 * @notes
 * 
 * The carousel has some parameters which can be modified by adding classes to the carousel element
 * 
 * 1. Autorotation speed. Add a class of `carousel-autoplayspeed-<number>` where `<number>` is the speed in milliseconds to adjust the default rotation speed
 * 3. Show indicators. Add a class of `carousel-show-indicators` to show indicators
 * 4. Show navigators. Add a class of `carousel-show-navigators` to show navigators
 * 
 */
function createCarousel(
  carouselElement,
) {
  // nope
  if (typeof window === 'undefined') {
    return;
  }
  // also nope
  if (!carouselElement) {
    return;
  }
  
  /** @type { HTMLElement[] | HTMLCollection | null} */
  const carouselItems = carouselElement.children;
  
  // still nope
  if(!carouselItems) {
    console.info(`carousel doesn't seem to have any items`)
    return;
  }
  
  let autoRotationInterval = 3_000;
  
  const autoPlayClass = carouselElement.className.match(/carousel-autoplayspeed-\d+/g)
  if(autoPlayClass) {
    const autoPlaySpeed = autoPlayClass[0].split("-")[2]
    autoRotationInterval = parseInt(autoPlaySpeed);
  }

  /** @type {HTMLElement[]} */
  const carouselItemsArray = Array.from(carouselItems).filter(item => item instanceof HTMLElement);

  // 1. set carousel to be full width of the containing area
  carouselElement.style.width = '100%';

  // 2. Set holding variables for new width / height of carousel
  let proposedCarouselHeight = 100;

  // if not, use height of the tallest image
  carouselElement.style.display = 'block';
  carouselItemsArray.map(item => {
    if(item.offsetHeight > proposedCarouselHeight) {
      // set the proposed height to the height of the tallest image
      proposedCarouselHeight = getHiddenElementHeight(item);
    }
  });
  carouselElement.style.height = `${proposedCarouselHeight}px`;

  // finally, set the carousel to be visible
  carouselElement.classList.add('tw-block', 'tw-relative', 'md:tw-overflow-hidden');

  // define interval timer so we can clear it later
  let intervalInstance = null;

  /**
   * HELPER FUNCTIONS
   */
  /**
 
   * Gets the currently active slide
   * @returns {HTMLElement | Element} item
   */
  function getActiveItem() {
    const activeItem = carouselElement?.querySelector('[data-carousel-item-current="true"]');
    
    if(!activeItem || !(activeItem instanceof HTMLElement)) {
      // @ts-ignore carouselItems is definitely defined by this point
      return carouselItems[0];
    }
    return activeItem;
  }
  /**
   * 
   * gets the position of the item in the array
   * @param {HTMLElement | Element} item 
   * @returns {number} itemIndex
   */
  function getPositionOfItem(item) {
    const position = carouselItemsArray.findIndex((carouselItem) => {
      return carouselItem === item && carouselItem.getAttribute('data-carousel-item');
    });
    return position;
  }
  /**
   * 
   * Sets the carousel to the next slide
   * @param {HTMLElement | Element} carouselItem
   * @returns {void}
   */
  function setItemAsActive(carouselItem) {
    carouselItem.setAttribute('data-carousel-item-current', 'true');
    carouselItem.classList.remove('tw-opacity-0');
    carouselItem.classList.add('tw-opacity-100');
  }
  /**
   * 
   * @param {HTMLElement| Element } item
   * @returns null
   */
  function setItemAsInactive(item) {
    item.setAttribute('data-carousel-item-current', 'false');
    item.classList.add('tw-opacity-0');
    item.classList.remove('tw-opacity-100');
  }

  /**
   * ACTIONS
   */
  /**
   * Set an interval to cycle through the carousel items
   * @returns {void}
   */
  function cycle() {
    if (autoRotationInterval <= 0) {
      return;
    }
    intervalInstance = window.setTimeout(() => {
      next();
    }, autoRotationInterval);
  }
  /**
   * Clears the cycling interval
   * @returns {void}
   */
  function pause() {
    clearInterval(intervalInstance);
  }
  /**
   * Slides to the next position
   * 
   * @param {HTMLElement | Element} nextItem 
   * @returns {void}
  */
  function slideTo(nextItem) {
    const activeItem = getActiveItem();
    if (!activeItem || !nextItem) {
      return;
    }

    setItemAsInactive(activeItem);
    setItemAsActive(nextItem);
    showActiveIndicator(nextItem);
    pause();
    cycle();
  }

  function showActiveIndicator(nextItem) {
    const nextItemIndex = getPositionOfItem(nextItem);

    const indicators = carouselElement.querySelectorAll('[data-carousel-indicator-for]');

    indicators && Array.from(indicators).map((indicator, index) => {
      if (index === nextItemIndex) {
        indicator.setAttribute('aria-pressed', 'true');
      } else {
        indicator.setAttribute('aria-pressed', 'false');
      }
    });
  }
  /**
   * Based on the currently active item it will go to the next position
   * @returns {void}
   */
  function next() {
    const activeItem = getActiveItem();
    const activeItemPosition = getPositionOfItem(activeItem) ?? 0;

    if(!carouselItems) {
      return;
    }
    if (activeItemPosition === carouselItems.length - 1) {
      // if it is the last item, set first item as next
      return slideTo(carouselItems[0]);
    }
    const nextItem = carouselItems[activeItemPosition + 1];
    
    if(!nextItem.getAttribute('data-carousel-item')) {
      // if it's an indicator, set first item as next
      return slideTo(carouselItems[0]);
    }
    slideTo(nextItem);
  }

  /**
   * Based on the currently active item it will go to the previous position
   * @returns {void}
   */
  function prev() {
    if(!carouselItems) return;
    let activeItem = getActiveItem();

    if(!activeItem) {
      console.log('no active item');
      activeItem = carouselItems[0];
    }

    const activeItemPosition = getPositionOfItem(activeItem);
    
    const prevItem = carouselItems[activeItemPosition - 1];
    
    const actualCarouselItems = carouselItemsArray.filter(item => item.getAttribute('data-carousel-item'));
    
    if(!prevItem && actualCarouselItems) {
      return slideTo(actualCarouselItems[actualCarouselItems.length - 1]);
    }
    slideTo(prevItem);
  }

  /**
   * INIT FUNCTIONS
   */

  /**
  * Create the indicators for the carousel
  * @returns {void}
  */
  function createIndicators() {

    if(!carouselElement.classList.contains('carousel-show-indicators')) {
      return;
    }
    
    const indicatorContainer = `
      <div class="indicator-container tw-absolute tw-bottom-4 tw-left-0 tw-right-0 tw-flex tw-justify-center tw-mb-4">
        ${carouselItemsArray.map((item, index) => `
          <button data-carousel-indicator-for="${index}" aria-pressed="${index === 0 ? "true" : "false"}" class="tw-w-4 tw-h-4 tw-mx-1 tw-rounded-full tw-border tw-border-primary-600 tw-transition-colors tw-duration-300 tw-ease-in-out tw-cursor-pointer hover:tw-ring-2 hover:tw-ring-primary-600 tw-bg-white aria-pressed:tw-bg-zinc-500" aria-label="Slide ${index + 1}">
          </button>
        `).join('')}
      </div>
      `;
    
      carouselElement.insertAdjacentHTML('beforeend', indicatorContainer);

      const instantiatedIndicators = carouselElement.querySelectorAll('[data-carousel-indicator-for]');

      const instantiatedIndicatorsArray = Array.from(instantiatedIndicators);

      instantiatedIndicatorsArray.map(indicator => {
        const clickedCarouselItem = indicator.getAttribute('data-carousel-indicator-for');
        indicator?.addEventListener('click', () => {
          clearTimeout(intervalInstance);
          const carouselItem = carouselItemsArray.find((carouselItem) => carouselItem.getAttribute('data-carousel-item') === clickedCarouselItem);
          carouselItem && slideTo(carouselItem);
          
          instantiatedIndicators.forEach((indicator) => {
            indicator.setAttribute('aria-pressed', 'false');
          });

          indicator.setAttribute('aria-pressed', 'true');
        });
      })
  }


  function createNavigators() {
    if(!carouselElement.classList.contains('carousel-show-navigators')) {
      return;
    }
    const navigatorPrev = `
      <button class="carousel-navigate navigate-prev tw-absolute tw-left-0 tw-bottom-0 tw-top-0 tw-text-white tw-text-2xl tw-shadow-sm tw-transition-all hover:tw-scale-110" type="button"><span class="tw-block tw-rounded tw-transition-opacity tw-bg-white/20 tw-border-white/50 hover:tw-bg-zinc-400">&larr;</span></button>
    `;
    const navigatorNext = `
      <button class="carousel-navigate navigate-next tw-absolute tw-right-0 tw-bottom-0 tw-top-0 tw-text-white tw-text-2xl tw-shadow-sm tw-transition-all hover:tw-scale-110" type="button"><span class="tw-block tw-rounded tw-transition-opacity tw-bg-white/20 tw-border-white/50 hover:tw-bg-zinc-400">&rarr;</span></button>
    `;
    carouselElement.insertAdjacentHTML('beforeend', navigatorPrev);
    carouselElement.insertAdjacentHTML('beforeend', navigatorNext);

    carouselElement.querySelectorAll('.carousel-navigate')?.forEach((navigator) => {
      navigator.addEventListener('click', () => {
        navigator.classList.contains('navigate-prev') ? prev() : next();
      });
    });
  }

  /**
   * Function to initialise carousel
   * @returns {void}
   */
  function init() {
    const activeItem = getActiveItem();

    if(!carouselItems) {
      return;
    }

    carouselItemsArray.map((item, index) => {
      item.classList.add(
        'tw-absolute',
        'tw-inset-0',
        'tw-transition-opacity',
        'tw-opacity-0',
        'tw-duration-600',
      )
      item.setAttribute('data-carousel-item', `${index}`);
    });
    /**
     * if no active item is set then first position is default
    */
    if (activeItem) {
      slideTo(activeItem);
    } else {
      carouselItems && slideTo(carouselItems[0]);
    }
    /**
     * Add event listeners to the buttons if they exist
     */
    const nextButton = carouselElement.querySelector('[data-carousel-next]');
    nextButton && nextButton.addEventListener('click', () => {
      next();
    });
    const prevButton = carouselElement.querySelector('[data-carousel-prev]');
    prevButton && prevButton.addEventListener('click', () => {
      prev();
    });
  }
  createIndicators();
  createNavigators();
  init();
  cycle();
}

// GO!
document.querySelectorAll('.carousel').forEach(carousel => {
  createCarousel(carousel);
  carousel.classList.remove('tw-hidden');
})

Code really is Poetry

Tue, 02 Jun 2015 00:00:00 GMT

I'm not sure if you remember the early days of the web. I can. I remember when my friend and I would sit at home, bored of talking about Babylon 5 (yes, really) and yet not wanting to part company again.

"Let's try to get on the internet!" I would excitedly yelp, and plug in the 14.4k modem, dial into Apple's eWorld online community, and download an early version of Mosaic or Netscape.

It never worked.

Mostly because my mum kept noticing the £400 extra on the telephone bill each time we tried. So we stuck to writing science fiction stories and, after a brush with classics (via an old copy of This Side of Paradise by F Scott Fitzgerald), I started to get into poetry.

Poetry represented for me the biggest leap in my adolescent life so far. My interests went from material that could be taken at face-value, or so was my perspective at the time, to deeper things with hidden meaning.

Every time I tried to comprehend a poem it seemed to change depending on my mood, on how I read each word and stanza. I was genuinely moved by things I could barely understand ... the underpinning structure and esoteric references being quite alien to my understanding.

The Poets Write HTML #

I'm not sure if you'll remember also that many of the first users of the web were writers, if not poets?

For them, the internet represented freedom ... absolute freedom to be published no matter the quality of their writing, no matter the popularity or availability (or distinct lack thereof) of willing publishers. A writer could reach an audience of millions just by learning to write a few lines of code.

But code was pretty hard to grapple with. One poem titled "The Poet Writes HTML" published in Cardiff's The Yellow Crane told how the author felt he was getting further and further away from writing because of the time he spent learning to code. For many years, that account alone turned me away from technology altogether.

But when I came back ... that was when a new revelation hit me.

The Language of the Internet #

At once, there was a new world of hidden meaning and esoteric references, whose structure I was only beginning to comprehend. Code could be controlled, tamed, be bent to your will. It could be made to sit, roll over and beg like a dog. I found it truly is a language with all that implies: a language made up of words with meanings that once again had weight and mystery and consequences."

Coder Career: Ben Read on Becoming a Senior Engineer

Fri, 01 Oct 2021 00:00:00 GMT

Ben Read is a Senior Software Engineer at PurpleBricks, specializing in JavaScript development. Ben joins TCC to discuss his journey from junior to senior, accidentally starting a web development agency, and the early days of the internet.

I had a lot of fun with my personal friend Cameron on this podcast episode.

Listen to it here"

Building a Multitenant Kubernetes System

Sun, 27 Oct 2024 00:00:00 GMT

I recently decided to delve a little deeper into Kubernetes by building my own multitenant platform using Capsule.

This isn't meant as a tutorial really, I just want to document my learning so far.

Firstly, I must say that my explorations of Kubernetes and friends would have been a lot more challenging if if hadn't been for Warp, a terminal emulator that has great multiplexing and a built-in AI trained on the publicly available docs of relevant tools.

I set myself the following key requirements:

To utilise baremetal or virtual servers, not tied to one service or hosting company
To isolate each environment so they can't accidentally visit each others' sites
To facilitate easy creation, deletion and "pausing" of the environment
To use a custom domain for each

That last one is a bit more of a challenge since I am not all that familiar with load balancers yet, but we'll get there...

Reproducability

I had my own set of requirements to fulfil alongside these that I thought would help, most of these revolve around reproducability.

I wanted to be able to destroy and spin up new environments easily. That meant using the Helm package manager, Helmfile. With this tool I can declare the system I want, which would also help to roll the system back to a previous state more easily.

I also wanted the environment to be built with Infrastructure-as-code. A machine can easily become corrupted or attacked. For these reasons, I am much more inclined towards thinking about them as disposable tools than precious investments.

So here's the Helmfile I cam up with (with a little help from Warp):

repositories:
  - name: clastix
    url: https://clastix.github.io/charts
  
releases:
  - name: capsule
    namespace: capsule-system
    chart: clastix/capsule
    version: 0.4.6
    values:
      - capsule:
          config:
            forceTenantPrefix: false
            protectedNamespaceRegex: "^(kube-system|kube-public|kube-node-lease|kube-kube-.*|default|capsule-system)$"
            protectedNamespaceLabels: []
            enableTLS: true
            logLevel: info
          ingress:
            enabled: false
            hostName: capsule.local
            annotations: {}
            tls:
              enabled: true
              secretName: ""

        resources:
          requests:
            cpu: "100m"
            memory: "128Mi"
          limits:
            cpu: "200m"
            memory: "256Mi"

          nodeSelector: {}
          tolerations: []

          additionalLabels: {}
          additionalAnnotations: {}

          repositories:

I had a voucher from Linode (Linode is quickly being rebranded as Akamai Connected Cloud), which I was grateful for since that helped me get familiar with the difference between nodes and pods: nodes are external resources such as my Linode VMs, pods are internal resources. The number of nodes you have depends on how many VMs you have (on Linode at least, one will be a controller node from which to manage the others).

However I was afraid of running out of trial credits, so I also deployed this environment locally using Minikube. I have to say, this is a great tool for trialling software locally. It deploys it's own VM (which shows up in VirtManager on Linux), and it also creates the Kubeconfig for you automatically.

So I guess I'd better explain Kubeconfig.

How to manage Kubernetes

Kubernetes is the thing that manages your pods. To manage Kubernetes, you need Kubeconfig, a command-line tool that you can use to authenticate yourself and apply configuration to your Kubernetes namespace.

A namespace is a group of resources. In my system, the main namespace is minikube, then there's my Capsule namespace which manages resources for my multitenant system. And of course, each of those tenants has it's own namespace too.

You can find out what namespaces you have by running the following:

kubectl get namespaces

Each time you want to change a resource you have to load the correct namespace, this is usually a switch or flag on the command line like

kubectl apply -f ./values.yaml --namespace=capsule-system

Or like this

kubectl apply -f ./values.yaml -n capsule-system

This is telling Kubectl to apply the values from the file (-f) that follows to the namespace (-n) that follows.

As you can imagine this gets a little confusing because you can easily forget which namespace you're in. Especially when I was setting up my first tenant and was trying to run two kubectl configurations in one terminal session. Yeah don't try that.

To find out what user you are you can run

kubectl config view --minify

The Capsule system

What I liked about Capsule is that it has strict namespacing. At first I tried to call the Capsule system something else, and that got me in a tangled mess. It seems you need to call it "capsule-system" or else it doesn't work. But that helped me because I could more easily see the distinction between other namespaces (I also tried to spin up a Kubernetes dashboard).

It also, like Kubernetes does natively, uses RBAC (role-based access control). This is similar to what you see in AWS: you need to create a user first, then assign them a role, and the permissions they need live inside that role, not with the actual user.

This is a useful abstraction because if your user is compromised you can withdraw their permissions which will preserve the resources and the permissions they need to function.

But it does take some getting used to. If your kubectl is set to use a tenant, they won't be able to do a lot unless you first load the capsule system user which has all the necessary permissions to grant those to your users.

kubectl get rolebindings,clusterrolebindings --all-namespaces -o wide

This is useful for finding out what permissions each user has.

You can imagine this can get a bit frustrating, especially if you're just trying to work this out.

I like to drop myself in the deep end like that.

However I did get more than a little frustrated that this whole thing is only partially declarative.

What I mean by that is that you have to perform the following steps to do anything significant:

Alter your YAML files to modify permissions
Apply the config
Check it worked
Undo your alterations if it didn't
Apply the config to roll it back

This gets tedious and more than once I forgot to un-apply my changes and therefore had to unpick things a few times.

For me it was like having committed to a remote Git branch and then only being able to see the effects on a hosted environment. It's a rather wide feedback loop that I want to shorten.

There's not a lot of documentation for multi-tenancy systems. I guess if you're building one you're not exactly going to shout it from the rooftops.

Another approach

I did find this documentation from another IAC platform I really like, Pulumi. It's been AI generated but might be a good starting point. I like the fact that I can programatically declare my namespace and tenants in one file:

import * as k8s from "@pulumi/kubernetes";

// Create a Kubernetes cluster using the preferred cloud provider.
// This is an abstract example; specifics would depend on the cloud provider in use.
const cluster = new k8s.Cluster("multi-tenant-cluster", {
    // Configure the cluster settings here.
    // For example, on AWS you would set the version and node type,
    // and the Pulumi AWSX library would provision an EKS cluster for you.
});

// Configure Kubernetes provider to use the generated kubeconfig from the cluster above.
const provider = new k8s.Provider("k8s-provider", {
    kubeconfig: cluster.kubeconfig,
});

// Create namespaces for each tenant.
const tenantA = new k8s.core.v1.Namespace("tenant-a", {}, { provider });
const tenantB = new k8s.core.v1.Namespace("tenant-b", {}, { provider });

// Now we might apply a Kuma installation to our cluster
// Note: Specifics would vary based on your use case and would likely involve
// custom configurations, which are beyond the scope of this program.
// We assume that we have a definition file `kuma-control-plane.yaml` that contains
// the resources to set up Kuma, including a Namespace, Deployments, Services, etc.
const kuma = new k8s.yaml.ConfigGroup("kuma", {
    files: ["kuma-control-plane.yaml"],
}, { provider });

// Export the kubeconfig to access your cluster.
export const kubeconfig = cluster.kubeconfig;

And yes, that's infrastructure as code with TypeScript. A significant abstraction but one I can get behind.

This is far from complete of course, there's a long list of things to set up besides this, which since there's no copyright on AI generated docs I can paste here directly in case it's taken down:

Define a ClusterRole and associated ClusterRoleBinding (or RoleBinding in each namespace) for each tenant.
Set up network policies to restrict traffic flow between the tenants.
Install Kuma following its documentation, tailoring the setup to your cluster's network configuration and the permissions required.
Ensure that your Kuma setup works with your multi-tenancy setup, e.g., by making sure that Kuma's control plane respects namespace boundaries and RBAC rules.

I think I'm going to try this again next time the kids are in the pool.

Next steps

I've got a long way to go still: I want to automate the creation of a very specific application setup for my tenants, and each of them will need to have an ingress, I know nothing about how load balancers work yet so I'm going to have to figure that out.

But this has been an enjoyable learning exercise.

Advanced Custom Fields and Bootstrap Tabs

Tue, 02 Jun 2015 00:00:00 GMT

I'm not a huge fan of Advanced Custom Fields, but there was a requirement to use it in a recent project that had Bootstrap as a basis for the UI. The challenge for me was to get Bootstrap nav-tabs to play nice with an ACF repeater field.

I started with the basic HTML markup for Bootstrap's Nav Tabs:

<ul class="nav nav-tabs">
  <li role="presentation" class="active"><a href="tabone">TabOne</a></li>
  <li role="presentation"><a href="tabtwo">TabTwo</a></li>
  <li role="presentation"><a href="tabthree">TabThree</a></li>
</ul>
<div class="tab-content">
  <div class="tab-pane active" id="tabone">
     Some content in tab one
</div>
  <div class="tab-pane active" id="tabtwo">
     Some content in tab two
</div>
  <div class="tab-pane active" id="tabthree">
     Some content in tab three
</div>
</div>

In the Field Groups settings, I created a Repeater (this is a paid-for add on to the standard Advanced Custom Fields) called "tab Panes", with 2 sub-fields, "Tab Title" and "Tab Contents".

<?php
<!-- Check for parent repeater row -->
<?php if( have_rows('tab_panes') ): ?>
  <ul class="nav nav-tabs" role="tablist">
  <?php // Step 1: Loop through rows, first displaying tab titles in a list
   while( have_rows('tab_panes') ): the_row();
?>
    <li role="presentation" class="active">
      <a
        href="#tabone"
        role="tab"
        data-toggle="tab"
        >
      <?php the_sub_field('tab_title'); ?>
      </a>
    </li>
    <?php endwhile; // end of (have_rows('tab_panes') ):?>
  </ul>
<?php endif; // end of (have_rows('tab_panes') ): ?>

The PHP above displays the tabs. The code below, very similarly, displays the tab panes:

<?php if( have_rows('tab_panes') ): ?>
  <div class="tab-content">
  <?php// number rows ?>
  <?php // Step 2: Loop through rows, now displaying tab contents
   while( have_rows('tab_panes') ): the_row();
  // Display each item as a list ?>
      <div class="tab-pane active" id="tabone">
          <?php the_sub_field('tab_contents'); ?>
      </div>
      <?php endwhile; // (have_rows('tab_panes') ):?>
  </div>
<?php endif; // (have_rows('tab_panes') ): ?>

By looping through the same repeater, we can get all the tabs out of the database, no problem. But we still have two problems: 1) linking the tab to the pane 2) Assigning the class of "active" so the Javascript is able to add and remove the CSS to reveal / hide the appropriate pane.

1. Linking to the Pane

There are a number of ways to do this. I could ask the user to input a number to uniquely identify the tab pane. But that would add extra work to the users flow, and they might easily find themselves out of their depth. I want to make this as easy as possible for the user.

On the other hand, Wordpress has a very useful function called Sanitize HTML, which we input the value of the title, take out spaces and capitals, and use this as the link:

<a href="#<?php echo sanitize_html_class( the_sub_field( 'tab_title' ) ); ?>"

2. Assigning the 'Active' Class

So now we need to get a class of 'active' only on the first tab. The Bootstrap Javascript will do the rest for us. How do we do that?

I added this code just inside the while loop, inside the ul tag:

<?php $row = 1; // number rows ?>

This php is a counter. So we can identify the first instance and assign an if statement to it.

<a class="<?php if($row == 1) {echo 'active';}?>">

The final thing to do, is to keep the counter running, but adding this just before the endwhile.

<?php $row++; endwhile; // (have_rows('tab_panes') ):?>

Once you've added these to the tab panes in a similar way, you'll be up and running with Boostrap Tabs.

Full code:

<?php if( have_rows('tab_panes') ): ?>
  <div class="tab-content">
  <?php // Step 2: Loop through rows, now displaying tab contents
   while( have_rows('tab_panes') ): the_row();
   $row = 1; // number rows ?>
  // Display each item as a list ?>
      <div class="tab-pane <?php if($row == 1) {echo 'active';}?>" id="tab">
          <?php the_sub_field('tab_contents'); ?>
      </div>
      <?php $row++; endwhile; // (have_rows('tab_panes') ):?>
  </div>
<?php endif; // (have_rows('tab_panes') ): ?>
```"

A Collection of Tailwind UI Loading States

Wed, 04 Oct 2023 00:00:00 GMT

I made a collection of loading states for Tailwind. It was relatively easy exercise, and I'm surprised I didn't find anything like it out there already.

You might recognise the first element from the Tailwind Docs, which demonstrates how to use the animate-pulse class to make it look like it's in the process of loading in.

Here's the CodePen:

<p class="codepen" data-height="300" data-default-tab="html,result" data-slug-hash="eYbPBjE" data-user="endymion1818" style="height: 300px; box-sizing: border-box; display: flex; align-items: center; justify-content: center; border: 2px solid; margin: 1em 0; padding: 1em;"> <span>See the Pen <a href="https://codepen.io/endymion1818/pen/eYbPBjE"> Tailwind Loading UI Elements</a> by Ben Read (<a href="https://codepen.io/endymion1818">@endymion1818</a>) on <a href="https://codepen.io">CodePen</a>.</span> </p> <script async src="https://cpwebassets.codepen.io/assets/embed/ei.js"></script>

Feel free to copy or remix this pen, I'm not precious about it and just wanted a collection I could reference for some other work I was doing.

Comments are now live

Thu, 21 Mar 2024 00:00:00 GMT

A system to allow visitors to comment on my blog posts has just gone live.

It's amazing to me how quickly the internet is being eroded. It's been under way for years, but with generative AI technology, it seems to have accelerated beyond anyone's ability or desire to fix it.

Are We Watching the Internet Die?

It increasingly looks like there will be two internets out there in future: the Big Tech internet, where AI generated content is indistinguishable from that made by humans. It's a homogenous place where ad tech and cookie banners flourish.

The other is a return to personal websites, webrings, manually curated directories, and ... just generally the web we almost lost.

And the ever increasing mass of AI generated goop? They can stay outside. We don't serve their kind here.

I don't feel bad about this. It's kind of exciting to wake up and find the landscape around you quite changed.

<div class="[&>pre]:ml-0! [&>pre]:mr-0! [&>pre]:bg-transparent!">

April is the cruellest month, breeding
Lilacs out of the dead land, mixing
Memory and desire, stirring
Dull roots with spring rain.

- TS Eliot

</div>

But anyway, in eager anticipation of a revival of sorts, I've added a comments section to my website.

Now I've got that, I'm planning to add a guestbook and to join more directories and webrings.

So, please do leave me a comment! You don't need to be logged in, and once I approve and rebuild the site, it'll be live for all to see albeit without any attributing email address.

Good morning. It's a brand new day out there.

PS If you're looking for some of those directories, here are 3 I have found useful:

https://blogroll.org/
https://indieseek.xyz/
https://ooh.directory/

Comparing two redux setups

Mon, 03 Aug 2020 00:00:00 GMT

I recently came across two different examples of setup that used Redux to manage state in a sample app. They were so different that I thought it was worth delving a little deeper into why, and how we can write Redux code so it's a good fit for our needs.

Minimum Viable Redux #

Redux used to be pretty much the only way of managing state in React without "prop drilling" or mutating state (which means changing it so that the original state is lost) as you go. There are a bunch of different options now, but Redux is still in common use. However that use doesn't always have a common pattern. Take this example I found on a JSFiddle by Caner Dagli:

import { Provider, connect } from "react-redux";
import { createStore, applyMiddleware } from "redux";
import thunk from "redux-thunk";

function fetchPostsRequest() {
  return {
    type: "FETCH_REQUEST",
  };
}

function fetchPostsSuccess(payload) {
  return {
    type: "FETCH_SUCCESS",
    payload,
  };
}

function fetchPostsError() {
  return {
    type: "FETCH_ERROR",
  };
}

const reducer = (state = {}, action) => {
  switch (action.type) {
    case "FETCH_REQUEST":
      return state;
    case "FETCH_SUCCESS":
      return { ...state, posts: action.payload };
    default:
      return state;
  }
};

function fetchPostsWithRedux() {
  return (dispatch) => {
    dispatch(fetchPostsRequest());
    return fetchPosts().then(([response, json]) => {
      if (response.status === 200) {
        dispatch(fetchPostsSuccess(json));
      } else {
        dispatch(fetchPostsError());
      }
    });
  };
}

function fetchPosts() {
  const URL = "https://jsonplaceholder.typicode.com/posts";
  return fetch(URL, { method: "GET" }).then((response) =>
    Promise.all([response, response.json()])
  );
}

class App extends React.Component {
  componentDidMount() {
    this.props.fetchPostsWithRedux();
  }
  render() {
    return (
      <ul>
        {this.props.posts &&
          this.props.posts.map((post) => {
            return <li>{post.title}</li>;
          })}
      </ul>
    );
  }
}

function mapStateToProps(state) {
  return {
    posts: state.posts,
  };
}

let Container = connect(mapStateToProps, { fetchPostsWithRedux })(App);

const store = createStore(reducer, applyMiddleware(thunk));
ReactDOM.render(
  <Provider store={store}>
    <Container />
  </Provider>,
  document.getElementById("container")
);

I really think this example is great, it shows us how to write the most basic boilerplate code for a Redux store and how you might use it for a simple use case, one data type and a single action.

So does Redux require a lot of boilerplate code? Not in this example. However, if you want to be able to scale it to use different data types (think posts and users and tags and pages ...), the above code isn't going to scale.

Redux at scale #

Take a look at this example, it's a GatsbyJS starter but I hope you can still find your way around the code: https://github.com/Evaluates2/Gatsby-Starter-TypeScript-Redux-TDD-BDD

If you peek inside the /src/ folder, you'll see another folder, /state/, with another 4 folders, /actions/, /middlewares/, /reducers/ and /types/.

Just look inside the /actions/ folder and you'll see four files. Two are tests, but there's an action for each data type (login and todos respectively). Similarly, there's a reduce for each data type, middleware for each type ... etc.

This code is built for extensibility. By compartmentalising code like this I can see that I will be able to add another type (like lists or pages or transactions) without having to make fundamental changes to how this project is set up. Smiliarly, if something changed in my todos, I can see more easily where I need to go to change something so that the app still renders in the way I expect it to.

I liked this example because it shows us a totally different way of writing Redux code. It helps us see the separate concerns we need to care about (reducers, actions, middleware even) if we are going to write a robust application that will be able to serve many product iterations over a longer period of time without major refactors.

I think these two approaches also teach us that Redux is a powerful suite of tools that help us manage state in our applications — and that it can be used in different ways depending on your desired approach.

But they also tell us something about software development: the fastest approach is great for some small project you're only going to touch once. But architecting an application that will withstand the test of time is a different game altogether.

That's the great thing about the way Redux is built: it doesn't get in the way of writing new code ... providing we start out in the right way."

A congratulatory message with hidden meaning

Wed, 30 Nov 2022 00:00:00 GMT

One of the objectives I set out to achieve at Webiny was to increase the positivity among members of our audience in the hopes that they would become more vocal participants of the community and join in a bit more with our discussions.

To achieve that I wanted to create messaging that would be shared primarily on the company newsletter to emphasize how much we valued our community members and to thank them for being there.

This is the artwork I came up with:

I really liked how this turned out for a few reasons: it was a clear message and wasn't too cluttered and it only too me a few minutes to put together. But I liked it for other reasons too.

Although the primary message in the emoji icons says "you" and "awesome", the message is coming from us (ie. inside the screen), so in fact we are identifying ourselves as the "superhero" pointing out towards the audience. The subliminal message is that we are awesome because we're enabling you.

Also by featuring the "Webiny" logo behind the superhero figure, we are further emphasizing that the power to be awesome comes from Webiny.

This artwork was sent out a few months ago as the last segment of our monthly email newsletter. I particularly wanted to feature it at the end of the newsletter as an unexpected twist and a pleasant surprise for anyone who got that far through the newsletter. I don't know if anyone caught onto this messaging but that's not the point; by continually sharing such positive thoughts, positivity towards Webiny will inevitably result, and the result of that is organic growth.

I thoroughly enjoy making these little surprises that reward people that notice and make them feel valued."

Content Authoring with StrapiJs

Sat, 18 Aug 2018 00:00:00 GMT

For years, WordPress has been the de facto standard for content authoring on the web. The reasons for this are many, but principal among them is it's clean, easy-to-use interface that gives content authors control over nearly every aspect of their content. WordPress isn't going away, but hosting and maintaining it safely can be a challenge.

At the organisation I work for, we have a WordPress blog, but almost no PHP expertise as we were moving from Ruby towards a JavaScript stack. We also have another service that used a WordPress-like interface, but were tied into expensive hosted services - which was deprecated, and yet still fulfilled an essential function as a core part of our content strategy.

For this reason, I started to investigate a way of amalgamating this content into one location, on one platform (preferably JavaScript), with an API so that I could pull in content to our static site built with Gatsbyjs.

The Elusive Dashboard #

There are lots of services out there that could have fulfilled our requirements, but all of these stored content on another platform, an approach we patently wanted to avoid. For that reason, services like Forestry, Contentful, and even Sanity with its installable api and hosted backend, were out of the equation.

Instead I looked into dashboard / content authoring UIs that we could build an API around. I saw that Meteorjs had an open-source project in this vane, but that there really weren't that many active projects on NPM that we could utilise.

Then I discovered Strapi.

A Content Interface #

Strapi quickly started looking like it was a product that could fit our requirements. A self-hosted, API-focused system with a well designed dashboard and a useable interface. I particularly liked that we could create content types on the frontend, designate the fields, and start using them straight away.

Strapi showed promise, however version 1 was depracated, v2 hadn't made it out of the door, and v3 was still in Alpha release stage. This is a cause for concern, however the team were progressing with a good deal of focus on fulfilling key user needs. I decided I would get involved in the project and start using it for personal work.

Deploying Strapi on Heroku #

Since my focus over the past few years has been Severless technology, I wanted to deploy Strapi on Heroku, a managed platform with a generous free tier that allows you to host applications in a raft of different languages, from PHP and Ruby, to Go and JavaScript.

Fortunately for me, there's already a decent Strapi starter for Heroku, which is advertised right on the Strapi repo readme. One click and I had started my own Heroku instance, using mLab to host the MongoDB database, all connected up and ready to be used.

This is great for giving Strapi a spin, but I wanted this to be my content platform. To do that I needed to create my own fields and add some new content types. I added some, only to find they'd rolled back again afterwards. I tried uploading images, to find they disappeared after a few hours. What was happening?!

What I hadn't realised is that Heroku's file system is ephermeral, which means data stored there won't persist. You application's instance can be destroyed and rebuilt from the git store depending on Heroku's, or your, needs. So making changes to the API, storing images locally, or even data, needs to be done differently.

The core issue was that I needed to clone the site locally, add my changes, then to Heroku's Git repo, and push it to Heroku's repo.

Cloning Locally #

Cloning the Heroku build pack so you can work on it locally is a bit tricky. I found this helpful guide and worked through it, and eventually I was able to spin up my Strapi app locally.

I work on Linux, and I had to be careful when installing MongoDB locally. This is because I needed to add MongoDB via the APT package manager, and I inadvertently tried to install a version meant for older Linux OSes. So I broke my computer and had to re-install the OS.

After that, I decided to stick with the remote service mLab until I can decipher the Docker documentation and get it all running together inside a container.

Once done, I was able to add content types and fields, and add Cloudinaryas an image storage provider, and push my changes to Heroku.

Querying the API #

One last hurdle remained, which was to query the API from GatsbyJS. To start with, I found I kept geting 401 Unauthorised notifications. This is because every content type you define is private by default. You have to enable the public user, or submit auth details, in order to query the data. Here's a handy guide on how to do that.

That was it, I now had a fully functional API hosted for free on Heroku. I could start the app (Heroku's free tier means that you app will sleep until its needed), run Gatsbyjs, and get my data using the gatsby-source-strapi plugin.

Strapi - Ready for Production? #

As I write this, Strapi is moving steadily towards an initial Beta release. It's positioning - a self-hosted API dashboard and content authoring platform - is pretty much unique in the JavaScript world. I'm using it to store content for my new personal site https://discovermikeoldfield.info, and I'm putting it forward at work as a replacement (and enhancement) over those separate, difficult to maintain or retired APIs I mentioned at the start.

However, this is a product that's very much in development, and there are certain inconsistencies to watch out for. Plugins and even features may change frequently. I had some serious issues with the date picker in the version I was using, and I couldn't get the Cloudinary plugin to work so I instead created a text field and uploaded my images to Cloudinary independently.

However, it still met my requirements and proved to be a product I recommend investigating, and using if it meets your requirements. If you are after this type of thing I ask you to be generous with your support for the time and money the contributors have put into it. It takes not only serious programming chops but also a good investment in design for such a thing to be useable, and I think the Strapi team have achieved that."

Continuous Deployment with Netlify and Hugo

Thu, 18 May 2017 00:00:00 GMT

Over the past few days, I've moved tech stack entirely for deliciousreverie.co.uk, from reseller hosting using PHP to cloud hosting on AWS by Netlify, and using continuous deployment. This post details some of the hazards and benefits I encountered.

Since I created this blog, I've always been a very proud user of Perch CMS and have recently upgraded to Perch 3, which brought loads of enhancements as well as an improved interface. So why the switch?

The truth of it is that I've been paying for reseller hosting with my previous provider, UK Web Solutions Direct, for 7 years. They've always been quick to respond to my support questions, and I'd highly recommend them.

However, since I gave up my freelance work, it came to the point where I only had this website hosted on it, which seemed a bit of a waste. I also wanted to try out Netlify's continuous deployment service as an alternative to FTP.

Building a Hugo Site #

I felt that Hugo was fairly easy to get a grasp of. Once you get around how strange Go templates look initially, I grabbed the theme called base from the Hugo theme repo and began rolling my own.

I've since found that Base theme isn't maintained and doesn't contain some of the new shiny that's present in Hugo 0.20. Just a caution if you were thinking of doing the same. Sadly, there isn't yet another theme that can replace it to give you a quick leg-up without any CSS already present in the theme.

The hardest thing I found when using Hugo was ... how to build my site. Hah, yes it's hilarous, but having used commands like hugo server and hugo new site and hugo new theme it totally passed me by that to actually build my static site, all I needed to do was run hugo.

There you are. Laugh all you want. I'm maybe being a little bit dense, but I think it's counterintuitive. Why not hugo build or something?

So, I had my theme and set up a Netlify repo. Next was deployment.

Deploying to Netlify #

Netlify's interface makes deployment pretty painless. Once I authorised GitHub on my account, it slurped the code, installed dependencies, ran the hugo build process ... and failed miserably.

It failed because currently, by default, Netlify seems to default to an older version of Hugo (v0.16 or something) which meant I was getting build failures.

I eventually found out that you need to add a netlify.toml file in the root directory to specify the Hugo version you need:

[context.production.environment]
  HUGO_VERSION = "0.20"

That seemed to work! I switched my DNS over to Cloudflare and the site went live!

Using Pygments on Netlify #

One thing I noticed post-launch was a that a Python dependency I had started to use wasn't working on Netlify. My code blocks would render as plain text, making them unfathomable and messy.

I came across this post - entitled Hugo on Netlify - which helped me to see I could add that dependency to Netlify's build process by adding another config file, requirements.txt to the root, and adding Pygments==2.1.1 as the content.

So I ran hugo again, added the commit, and waited ... and waited. Quite a long wait this time actually. Pygments does slow Netlify down quite a bit. But the alternative to Pygments is a Javascript library, which would slow down my users. So I don't mind taking a bit of a hit to save them the extra load time.

HTTPS #

I initially had trouble setting up by DNS with Cloudflare and applying Netlify's HTTPs certificate. It turns out, after having talked to Netlify, that they don't have IPv6 yet. Cloudflare adds an AAAA record to your DNS which messes up the HTTPS somehow.

Once I'd disabled Cloudflare's CDN, I reapplied the certificate and everything worked fine, and I was able to secure my content.

Aims #

My site now loads in 1.526 seconds and serves 182kb of content in 8 requests. I'm going to continue to refine this so that hopefully I can find out how to preload fonts and the background image, inline my CSS and JavaScript, or find other improvements that'll hopefully bring this down as much as I can.

I also aim to add commenting via Staticman, contact forms using Formspree, and Forestry as a CMS so that I can author posts online.

Ahh, static sites aren't so static after all, are they?

Verdict #

As the web becomes more bloated, I really believe static site generators are going to be the way to get ahead. Once you add the previously mentioned services, they can adequately compete with some of the lumbering, huge CMSes out there (of which Perch isn't one, I hasten to add).

Continuous delivery can a bit cumbersome to manage for some projects in my view. I say this because my build and production environments were different and that caused me a few issues. I'm on the free Netlify plan, which means I get a lot less in terms of build previews. Looking through their documentation, it seems there's a lot more available on their paid tiers.

That being said, it's certainly a lot more safe than the somewhat shakier transfers of FTP/SFTP. One of the benefits became real to me when a friend of mine submitted a pull request, but I was able to see before I merged the code that the change would've broken my site, resulting in down time. That was a really useful feature. Now I know I can change my theme, and even if it succeeds locally, if it fails on production you won't break your site, which I know I've done a few times.

Addendum #

This article was edited to remove the references to custom headers, which aren't yet available on Netlify yet."

Creating my homepage animation

Sat, 21 Nov 2020 00:00:00 GMT

I made the original animation for this site when I was using Hugo, but it's taken some time to refactor to React. Here's the story of how I achieved that.

I wanted this site to be pretty minimal, but a few nice little touches can really make a website stand out. That's why I made the animation here on the homepage. You have to scroll up & down slowly to see it ...

First steps #

The original animation was created with both ScrollMagic and GreenSock, two of the most incredibly diverse animation libraries available. The combination of these two is I think the easiest way of creating JavaScript animations when you're scrolling around, and I have used them to provide some pretty nice effects on sites I've worked on before.

However, the bundle sizes of GreenSock and ScrollMagic are ... quite large. I wanted to rewrite it because (1) I care about how much JavaScript I'm sending down (2) I want to see what I can achieve in plain JavaScript (3) I have the luxury of having no deadlines.

Here's the original animation on CodePen (137 lines)

Animations, especially ones where you're telling a story like this one, depend much on timing. I was particularly happy with the specific points that each rabbit became visible in this iteration, especially the last three, where the rabbit appears to be listening out for something, before dashing off the screen. It took a considerable investment of hours to get this right, so I've more or less kept this though the other implementations.

I've been told that this animation isn't prominent enough for most people to notice it. But I think that's partly the point: I like the idea that most people might not notice there is an animation: after all, most people aren't looking for that when they come to my site, so I didn't want to distract further their already distracted minds.

However, for those that do stop to notice, the brevity almost emphasises the subject matter more: it's only when we slow down ourselves that we start to notice the wonderful little things that are going on in nature around us.

The SVG images are free ones I found after trawling through a huge raft of websites selling, giving away, and pretending to give away, svgs.

Rebuild in JavaScript #

For this re-implementation I made one significant change: the opacity of each rabbit is controlled by CSS, and doesn't fade in and out whilst you're scrolling. It's a a little bit of a cheat but I quite like how it makes the effect a little more dream like:

See it on CodePen

This is only 35 lines, and without any libraries, it's much faster and better for users.

Rebuild to React #

When I switched to React I was conscious that my JavaScript bundle had increased quite a lot, and I wanted to try to make compensation for that by implementing some other performance gains before I started getting fancy.

It took some time to get this to work in React. Building this honestly got me to question whether React is "just javascript".

The actual animation code is 43 lines, a significant step up. There were a few tricky things I had to work out too:

View on CodeSandBox

React re-renders the page at different stages, so needs a way of getting the current element that's in the DOM. The useRef() hook allows us to do that. There's a caveat with that though: the initial value of current is undefined, it wasn't until I realised I needed to get the elements first then call them inside of a useEffect() hook.

It was also easier to use a pre-built hook, react-use-scroll-position, rather than write my own code. It's pretty minimal so I don't mind that too much.

TypeScript implementation #

I'm using this one on my homepage now, but with TypeScript:

const Rabbits = () => {

  if(typeof window === 'undefined') { return <></> }
  if(window.innerWidth < 998 ) { return <></> }

  const scrollYPosition = useScrollYPosition()

function getScrollPercent() {
    return (
        scrollYPosition || document.body.scrollTop) / (
          (document.documentElement.scrollHeight || document.body.scrollHeight) - document.documentElement.clientHeight
      ) * 100;
}

  const useScrollHeightToChangeOpacity = (
    domElement:HTMLDivElement,
    inHeight:number,
    outHeight:number
  ) => {
    if (getScrollPercent() > inHeight &&  getScrollPercent() < outHeight){
      domElement.style.opacity = '0.8'
    } else {
      domElement.style.opacity = '0'
    }
  }

  const treeSummer = useRef(null)
  const rabbitOne = useRef(null)
  const rabbitTwo = useRef(null)
  const rabbitThree = useRef(null)
  const rabbitFour = useRef(null)
  const rabbitFive = useRef(null)
  const rabbitSix = useRef(null)
  const rabbitSeven = useRef(null);

    useEffect(() => {
      useScrollHeightToChangeOpacity(rabbitOne.current!, 5, 10)
      useScrollHeightToChangeOpacity(rabbitOne.current!, 8, 15)
      useScrollHeightToChangeOpacity(rabbitTwo.current!, 20, 25)
      useScrollHeightToChangeOpacity(rabbitThree.current!, 30, 40)
      useScrollHeightToChangeOpacity(rabbitFour.current!, 45, 60)
      useScrollHeightToChangeOpacity(rabbitFive.current!, 65, 80)
      useScrollHeightToChangeOpacity(rabbitSix.current!, 85, 90)
      useScrollHeightToChangeOpacity(rabbitSeven.current!, 90, 95)
    }, [getScrollPercent]);

  return (
      ...
  )
}

The hardest part of the TypeScript implementation was to find out how to get the .current value of useRef(), because the initial value as you can see is null (the dom element doesn't exist yet). Unfortunately the only help you get from the compiler is:

Argument of type 'null' is not assignable to parameter of type 'HTMLDivElement'

To tell the compiler that it's not possible for this value to be null at this point, you append it with a !, the non-null assertion operator.

Annoying, but not annoying enough for me to change my position on TypeScript.

Future plans #

If you have to ask why I chose this particular scene, it's because it reminds me both of my childhood days roaming the countryside of County Durham when such wildlife was abundant, and the film "Watership Down" (I watched that film when I was far too young!).

Originally, I planned ot have different animations on other pages too, like a bird or butterfly crossing the viewport, and a little girl sitting on a tree swing. I'm still thinking about whether or not to implement those, since the SVGs are quite large and it might not be good for my bundle size.

However I do really like how much character it lends to the site."

My Recipe for Crispy Beef

Thu, 28 Jul 2022 00:00:00 GMT

I've tried several recipes for crispy beef, and have finally gained confidence enough to make it on my own terms. Here's a simplified version that I now follow. It's quite rough, and it's meant to be adapted.

The trick with crispy beef is to keep it dry enough so that it will firm up on cooking. This also depends on getting enough heat for it to caramelise.

Ingredients:

Minced beef
Keçap manis (or a syrupy mix of dark soy sauce and brown sugar)
Sesame oil
Corn flour

This isn't a comprehensive list. You can add other things as you go like red peppers, spring onions, sesame seeds, chilli flakes.

Step 1 Mix beef #

Mix the beef in some corn flour with your hands until its dry and flaky. Add a good squeeze of keçap manis (about 200ml). Keçap manis is basically just dark soy sauce and sugar, so don't feel compelled to buy this item if you want to mix your own.

By the end of this mix, you should have a disgusting moist mixture. Don't worry about the consistency at this stage, things will improve with the application of some heat!

Step 2 Cook #

Its important to heat the sesame oil before putting in the beef. This is so the oil doesn't soak into the meat but instead heats up the sugar until it caramelises. Put in the whole mixture and keep stirring and mixing until it caramelises.

Gradually the mixture will darken and separate. Take it off the heat before it starts burning but when it's sufficiently dark and crispy."

Deliverables and Trust

Fri, 27 Feb 2015 00:00:00 GMT

It's vitally important to have success before you start a project-perhaps the most critical stage to have success. But how far should we go in order to get our clients on board? Should we try to "wow" clients into signoff? Here's why that is a bad idea.

So you're taking this proposal to your new client. You sit down in the boardroom, a little nervous but trying to hide it. You naturally have done everything in your power to ensure the success of the meeting. What will win over this set of decision makers and stakeholders to invest in your proposal?

I can see why we are so tempted to take in visual designs as deliverables to these early meetings. It's only natural to want to impress ... but there's a danger lurking at this tender stage of the project which could ruin the conclusion, and cause stress and frustration for every team member in between.

Swaying the opinions of stakeholders is vitally important. But I argue there are 2 ways of doing it. We can win stakeholders over with demonstration of genuine professionalism, interest in their affairs, and understanding of their business needs.

It doesn't take much for a client to feel let down. Unmet expectations are a slippery slope.

So take time to get to know their assumptions. Ask these kinds of questions:

Why do you want a site redesign?
What are your business goals for this project?
At the end of the project, what would you consider to be successful?

At this stage it's not just the business goals that are important. Depending on the client, it's beneficial to try to train them to see things from their users perspective. Help them to see that their new site is for their users—new and repeat customers. If they are aware of that, if you're an advocate for the user in these meetings, you demonstrate professionalism and help them see that you have business goals in mind.

Deliverables and trust are difficult to negotiate on. But if we avoid "wowing" our clients and instead seek to understand them, the outcome can be much better—for us and for them."

Delaying click event in vanilla JavaScript

Tue, 22 Mar 2016 00:00:00 GMT

I was inspired by Rachel Nabors' site to add a short delay to users' clicking on a link on my website, so that I could add a brief animation that would run before the redirection took place. Here's how I did it.

"Animation" seems to be the theme of my year. Animation is what I think is going to be one of the big differentiator on websites in the coming months and years. And I really, really love the CSS animations API.

I've used waypoints.js a lot lately so that I can run animations at different stages during the time that a user scrolls around my site. But for my personal blog, I wanted to do something a little trickier.

My Requirements

I wanted to fade in page elements when the user arrives, and fade them out again when they left. I know that it could potentially have been easier to write this in jQuery using the delay() function, but I decided to go the route of a custom bit of JavaScript. It was a good practice exercise for me, and I don't want to load the jQuery library here unless I have to.

The Code #

It took me a while to come up with the strategy for this. I started looking at onpageunload(), hoping to add a delay to that. But that's quite locked down, and didn't work for my requirements.

After I had fiddled for a while, I ran up against a brick wall. For some reason, my JS event listener wasn't working properly.

So I turned to StackOverflow and posed the question, which helped immensely, and validated what I was already trying to do. The thread is here if you'd like to read it.

Step 1 #

Reading some other posts on Stack Overflow helped me see that I needed to grab all of the <a> tags into a variable, then listen for an event.

var links = document.getElementsByTagName('a');for( var i=0,il = links.length; i< il; i ++ ){links[i].onclick = clickHandler;}

Step 2 #

Then, I needed a function called clickHandler that would fire on click:

function clickHandler(event) {

event.preventDefault();

var travelTo = this.getAttribute("href");

The first job of this handler was to stop the link from being followed, using preventDefault(), then to grab the link the user was going to so that we could use it later on.

Step 3 #

The next step was to add my animation class to the elements I wanted:

var animOut = document.getElementsByClassName("animateOut");
// iterate `animOut` 
elementsfor (var i = 0; i < animOut.length; i++) {   
  // add `out` `className` to `animOut` element at index `i`   
  animOut[i].classList.add("out");
};

This looks for each element that has a class of animOut, and adds a further class, out when the function runs.

Step 4 #

Next, we needed to send the user on their way using the travelTovariable we stored earlier and using the setTimeout function to add a bit of a delay:

setTimeout(function() {  window.location.href = travelTo;}, 1000);

I played with the durations quite a bit. It was important for me not to make the animation and/or the delay too long because I thought it would probably be distracting to the user. I reduced it from 1 second to just half a second for this very reason.

The Result #

Well, click on a link and see it working!!

It was great to be able to work this out, and the Stack Overflow community once again proved to be a fantastic resource.

I must also credit Rachel Nabors for inspiring me, since I first saw this being used on her website.

Here's the full code in a Gist if you'd like to adapt it for your site:"

On Having Delusions of Grandeur

Tue, 02 Jun 2015 00:00:00 GMT

I'm not as good at this as I thought I was. This much became apparent to me recently, when I tried to run with some industry leaders and found that my abilities in web development are still comparatively lacking.

As developers, we find no comfort in the thought that we'll never know enough about the languages we use, the frameworks we work in, the libraries we utilise and the platforms that underpin our whole industry. It's impossible to absorb and retain all of this information.

What really shook me up was discovering the W3C Validator tool, and finding out what I've being doing wrong all this time. I've started to retroactively go through my sites and fix the errors that were revealed, but it still smarts that I've been ignoring these standards for far too long.

It doesn't help either that I wasn't even aware of them.

What I take away from this experience is that I need to "stay foolish", and instead of having delusions of grandeur, be prepared to take knocks like this as jolts of advancement. I realise too that I want to improve—I'm not content to go through life without continuously improving my professional abilities.

And that, I think, can count for something.

Deploying Nx monorepos to Netlify in 2023

Wed, 31 May 2023 00:00:00 GMT

Monorepos with Nx on Netlify has been my stack of choice over the past few years. I really enjoy using both tools to publish an ever-growing series of websites for my hobbies. But there have been a few changes to both Nx and Netlify recently. This article updates one on the Netlify blog that's a little out of date.

Here's the original article, which is still really useful and goes into a lot of background as to why you might want to use a monorepo to build your frontends. It's a great article and I recommend you read the first half just to get some background. But later in the article the author for some reason digresses into setting up a CircleCI pipeline. Personally I don't find a lot of value in doing that for my frontends, and I think it distracts from the core subject a little.

Nx Cache

The first thing I wanted to mention that has changed is Nx aggressive caching. Their Nx Cloud plugin automatically pushes builds to their cloud without any configuration, making it accessible to your pipeline in Netlify. As this Netlify support article shows, you might see in your build logs "0 new files to upload". This means your build has been diffed with what's stored on Nx Cloud and there were no changes.To get around this you can append --skip-nx-cache to your build command.

So do we need a build plugin at all? In fact, we don't unless you hit some edge case where you cannot use Nx Cloud. This is probably the only reason you might want to write your own implementation.

To utilise Nx Cache, run your deploy command as usual. It will discover if there have been code changes and deploy based on those changes. Otherwise it will skip the build and deployment steps. If you still want to deploy (perhaps you have changes in an external CMS, something that won't be detected on build), then add `--skip-nx-cache` to the build command and re-run it. You should then see it has fetched fresh data from your CMS and built the site anew.

In the case where you cannot use Nx Cloud, here's how to do it:

Updated Build Plugin

So the original article wasn't super clear about where your build plugin should be located. I put it in the tools directory under a new directory called skip-build. You should then have a netlify.toml in the root of your project referencing the folder (not the JS file) in the project:

[[plugins]]
package = "./tools/skip-build"

Now there are a few updates to the plugin too, let's go through them.

First, as this utility function returns a boolean you'll notice I renamed it to something that's much easier to follow. This new name shows the intent of the function so it makes it much easier for a newcomer to identify what's going on.

function hasProjectChanged(currentProject, lastDeployedCommit, latestCommit) {
  const execSync = require('child_process').execSync;
  const getAffected = `npm run nx print-affected --base=${lastDeployedCommit} --head=${latestCommit}`;
  const output = execSync(getAffected).toString();
  //get the list of changed projects from the output
  const sanitizedOutput = output
  .replace(/\r?\n|\r/g, '')
  .replace('> frontends@0.0.0 nx> nx print-affected', '')
  const changedProjects = JSON.parse(sanitizedOutput).projects;
  
  if (changedProjects.find(project => project === currentProject)) {
    return true;
  } else {
    return false;
  }
}

The other thing I did here was to sanitise the output of the execSync function, since I found there were a lot of unescaped characters in the output, and it also contained some other metadata. Once I replaced these with empty space, I was able to parse the valid JSON and be able to find out if my current project has changed or not.

Netlify's API has also changed slightly, instead of having onInit, we now have onPreBuild, so I've updated that here:

  onPreBuild: ({ utils }) => {
    const currentProject = process.env.PROJECT_NAME;
    const lastDeployedCommit = process.env.CACHED_COMMIT_REF;
    const latestCommit = 'HEAD';
    const projectHasChanged = hasProjectChanged(
      currentProject,
      lastDeployedCommit,
      latestCommit
    );
    if (!projectHasChanged) {
      utils.build.cancelBuild(
        `Build was cancelled because ${currentProject} was not affected by the latest changes`
      );
    }
  }
};

Once I had this plugin I was able to see that builds were skipped when I didn't update the project.

The only trouble I'm now having is that all of my content is stored in Webiny CMS, and the build plugin is unaware of whether that content has changed. I'm thinking of doing an API call out from the build to discover if there are any recently published articles ..."

Deserializing Data in GatsbyJS

Sat, 17 Mar 2018 00:00:00 GMT

One of the great strengths in static site generator Gatsbyjs is it's node API. This API gives Gatsby flexibility by allowing data to be transformed from myriad sources into a format that can easily be rendered as HTML: JSON. However, this can present a few issues when content is stored as escaped HTML, such as in WordPress posts and pages. Here's how a colleague and I worked around a tricky problem we discovered when working with content pulled in from WordPress.

I recently built a GatsbyJS site that stored content in Markdown and used NetlifyCMS to allow content authors to work directly with the site in editing and adding content. Using Markdown with NetlifyCMS was an interim solution. We needed to build a site in only a few days, but knew that requirements would at some point grow substantially, at which point we wanted to switch to a WordPress backend.

When I switched to WordPress it took me only a few hours to implement 95% of the work, most of which was spent generating the existing content as WordPress pages. I modified my gatsby-node.js file and my page template, but I came unstuck when using Helmet to generate page titles.

The Issue #

Using the existing data structure I was able to generate a page layout that looked like this:

This results in the result we expected, the content rendered as HTML inside the respective tags.

However, the post.title is stored as escaped HTML, so when you try to pop the post.title into the meta tag using Helmet, like this:

<Helmet title={`${post.title}`} />

you get whatever markup is stored. This is fine in most cases ... however, in WordPress land you'll quite often find people using things like <br> tags to adapt their layouts a little, or in my case, using a dash. When rendered as a meta tag, that looked like this:

mytitle \&#8221; contains a dash

Not great for presentation, or SEO.

The Workaround. #

There wasn't a simple way around this issue that my colleague David Hewitt or I could find.

We knew we needed to parse the element so that it returned as HTML again, but there wasn't an easy way to do this in Gatsbyjs. We knew we could use the DOM parser to achieve these results but here there was no document, hence no DOM.

In the end we turned to the xmldom package on NPM, and pulled the DOMParser method in, decoding the string and then grabbing the textcontent.

import React from 'react'
import Helmet from 'react-helmet'
import { DOMParser } from 'xmldom'

class pageTemplate extends React.Component {
  render() {

    const post = this.props.data.wordpressPage;
    const siteTitle = this.props.data.site.siteMetadata.title;

    const dom = new DOMParser().parseFromString(`<div>${this.props.data.wordpressPage.title}</div>`);
    const decodedString = dom.childNodes[0].textContent;

      return (
      <div>
        <Helmet title={`${decodedString} | ${siteTitle}`} />
...

Showing Gatsby's Strengths #

Although we encountered an issue here, in a lot of ways it has hilighted to me the strengths of GatsbyJS, not what you might perceive as weaknesses.

In the first instance, the issue originated from WordPress which stores escaped HTML instead of JSON, MarkDown or any other format. I can see the benefits to this (being able to add <br> tags to titles etc) but it does present some unique challenges—and not just to GatsbyJS.

This has also hilighted one of the fantastic strengths of GatsbyJS:- that I could pull in a package from NPM and use it in my build process. This opens up a world of possibility and makes GatsbyJS a lot more customizeable than I had thought previously.

And the other strength is that it still took me only a few hours to switch from using MarkDown files to a WordPress backend, such is the strength of the node API and the gatsby-source-wordpress in the way that it transforms data so that there was no difference between the two builds.

I've filed an issue on GatsbyJS on the repo, but there are a few issues around how to write a normalizer to deal with this. For instance, in most cases we do want to use the escaped HTML, so that the Title field is rendered in the component. It's only when using Helmet that we want the string deserialized.

So I'm not sure how to proceed with this .. at least there's a workaround for now, and I'm sure as a community we can address this issue so that we can drop the external dependency or build it into the existing processes in some way.

UPDATE: use CreateNodeField API #

So the way I rendered the title using the xmldom library was the best I could do at the time ... however, since I've learned more about Gatsby's API, I can now understand why transforming the title on the frontend isn't the best idea.

it's not performant. Users have to download that library before they see a rendered title. This has an impact on how fast people can see the rendered site ... Gatsby is all about performance, so this is a bad idea
rendering on the frontend can cause issues. Sometimes rehydration doesn't work properly when you're doing heavy manipulation on the frontend ... that's why we have backends, after all.

Here's a better way to do it:

In your gatsby-node.js file

import { decode } from 'he'

Then in the onCreateNode function:

createNodeField({
  node,
  name: 'renderedTitle',
  value: decode(node.title.rendered)
})
```"

Design is Broken

Sat, 14 Feb 2015 00:00:00 GMT

I was getting nowhere. For the previous four years, I had been self employed and had managed the transition from print design to web design. I was enjoying life, but then a desire to build on my success as part of a forward thinking and dynamic team motivated me to submit my CV to a number of local agencies.

But I was getting nowhere.

I had had a few interviews, and turned down one or two opportunities on principle. I knew the market was competitive, but the skill set I was presenting - that of a web designer - wasn't generating enough traction. Or at least, not the right kind of traction.

Then I hit on an idea. I would change one word on my profile in order to highlight web development instead of straight design. Yes, it's true that I loved design. I could argue about typefaces until the cows come home, I'd fuss for hours about alignment, think twice or three times about how colours I used might create the desired resonance for a piece of artwork or a page on a website. But I also loved wrangling with Sass, delving into PHP and had begun to tackle JavaScript seriously. I admit I had a way to go to perfecting my skills but I was ready for a challenge, and was honest about this in my approach to prospective employers.

So I switched the title from Web Designer to Web Developer - and then things started to change.

What is a designer? #

"A design is an ecosystem. Disrupt one thing and you disrupt others. A designers work is to foresee what will be disrupted and how." - Robert Hoekman Jr

What struck me when I began to read The Tao of User Experience is that design is no longer design.

Design is still about wrangling with the placement of elements on a page - so that content could fit within the desired space and convey the desired message. But it has become so much more than that.

The far larger part of design used to comprise of nuances of spacing, of detailed discussions about Gill Sans, it's history and pedigree. About spot colours and page bleed. All these are vital factors in print but these elements do not factor in web design.

Where it differs #

Rather than a reader, you have a user. Instead of a page, you have a system (known as a web site) and designers can no longer afford the somewhat arrogant approach of being unchallengeable masters or stroppy perfectionists because they are 'creatives'. This approach simply does not work in modern web design.

Designing a website is not mere decoration. To design a site one must properly examine what the end users will do, coupled with the business objectives of the client, and going on evidence to back up approaches not just to appearance but to interactive elements as well.

Especially to interactive elements.

A user doesn't interact with a page by reading it. He may scarcely read anything at all. But he will interact with it, and this is becoming the vital factor in web design. What happens when that button is pressed? How long does the page take to load? At what stage in the users journey are they likely to encounter this page? What happens when interactivity is driven by touch rather than a mouse?

These factors mean that the definition of design has changed. We are no longer designing static documents, we are designing interactions. Designers are also architects. We must be. Otherwise we must be merely decorators.

And I'm not in favour of allowing a decorator to draw up plans for my house."

A Developer's Guide to Professional Development

Sun, 19 May 2019 00:00:00 GMT

I got asked today why I was applying to attend two different conferences this year. Neither of these conferences were cheap; I feel very lucky to have a training budget where I work. On top of these my company generously provides me with access to 2 learning platforms as to help me become a better developer. As well as that, I pay for another learning platform myself. Is so much needed to keep ahead?

Is any of it really necessary?

Implementation Details #

The one thing constant about web development is change. Our industry iterates very quickly, and things that were best practice or popular a year ago soon make way for newer ways of doing things.

One good example of this is state management in React. Only a short time ago, Redux was a standard requirement on front end job ads, it was almost inseperable from React itself.

Now there are other ways of having a data layer on the frontend that can be considered alternatives to Redux, like React's own Context API, or Apollo GraphQL.

To keep my skillset relevant, and make sure I can easily switch jobs if I should need to, I feel I need to keep up with these trends.

I think of this part of my learning as the implementation detail of what we do. We need to keep relatively informed about new ways of doing things (even if we don't know them in depth) so that we can adapt if we need to.

To continue to develop in this regard, I mostly follow tutorials on one of the learning platforms I mentioned. But I also think it's important to get a more general overview by attending a conference that might cover several of these ideas in one go.

But there's something else that I think is important too.

Wider Implications #

I think it's also very valuable to keep a close eye on the wider implicationsof web development as an industry, and its intersection with others, what's going on in the wider world, and how that might affect what we do as developers.

There might be new technologies or ideas brimming on the horizon which could explode into our day-to-day development work. If we don't watch out for that we could be caught unaware.

An example of this could be responsive design. When "A Dao of Web Design" by John Allsop was first published on "A List Apart" it was an early indicator that something was on the horizon.

Later, when Ethan Marcotte published "Responsive Web Design" which made practical application of Allsop's ideas, we could begin to see how our industry would be altered forever.

To cover this aspect of my development I try to keep up with people I know who are thinking along these lines. I also subscribe to A List Apart and Smashing Magazine, two publications who are at the bleeding edge of web development.

But also, conferences such as Mozilla's MozFest, or JAMStack Conf, tackle wider issues such as privilege and equality, the latest ideas from the CSS Working Group, and new approaches to systems architecture.

Continuing... #

I think as developers we should be aware of our "CPD" (continuing Professional Development), much like an architecht or doctor would keep up with their industries, and particularly with that's happening in their area of expertise.

The Web Development industry must be one of the only self-taught professional industries out there that demands so much of it's practitioners. We can't possibly keep up with it all. I certainly can't.

I think it's enough to keep and overview of the trends and latest news just so we aren't taken completely by surprise when something changes, and choose to deep dive into subjects that appeal to us, or are relevant to our situations."

Don't fear the JAMStack

Thu, 03 Sep 2020 00:00:00 GMT

WordPress has long been the de facto publishing platform for the web. But many developers are hearing more about the JAMStack. Here's why I switched, and my take on where I think things are going to go from here.

I would like to acknowledge and address some recent efforts of those in WordPress circles to be vocal against the new approaches to publishing that have arisen in recent years, most significantly these 2 that I've seen:

"WordPress Co-Founder Matt Mullenweg Is Not a Fan of JAMstack"
Tweet: "Switching back to WordPress after a year on Hugo"

First of all, I must acknowledge a debt of gratitude to WordPress. The efforts the team have made to democratise publishing over the past 20 years have been hugely successful, and I still am really happy that there's a solution out there that people with little technical knowledge can utilise. No other publishing platform has done that, and even now its real competitors cannot achieve this because they are closed silos.

However, a democracy can never consist of one. A lot has happened in recent years which has preceded a sea change for a lot of developers to the JAMStack, a myriad of alternative separate technologies – one that still includes WordPress, but which is radically different from the approach it has taken.

Progression #

For me, this change happened when I realised I was struggling to progress as a developer, or even to find enjoyable work, where I could produce things I could be proud of. And that in order to move forward, I needed to move away from that platform. I found 3 things that were damaging my development career:

The WordPress API is often inconsistent. get_the_content() vs the_content(), WP_Post vs WP_Term, differing returns (sometimes void, sometimes not) for different get_*() functions ... trying to find my way around this API could never help someone cultivate good development habits, or understand how an API should work.
WordPress hasn't moved on from PHP 5. This means learning newer aspects of the language has been impossible and left me unprepared to work on other platforms. I could only get another job working with WordPress, which meant I was unlikely to ever move out of the cottage industry that provides.
WordPress development practices haven't moved with the times. I worked for a short time on a Laravel project, and I was dumbfounded by things I had to learn, like build pipelines, database migrations, object oriented language principles, and even how to use a package manager.

In short, I was faced with a decision: either continue to develop solely with WordPress for the rest of my career, or make the jump. I had already seen a large number of developers grow dissatisfied because of these issues, and similarly move off the platform.

I decided to do the same just around the time that Gutenberg was being released.

Gutenberg #

Whilst the new editor interface, Gutenberg, was a big step forward for users of the platform, it's no secret that it alienated a lot of developers.

Largely this was because of the ease with which page layouts could be created without developer input, bypassing the finesse and structure that a developers' input brings to a website. Now it's even easier for individuals to create blogs under their own steam.

I am overall pleased with this idea because it pulls the rug out from under a lot of cheap, low quality "web design" agencies that have built their empires on the free platform for many years. I've worked for a range of these companies and I personally would be happy to see them pivot or be made extinct.

But that also means there's a lot less work for a developer to do ... so not only has WordPress been keeping developers in a cottage industry, it is also forcing people out of that industry at an increasing rate.

When I looked to the horizon, my colleagues and I could see a world where WordPress was once again the de facto standard for small customers who needed a quick platform, and where agencies turn to using alternatives.

I needed to skill up quickly.

Voting Pebbles #

There's still room for WordPress in the new landscape of the JAMStack world. The WP GraphQL plugin is great for sourcing content. But yes, there are a lot more alternatives that have the economies of scale and financial power to pivot much more quickly than WordPress has.

A lot of progress has been made towards finding a better editor experience ... but we haven't quite solved it yet. I think projects like TinaCMS from Forestry are going to be the next iteration, and have the potential to bring an even better editing experience to users than WordPress can provide.

In the meantime, developers like me will continue to push the JAMStack agenda. Why? Because we have been pushed out of our cottage industry, and we need to continue to work, and to enjoy our work. And because we do have a lot of buying power for our clients, stakeholders and family and friends whom we build for.

WordPress is no longer my favourite option. But it is still an option.

You started the avalanche, and it's too late for the pebbles to vote.

(Wow there really is a Babylon 5 quote for every occasion.)"

Doughnut chart SVG Animation

Tue, 30 May 2017 00:00:00 GMT

For a recent project, I was asked to design an animation for a doughnut chart. The data could be dynamically editable by the content author, who could also choose how many animations to display on a given content area.

I started off by designing the assets in Adobe Illustrator. The design required a gray background with an overlay of a specific colour that would feature the main animation. To achieve this I layered 2 circles, one on top of the other.

To easily enable animating, these layers are SVG path elements. I'm cheating a bit by making the top layer a linear path, instead of a true circle, but GreenSock's DrawSVG plugin works by animating the path, not the fill, of an SVG element.

The Setup #

For the initial setup I grabbed the elements and a few values, wrapping these in a jQuery each() function to isolate them from sibling animations that were occurring on the same page:

  var values = $('.statistic');

  values.each(function () {

      var percentage = $(this).find('.figure__content .fig');
      var textcontent = percentage.text();
      var circle = $(this).find('.statcircle__animated');
});

As I already had jQuery as a dependency in this project I used the animate() function to handle the text, a number that would rise from 0 to the one specified by the user.

 percentage.prop('Counter',0).animate({
      Counter: textcontent
  }, {
      duration: 1000,
      easing: 'swing',
      step: function (now) {
          percentage.text(Math.ceil(now));
      }
  });

Fallbacks #

By setting the prop value to 0 means that if JavaScript isn't loaded in the browser, the percentage will still be visible to the user.

I'm still keen to make sure I provide a useable fallback just in case the user can't access JavaScript on their connection.

The Chart #

Then, using a Tween and Draw SVG I was able to find the path, and choose an appropriate animation for the effect.

 function initTweens() {
      // animate circle path using GSAP DrawSVG
      TweenMax.fromTo(circle, 1.5, {
          // animate!
          drawSVG:"0"
      },
      {
          drawSVG: textcontent + "%"
      }); // tween
  };
  initTweens();

Using fromTo allows me to set the initial value to 0 then scale up to the value I grabbed from the percentage field.

I absolutely love the simplicity of DrawSVG, which meant no difficult calculations for me. I simply gave it a value and it calculated how it would draw path for me around almost 360 degrees. This gave me inherent robustness because GreenSock handles edge cases, and reduced development time greatly.

Final Result #

For launch, I used Waypoints.js to specify that the animation shouldn't start until the elements were in viewport. I then created the meta fields in the back end and tied them up to the page template.

You can see the final results in situ on my CodePen: https://codepen.io/endymion1818/pen/ygvVgQ"

Dreamers, Builders and Cultivators

Fri, 09 May 2025 00:00:00 GMT

In a well established team, personality traits come into play. Here we group three main types that, when recognised and supported, can help the business and the technology team to thrive.

Have you ever worked with a fellow engineer who doesn't seem interested in investigating new tools that are available? Have you ever met someone who annoys you because they push untested, proof-of-concept code into production?

These aren't flaws in our engineering team members, they're features. And when we properly understand and support them, each can be an asset to the organisation you work for.

I'm going to just add a caveat here: any ideas about how personalities work are just theories. We can't really understand how people think, only observe the outcome. But having observed these traits in quite a few teams, and having discussed them with other people, I think I'm onto something that might be able to help you.

Personality traits

Each person on the planet has traits. Like being left handed or right handed, these are personal preferences that are deeply ingrained in all of us: although someone might work hard to use their right hand when they're left handed, they will likely always be conscious that they prefer the alternative.

If you watch the personalities of certain teams in your organisation, you might find certain teams share common traits. For example, I've observed that HR teams often have common traits that have steered them towards a career in that field. I'm not saying everyone on the team displays those same traits, but its likely that many will to some degree or another.

And I think it's likely that you are a leader because of the traits you have.

For simplicity, and to help us understand differences in individuals I've observed within a software team, I'm going to group them into three terms: "dreamer", "builder" and "cultivator".

I can't take credit for identifying these groups myself: they were first outlined in this Reddit thread by user j_d_q, which I'm now expanding on.

The Dreamers: thinking big, not small

Dreamers think big. They're the explorers of our teams. They are constantly enthusing about new technology or new ways of doing things. You'll find them on Discord channels for Nix and Biome. They'll use Arch Linux just so they can get the latest release of packages.

They are great at building enthusiasm in a team, and can help everyone keep up to date with new technology ... as long as their enthusiasm is tempered (but not smothered) with a dose of caution. Leverage their energy by getting them to build experimental new features. Give them a business problem to solve without specifying a solution. It's likely everyone on the team will be impressed with what they come up with.

Try to avoid asking these people to implement a simple feature that they can do in their sleep. If you do, they'll either be asleep as they do it, or the'll find a way to do it with this obscure new package they found and want to try out.

When they deliver, don't expect tested, production-ready code. Instead rely on Builders to shore up their creations.

Builders: the glue that guides

Builders are great at, well, building. They'll take the raw concept that a Dreamer has come up with, adding test coverage, improving type safety and generally making sure other team members can use it easily without it breaking unexpectedly.

Builders aren't focused on concepts as much as they are execution: it's more difficult for them to conjure up solutions without any guidance. If you ask them to, they could become stuck in a loop where they can't decide on which approach to implement. Assist them by encouraging them to shortlist 2 or 3 approaches, then decide with them. This will give them the confidence they need to progress.

Leverage their love of documentation, clean code and automation to refine and protect code that needs attention or that is difficult to work with. Allow them to introduce coding standards and tools that free up cognitive ability for the whole team.

Use builders to solve problems that cultivators cannot because they're engrossed in delivering on their current workload.

Cultivators: implementors that tend

Cultivators like structure. They see their role as tending to the land that the dreamers and builders have set up.

Have a project with a large bug backlog? Give it to a cultivator and watch them steadily and happily clear it.

They can take a product requirement that has a clear implementation path and be confident to carry it out. Anything well documented that has a lot of clarity, and less room for the unexpected and they will thrive.

Leverage their deep knowledge of the codebase by pairing them with one of the other types. Arrange regular check-ins to make sure they're on track and don't feel blocked by something that they deem as outside the requirements of the task.

Try to avoid giving these people a sketchy feature with no description and expect them to fulfil it.

Cultivators are essential to carrying out tasks that builders and dreamers would struggle with.

There are no rules

At the end of the day, nobody's traits are set in stone; a left-handed person can learn to use their right hand pretty well. And traits are stronger in some people and not in others: there is a portion of the population that are ambidextrous and use both hands with equal aplomb.

As I said at the start, this is just a rough outline that I think can help leaders understand and work with team members who have different strengths and abilities.

We can leverage these abilities for the betterment of the whole team.

If you have the capacity, identifying and grouping different engineers according to project requirements can help you to fulfil them with greater speed and efficiency.

Understanding your team members this way can also help negotiate disagreements: if arguments erupt over different approaches engineers might have, you can point out respective strengths to dissenting team members. You can also work with individuals to balance their traits out a little.

Pushing every engineer to have the exact same set of skills isn't going work to the best advantage of the team and could lead to some valued members withdrawing from the team or even quitting.

Instead, cultivate a happier and more productive workforce by respecting differences and using them to everyone's advantage.

Dynamic client side routes in GatsbyJS

Thu, 18 Jun 2020 00:00:00 GMT

Would you use GatsbyJS for a dynamic app? Yes I would. The reason is that I can make good use of Gatsby's great developer experience, as well as do everything I would normally do in a React application. Here's how I recently implemented dynamic routing.

Disclaimer: My colleage Romina Moya originally discovered how Gatsby could do this, and showed it to me afterwards.

I've been learning more about Amazon Web Services recently, and found an excellent tutorial written by the makers of Seed, a sort of CI pipeline for serverless applications. You can find the tutorial at https://serverless-stack.com.

I wanted to diverge from the tutorial a little on the frontend, especially since I really enjoy working with GatsbyJS and wanted to use it's static rendering where I could.

One of the lessons uses React Router to render routes for notes like this:

/notes/2074d6b0-c5d7-11ea-bd39-5f447bbc7b39

where the last part of the url (the "pathname") needs to be generated client-side only. The reason is that the component would resolve to any number of records in the database, and needs be accessible only to the logged-in user.

To achieve this in Gatsby I referred to their excellent documentation (https://www.gatsbyjs.org/docs/client-only-routes-and-user-authentication/), but there's a lot of information in that documentation, and it can be a little difficult to see what you need to do. So here's some extra pointers that I thought might be useful:

Setup #

In my applications as in most Gatsby sites, I use a wrapping component that I name "layout" or "entry". If you don't have that, you could effectively do the same thing by adding a file in the root of your project named "gatsby-browser.js" and using the onClientEntry() api (https://www.gatsbyjs.org/docs/browser-apis/#onClientEntry).

In that file I needed to first import { Router } from "@reach-router", and also import { navigate } from "gatsby" then in the render function add this:

<Router>
  <BounceToHome default />
</Router>

The BounceToHome function is for any path the router comes across that isn't defined (as you can see from the default prop I passed to it). This is useful for if the user types anything That function uses navigate() that I imported earlier:

const BounceToHome = () => {
  useEffect(() => {
    navigate("/", { replace: true });
  }, []);
  return null;
};

As you can see I use the useEffect hook to navigate the user home on the first render.

Rendering client-only component #

At the top of the file I've imported the functionality I already defined for rendering, editing and deleting notes:

import Notes from "./Notes";

Now as a child of the Router component I'm going to add in my Notes component:

<Notes path="/notes/:noteId/" component={Notes} />

The path tells Reach Router that I want to accept paths with a prefix "/notes", and the colon after that is a variable that will be passed to your component. It could be called anything you like, I thought noteId was the most relevant name for what I was building.

On the following line I've passed to the router which component I want to render on that path.

Now I can use that component in the page in this way:

export default function Notes({noteId}) {
  useEffect(() => {
    function loadNote() {
      return get("notes", `/notes/${noteId}`, '');
    }
    async function onLoad() {
      try {
        const note = await loadNote();
        setContent(note.content);
        setNote(note);
      } catch (error) {
        onError(error);
      }
    }
    onLoad();
  }, [noteId]);
  return(
    ...
  )
}

Whenever the noteId changes the useEffect hook runs and loadNote() gets the note using functionality available in the aws-amplifypackage.

Gatsby is ready for Apps! #

If React is "just JavaScript", then Gatsby is "just a javascript framework", with all of the benefits that brings you, as well as some significant other things like better accessibility, static rendering, and loads of other cool stuff.

Let's not be too quick to pigeon-hole Gatsby into a certain corner: it's a versatile set of tools that allows us to jump start our projects and create any number of really cool things.

Update: don't put this in your Layout / Entry file! #

After playing with this a little more, I realised that client only routes override file routes ... so if you have the dynamic routing on the root path (usually "/"), it'll override everything else and you won't be able to navigate to any other page you have defined in the pages folder.

So it's best to scope your client routes to a folder (such as app), and avoid overriding your other pages!"

Some of my favourite books

Sat, 27 Jun 2020 00:00:00 GMT

I have found it very difficult to collect this list together, I go through phases with books and these might be considered my current favourites ... but here goes!

In no strict order of preference:

F Scott Fitzgerald - collected short stories. Life in the 1920s & 30s with just a pinch of melancholy
John Keats complete poems - the sheer genius of this 20 year olds ideas was amazing. Such a tragic life too (he died aged 26)
Victor Hugo - les Miserables (translated by Norman Denny). An undiluted masterpiece, Paris in minature and a lot of fascinating philosophical meandering in between.
James Joyce - Poems & shorter writings. JJ’s books are unnaproachable but he had a command of English like only Shakespeare, which makes his poetry magnificent.
Virginia Woolf - The Waves. Just read the opening page. She understood people on such an intrinsic level.
Marcel Proust - Swanns Way. He takes a long time to get to the point but it’s like climbing a mountain, the view from the peak is indescribably beautiful.
Foundation - Asimov. The scope of the initial concept for the series is mind boggling.
Frank Herbert - Dune. world building at its absolute best
Dan Simmons - The Hyperion Cantos. Requires a bit of a strong stomach but the author loved literature as well as high concept sci fi. Awesome.
China Mieville - Perdido Street Station, Embassytown or The City and the City. Some incredible concepts about language and world building, writing style is mesmerising and transports you into the his worlds with ease.
Mervyn Peake - Titus Groan. Brilliant, ponderous, odd, and wonderful.
Jorge Luis Borges - Collected Stories. Mind bending infinite and looping concepts.
Anne Fadiman - Confessions of a Common Reader. I re-read this slim volume every few years, a wonderful love letter to books, reading, and life in general.
Boris Pasternak - Doctor Zhivago. a grand, poetic, and empathic exploration of individuals in Russia in the civil war. "I feel for each of them / As if I were in their skin", he writes in one of the poems attached to the book.

Honourable mentions #

I am halfway through the Illiad, an old translation by a guy called Chapman who kept the original meter from the poem, it’s jolly wordy and doesn’t convey the barbarism of the original (probably a good thing!).

I also recently read Paradise Lost by John Milton. The charachterization is superb and language and turn of phrase is delightful.

Sophie’s World is good for covering the basic concepts of just about every branch of philosophy, in story form too!

I read Robinson Crusoe but the racism and colonialism were stomach churning."

First Steps with CSS Animations

Tue, 14 Apr 2015 00:00:00 GMT

CSS Animations are quickly complimenting if not replacing Javascript for generating a little bit of on-page whizz (that's the technical term). This is the first time I've delved into the syntax for CSS Animations that I hope can replace the tired carousels resident on many websites. Here's how I implemented it.

First of all, I've stuck to using CSS for this project although I'm using SASS for this site. The reason is that I wanted to get familiar with the CSS syntax first. I don't want to chain myself to SASS because it's one step removed from what the browser renders. Anyway, that's a story for another day.

Step 1: Scope the Animation #

Being a small marketing agency we're still formulating the best way to approach website builds. So this project came to me with a visual design but no scope. The designs showed what was required, that the circular images would change to reveal text on flat background.

What they weren't so good at displaying is the end state of the animation. Would it roll around again? The plan was originally for a Javascript carousel here, so I could make some assumptions about the functionality based on that.

Step 2: The HTML #

<div class="container">
    <a href="services.php">
      <div class="col-sm-3 home-circle" id="one">
          <img class="bottom" id="five" src="/assets/img/property1.png">
      </div>
    </a>
    <a href="services.php">
      <div class="col-sm-3 home-circle" id="two">
        <img class="bottom" id="six" src="/assets/img/property2.png">
      </div>
    </a>
    <a href="services.php">
      <div class="col-sm-3 home-circle" id="three">
        <img class="bottom" id="seven" src="/assets/img/property3.png">
      </div>
    </a>
    <a href="services.php">
      <div class="col-sm-3 home-circle" id="four">
        <img class="bottom" id="eight" src="/assets/img/property4.png">
      </div>
    </div>
  </a>
  </div>

This took me a few attempts to work out. I try to avoid using position:absolute wherever possible, so in this case I used a background-imageon the parent div with an img inside.

Step 3: Declaring the First Animation #


@-webkit-keyframes fadeOut { from { opacity:1; } to { opacity:0; } }
@-moz-keyframes fadeOut { from { opacity:1; } to { opacity:0; } }
@keyframes fadeOut { from { opacity:1; } to { opacity:0; } }
@-webkit-keyframes fadeIn { from { opacity:0; } to { opacity:1; } }
@-moz-keyframes fadeIn { from { opacity:0; } to { opacity:1; } }
@keyframes fadeIn { from { opacity:0; } to { opacity:1; } }

.home-banner .container a .col-sm-3 {
   opacity:0;
   -webkit-animation:fadeIn ease-in 1;
   -moz-animation:fadeIn ease-in 1;
   animation:fadeIn ease-in 1;

   -webkit-animation-fill-mode:forwards;
   -moz-animation-fill-mode:forwards;
   animation-fill-mode:forwards;

   -webkit-animation-duration:1s;
   -moz-animation-duration:1s;
   animation-duration:1s;
  }

Because I'm a visually-minded person, I decided to break my CSS into two parts and to declare the animations in areas that would help me logically follow what was displaying on the site, even if it would be slightly less efficient. I'm sure that as I get more used to the syntax I'll change this practice to be as efficient as possible (and to use SASS).

Step 4: Timing #

I've heard it said a few times that timing is the hardest thing to get right in an animation. It was quite difficult to work out the gaps that their should be between each animation state, and took quite a few goes before it appeared to flow smoothly, almost as if there was a short increase in speed in the middle two animations, although this is just a perception.

.home-banner #one {
  background:url('/assets/img/propcons.svg') center no-repeat;
  background-size: 100%;

  -webkit-animation-delay: 0.4s;
  -moz-animation-delay: 0.4s;
  animation-delay: 0.4s;
}
.home-banner #two {
  background:url('/assets/img/charsurv.svg') center no-repeat;
  background-size: 100%;

  -webkit-animation-delay: 0.8s;
  -moz-animation-delay: 0.8s;
  animation-delay: 0.8s;
}

What I was very conscious of was that the delay between the first animation (the property images) appearing, and the second (the text overlays) needed to be a) slow enough so the client felt they had good value out of the images we had provided and b) fast enough that visitors would see the second transition before they scrolled further down the page.

So we settled on .6 seconds, which seemed to be the optimum delay we could achieve without investing in further research.

Here's the result (sorry for the high compression - wanted to keep a smaller file size as possible!

Browser Caveats #

What we noticed going through the testing phase was that certain aspects of the animation spec are still under development. Some things like animation-reverse aren't supported in Vivaldi or Mobile Chrome (tested on iOS 8). But we felt that since the animation still works to a fair degree, it would be reasonable to keep using it.

CSS Animations - The Way Forward #

I got a huge amount of satisfaction trying out the CSS Animation spec. This isn't only due to my lack of JavaScript skills (which are steadily improving) but also my desire to build high quality websites that are as lightweight as possible.

The outcome was so well recieved by our team involved that it's opened our designers and other stakeholders eyes to the options available to them:- they now have many, many more options at their fingertips instead of only the aged and worn carousel default.

This demonstrated for me that even though it is often difficult to step out of our comfort zone, but the rewards of doing so are often great.

I'd love to know how you would improve on this code, or if you've used CSS Animations in a way that has challenged you recently."

First Steps with Flexbox

Sun, 15 Feb 2015 00:00:00 GMT

On a recent project for international research machine builder Anton Paar, I used the new flex-box CSS module for the first time. Here's how it went.

The flex box module has had a long gestation period, and there are 2 main specs of the module, so when researching methods of using it you have to be careful to find recent enough posts which outline the latest spec.

As always, CSS Tricks was particularly useful, along with Stack Overflow in getting familiar with the syntax and methods.

The Problem #

It's not a biggie, but I like things to line up nicely. As it stood, the four content boxes on the home page of Anton Paar UK's microsite would be populated by both static content and dynamic. We decided to use Flexbox to line the four columns up, something that has never been achieveable before in CSS without Javascript.

Here's the HTML:

<div class="row home-features">
	<div class="col-md-3 home-feature">
		<div class="well">
			<?php
			if ( function_exists( 'dynamic_sidebar' ) ) {
				dynamic_sidebar( "home-1" );
		} ?>
		</div>
	</div>
	<div class="col-md-3 home-feature">
		<div class="well">
			<?php
			if ( function_exists( 'dynamic_sidebar' ) ) {
				dynamic_sidebar( "home-2" );
			} ?>
		</div>
	</div>
	<div class="col-md-3 home-feature">
		<div class="well">
			<?php
			if ( function_exists( 'dynamic_sidebar' ) ) {
				dynamic_sidebar( "home-3" );
			} ?>
		</div>
	</div>
	<div class="col-md-3 home-feature">
		<div class="well">
			<?php
			if ( function_exists( 'dynamic_sidebar' ) ) {
				dynamic_sidebar( "home-4" );
			} ?>
			</div>
		</div>
	</div>
</div>

As you can see from the following screenshot from before the site went live, the problem was that the 4 columns weren't lining up. This has long been almost impossible to do in HTML / CSS, certainly without misappropriating the position: absolute property.

But using Flexbox allowed us to simplify the code by removing the wellDIVs as well as provide a solution to the problem:

.home-features {
  display: -webkit-box;
  display: -moz-box;
  display: -webkit-flexbox;
  display: -ms-flexbox;
  display: -webkit-flex;
  display: flex;
  padding: 0 10px;
}
.home-feature {
  -webkit-box-flex: 1;
  -moz-box-flex: 1;
  -webkit-flex: 1;
  -ms-flex: 1;
  flex: 1;
}

We added this attribute under a media-query of (min-width: 992px)so that the columns would stack at smaller viewport sizes.

This exercise has proved to be a great solution to an ongoing problem in web development.

After using it on this project I really don't want to go back, which is a shame because there are still some caveats with Safari (Both Mac OS X and iOS)."

First steps with the Temporal API

Thu, 08 May 2025 00:00:00 GMT

Dates in JavaScript are changing forever: The Temporal API is now at Stage 4, which means it's stable, and there's a polyfill available. It's time to start getting familiar with the API.

One thing I do a lot of is converting dates that are stored using Epoch dates. These are numbers that represent every millisecond since 1 January 1970. At the time of writing, the digit for the current epoch date was 1746706628000. A second later it was 1746706629000. And so on.

If you'd like to know more about epoch timestamps, this is a good resource: https://www.epochconverter.com/

These are ideal for storing data, but we've always had issues calculating these into dates in our applications because of time zones and locations.

Temporal is much better because you don't need to load extra JavaScript that you'd ordinarily have to get from a package: one of the most popular is moment.js.

But these packages tend to be fairly large in size. You could roll your own functions but ... they tend to be challenging reason about and difficult to maintain.

Polyfilling

There are two spec-compliant polyfills for Temporal, one of which I'm using until browsers are updated, at which point it should be just a case or removing the import.

import { Temporal } from '@js-temporal/polyfill'

Converting Epoch timestamps using Temporal

The first solution I needed was to convert that epoch date mentioned above into a datetime object so that I can then localise it.

Temporal has that easily covered:

const timeInstant = Temporal.Instant.fromEpochMilliseconds(1746706629000)

If you log this out you get a new Temporal.Instant type with the milliseconds and nanoseconds properties. Not useful at this point but an important step nonetheless. It's now a Temporal instance, much like new Date(mydate) converted a string into a Date.

But what if your Epoch date uses seconds instead of milliseconds? Temporal.fromEpochSeconds() doesn't exist, so you need to manually turn the number into milliseconds:

const timeInstant = Temporal.Instant.fromEpochMilliseconds(1746706629 * 1000)

Now we've got a Temporal instance, the next step is to convert this into a date that is human-readable and localised.

Localising

To localise a date we need to do three things:

Set a format to present the date
Set a locale that can be used to arrange the dates (this will determine, for example, if it's DD/MM/YYYY, or MM/DD/YYYY which is US style.)
Set the local time so that it reflects the users time, accounting for daylight savings etc

const localisedDate = timeInstant.toLocaleString(
  'en-GB',
  {
    timeZone: 'Europe/London',
    day: '2-digit',
    month: 'short',
    year: 'numeric'
  }
)

Temporal is going to save you time

Hah, see what I did there. But yes, kilobytes of extra JavaScript gone from your projects. A standard way to modify dates. And no more wrangling with Date(). That's what you can look forward to with the Temporal API.

First Steps with Story Animations

Sat, 15 Jul 2017 00:00:00 GMT

For a while now I've really enjoyed tinkering with animations as a way to bring a lot more quality to my projects. But as well as providing interactivity, I was curious to explore another way animations can be used in a project: to tell a story.

I've discovered that animations can really make a project stand out from the competition, and I find myself driven to this field more and more. Partly, I want to do this to showcase what the web is really capable of in difference to other mediums. But, being practical about it, it's also a great way of getting users to stay on your site more and for it to be memorable.

The style I adopted for this site was heavily influenced by my love of the English and Welsh countryside. I've also been thinking a lot about the animation film "Watership Down" lately. I watched that film as a kid and it seems to have made a lasting impression on me. So I took the idea from those influences.

Drawing the SVGs #

I used both Affinity Designer and Sublime Text to manipulate some SVGs I shamefully got free from the internet. I'm sorry, I'm not proud of that fact, but time was not on my side.

Although you percieve there to be one tree that transitions from winter to summer, and one rabbit that moves across from one side of the screen to the other, there are actually 2 trees and 8 rabbits.

This is because the easiest way to create the perception of movement I have found is to change the opacity.

Animating #

Each element is changed by modifying the opacity in GreenSock Animation Platform, which is tied to ScrollMagic, in order to perform the animations not on a timeline, but on scroll.

window.onload = function(){

  // smController = new ScrollMagic.Controller();
  smTreeController = new ScrollMagic.Controller();
  smRabbitController = new ScrollMagic.Controller();

  // getters
  var trigger = document.getElementById('main');

  var treeSummer = document.getElementById('tree-summer');
  var treeWinter = document.getElementById('tree-winter');

  var rabbit0 = document.getElementById('rabbit-0');
  var rabbit1 = document.getElementById('rabbit-1');
  var rabbit3 = document.getElementById('rabbit-3');
  var rabbit4 = document.getElementById('rabbit-4');
  var rabbit5 = document.getElementById('rabbit-5');
  var rabbit6 = document.getElementById('rabbit-6');
  var rabbit8 = document.getElementById('rabbit-8');

  // from winter to summer
  var treeWinterToSummer1 = new ScrollMagic.Scene({
    triggerElement: trigger,
    duration: '100%',
    offset: '400%'
  })
  .setTween(treeSummer, { opacity: 0.6 })
  .addTo(smTreeController);

  var treeWinterToSummer2 = new ScrollMagic.Scene({
    triggerElement: trigger,
    duration: '100%',
    offset: '500%'
  })
  .setTween(treeWinter, { opacity: 0 })
  .addTo(smTreeController);

  // 0
  var heyRabbit0 = new ScrollMagic.Scene({
    triggerElement: trigger,
    offset: '500%',
    duration: '15%'
  })
  .setTween(rabbit0, { opacity: 0.6 })
  .addTo(smRabbitController);

  var byeRabbit0 = new ScrollMagic.Scene({
    triggerElement: trigger,
    offset: '600%',
    duration: '15%'
  })
  .setTween(rabbit0, { opacity: 0 })
  .addTo(smRabbitController);

  // 1
  var heyRabbit1 = new ScrollMagic.Scene({
    triggerElement: trigger,
    offset: '650%',
    duration: '15%'
  })
  .setTween(rabbit1, { opacity: 0.6 })
  .addTo(smRabbitController);

  var byeRabbit1 = new ScrollMagic.Scene({
    triggerElement: trigger,
    offset: '750%',
    duration: '15%'
  })
  .setTween(rabbit1, { opacity: 0 })
  .addTo(smRabbitController);

// and so on...

};

There were actually 2 versions of this animation. I played with greensock to tween the appearance and disappearance of the rabbits. However, I decided against this approach beacuse in order to make the last-but-one rabbit stay in view for longer, I would have to set a pin, and release it later.

I didn't think this approach was robust enough, especially since I want to build on this animation in the future. Having lots of spurious divs positioned around my dom didn't feel right.

Performance Considerations #

Loading up these libraries does have a performance cost of course. A cost that is most noticeable on mobile devices. So instead of enqueueing my Javascript in the normal way, I've been working on a script that writes the tag into the DOM once you're at a certain breakpoint:

if ('querySelector' in document && 'addEventListener' in window) {
    var windowWidth = document.body.clientWidth;
    if(windowWidth > 1024){

        (function(H){H.className=H.className.replace(/\bno-js\b/,'js')})(document.documentElement);

        document.write(unescape("%3Cscript src='/js/deliciousreverie-noncriticalscripts.min.js' type='text/javascript' defer%3E%3C/script%3E"));
        }
};

I further scoped this hit by adding a partial to the footer in hugo, so that the partial containing the SVG would only load on the home page.

The home page loading time hovers around 3 seconds on desktop now. I know it could be better, but frankly, I feel the hit is worth it for something that I think will add plenty of value to my site.

Accessibility #

I don' think there are accessibility concerns with this project. The animation is purely illustrative and shouldn't affect screen readers or sight impaired ... I hope!

Please don't hesitate to let me know if it does.

Future Plans #

I already have plans for similarly-themed animations on other pages on the site, and have started building a bird that will fly across the screen, circle near the tree, and disappear off-screen.

These I'll hopefully get done before the kids leave home."

First steps with Sveltekit

Thu, 27 May 2021 00:00:00 GMT

Svelte is a new framework for JavaScript ... hear me out! It does seem to some people that there's a new JavaScript framework out every day. And there are a lot of new ones at the minute. But SvelteKit is great for one reason: performance. And it's this above everything else (except accessibility) that I'm excited about.

Svelte does have a lot of things going for it. It's as nice (if not nicer) to develop with than React, and getting started is a lot, lot quicker.

These days, if you're going to build a site with React, you either go with the slightly walled garden of create-react-app, or something like Next or Gatsby. Vue has it's own set of these problems too.

However Svelte ... isn't just a framework. It's a compiler too. This facilitates much smaller bundle sizes:

React is 42.2kb including react-dom. By comparison, Preact is 4kb. By contrast, Svelte is 1.6kb. That blows my mind.

In our day of multi-megabyte downloads for some websites (that I've seen), Svelte could really turn the tide.

Developer experience #

The other thing I want to talk about is the developer experience of Svelte. It's very pleasant to work with. Writing components feels a lot more like writing native HTML and CSS. There were certainly some unusual (to me) augmentations, for example writing conditional blocks has an unusual syntax, with { #if } and { /if} either side of the condition.

I have to say, I do kinda prefer React's slightly more JavaScript-y approach.

{condition && <div>conditional item</div>;}

However, when it comes to rendering HTML content, it's much more succinct.

{@html htmlContent}

Don't get me wrong, that reminder is pretty useful. However once you're familiar with the concept, it's not exactly a big deal any more.

SvelteKit vs NextJS vs Gatsby #

SvelteKit is definitely going more the route of NextJS rather than Gatsby. Gatsby is a little more prescriptive and great for newer developers (or those short on time), but NextJS allows much more freedom.

Similar to NextJS, SvelteKit will have options for rendering with SSG / SSR or on the client, and different adapters for interfacing with serverless functions from different providers.

This means you have to fetch data your own way, either on the client or server, and that means you have to be a bit more aware of the performance costs involved.

However, I couldn't find any reference to how you build a NextJS that compiles at build time, that doesn't send any JavaScript to the client. Yet, that's one of the excellent things about SvelteKit: I easily configured my site to send exactly no clientside javascript to the browser, and it works perfectly!

My project #

You can take a look at my live project at https://promatt.co.uk.

I wrote this small site using a starter built by Scott Spence. It fetches data from [Hygraph](: https://hygraph.com/) and is hosted on Vercel. I've used Formspree to forward emails from the /contact page to my friend's email address, but I could have just as easily done that with a serverless function.

Conclusion #

There's one other project I've got my eye on that will trump SvelteKit for my tool of choice. I think it's a great challenge to other frameworks, and I really hope it starts some kind of "performance war" where the performance costs of frameworks are driven down by competition in the space.

Even if that doesn't happen, SvelteKit is a wonderful addition to the JavaScript developers' tool belt."

First Steps with zsh

Wed, 24 Aug 2016 00:00:00 GMT

I'm one of those guys (coming from a design background) that was terrified of the command line. Literally, terrified. I knew enough that simple commands could irreparably break my computer, and that scared me. But gradually, I've begun to use more command line tools, and recently discovered ways to make my workflow easier by using zsh terminal commands.

I started using the terminal because of the benefits of build tools. I use Grunt in everyday production of my websites now, to minify & concat my JS and SASS files, and sometimes to minify images and SVGs. If it hadn't been for these benefits, I doubt I would have ever used the terminal in the way I do now.

GIT in its Native Environment

I recently ditched my 2 git graphical user interface programs (Github and Git Kraken) in favour of using the terminal. I liked how I could have more control over my commits this way, and not have the extra overhead of another program open whilst I'm trying to work (I have a very minimalist approach to working, as far as that's possible).

Git on the command line is fantastically useful, and I've never been more comfortable stashing, switching branches and doing all kinds of fancy things ... but I always wanted my terminal to do something more.

I find the following things really useful: This is the time I ran the command. So if I'm working through a project I can see exactly what time something was done ... this gets useful if you're tracking how long a certain task has taken, especially if your time is billable.

~/S/ech-ech

The next bit shows the current path in an abbreviated form. For example, the /S/ refers to the "Sites" directory.

Following that is the git information I was telling you about:

git:(master)

This shows that the current directory is a git project, and that the current branch is one called master.

This reference, in purple in the screenshot:

0dd6b659

Is the commit hash. I use this to track my commits through the day, and can easily scan up & down my commit history to see what changes are made, when.

A Note about Tools

To get this functionality, I'm using a tool called Oh My zsh, a tool which has hundreds of themes , of which I'm using this one: https://github.com/calebmeyer/cpm-zsh-theme, which you have to install manually (instructions are on the readme file).

I also enabled a few extra plugins for git functionality by adding this to the .zshrc file: plugins=(git git-extras git-remote-branch)

I would never have become this confident were it not for this course on Udemy: https://thoughtbot.com/upcase/videos/git-workflow. Check it out! There's lots more in the course, including some snippets you can run so that you can use git even more effectively.

Confidence in Progress

I'm still working on using the command line effectively. I've recently decided to start using webp images in my website builds ... there's a command-line tool that converts jpg files to webp and a script here that allows you batch process all images in a directory: https://stackoverflow.com/questions/26565191/batch-process-png-to-webp

So, if you're not confident with the command line, there are many useful, time saving things you can do. Perhaps now is the time to give it a go?"

First 2 weeks at Webiny

Fri, 11 Mar 2022 00:00:00 GMT

I've been at Webiny two weeks now. I have had the most fun I've had in a long time career wise. Here's what I've learned and what my objectives are going to be for the coming few weeks.

The first week at any company is always difficult. There's so much to learn, there's a new culture to adopt, and new personal dynamics to interact with. First steps always falter. But now, two weeks in, I'm beginning to feel more confident. I've made some contributions that I'm happy with, and I'm already feeling more settled with my decision to move.

I don't quite know what I was expecting from Webiny. I already knew the people here a little, and the reason I joined was because I could see such potential in the project. But looking in from the outside you can never get the whole picture.

I went in with ideas about organising campaigns, streamlining delivery of content across multiple platforms, doing SWOT analyses, doing some video content ... but when I stepped aboard, I found out that this is a really focused company. They have one product, one target market, and they want to fulfil the requirements of that market well. Through experimenting they'd already found out what works, and have a good idea of what is going to drive engagement, adoption and growth. Which is really reassuring for me: I don't have to start at "why", nor do I need to define "how", I only have to "do".

My first task was outlined pretty quickly: Build some starter kits with popular frontend frameworks. This would help me get used to using the Headless CMS part of Webiny (Webiny is much more than that, but we'll get to that later), give the company some more inbound links from authoritative domains, and help with user onboarding. It also would leverage my familiarity with these frameworks.

I quickly found a bug: we have a custom rich text renderer package for React, which didn't compile to common javascript, which meant I couldn't use it in Next. This was a great opportunity to make my first contribution to Webiny!

I struggled with figuring out how to customise our babel configuration, and Pavel, the CTO, helped me get through some silly misunderstanding I had about how to do that: how do you customise this babel configuration so that it overrides just this package, not any of the others in our monorepo?

module.exports = require("../../babel.react")({    path: __dirname, })

The answer is simply this:

const defaults = require("../../.babel.react")({
    path: __dirname,
 })

 module.exports = {
     ...defaults,
     plugins: [
         ...defaults.plugins,
         "@babel/plugin-transform-modules-commonjs",
     ]
 }

Pavel and I laughed about how this got me, and thats pretty indicative of the whole team; they've been welcoming, supportive and not critical of my mistakes. Of course, if I was a burden on the team I would expect there to be some further discussions, but you can anticipate a few teething problems from new team mates.

Contributing to open source again #

Contributing to open source is just ... so satisfying. I built something I was pretty proud of at Purplebricks, but it's something I can't even show you, and the code is lost to me. I'm pretty sad about that situation.

On the other hand, I've already made contributions to 2 Webiny repos, and have a draft PR open on NextJS docs. Because it's all public, it seems to me that this code is going to endure and be useful to other people. That is a great motivator for me.

Community interaction #

The other thing I've been excited about is getting more involved in the community via the Slack channel, it's been great to chat to other developers interested in Webiny, and it's helped me build my domain knowledge too.

But much more than that, I've been engaging on Discord servers, on Slack via The New Dynamic's community, as well as HackerNews, Reddit and in other places too.

This has been great fun so far. I know I can't expect every interaction to be positive, but most are. This kind of community engagement scratches the itch I had before, when I wasn't just helping developers out with specific problems but seeing if I could raise a few eyes to the horizon, to see potential where they hadn't before, and to try new things.

Conclusion #

First two weeks, as you see here, has been extremely positive. I'm really excited to see where Webiny as a product goes. Also, I'm blogging again ... which means things are looking up for my mental health."

Freelancing: options and suggestions

Mon, 07 Sep 2020 00:00:00 GMT

Many of us dream someday of freelancing and living a ... less restrictive? ... lifestyle. But is it truly the best option? I freelanced for 7 years before the Credit Crunch killed my business. Looking back, here are a few of the questions I've been asked about freelancing.

Is it better to freelance? #

Is it better to freelance, or to work full time at one company? I think in a lot of ways, it is better to freelance. It does depend on your personality: are you risk averse, or do you mind living off your wits a little?

I freelanced for a number of years, and it went OK. It provided about enough to support my wife and I until the 2008 recession hit … then it got bad really quickly. I used up all of our savings within the first year and had to find alternative employment. For myself, I don’t think I’d consider it now, but that’s mostly because I have 3 young kids to support. However, once they’ve grown up I’d really like to go back to it.

Do I need to be experienced? #

I would say don’t worry about your experience level. The hardest thing to learn is how to manage business relationships, not how to code, and you can only learn both by practice … so I don’t think you need to worry too much, especially if you’re already building things so you have a portfolio (ie you can prove you can do what you're saying).

When I started web development, it was a little bit by accident: I started a graphic design business, but one of my first clients wanted a website as well as branding. So I charged a small fee and learnt as I went, without having a clue how to do it.

I didn't make any money, but I learned enough to get more web development work off the back of that, and building my own site.

Should I find a niche or generalise? #

There are a lot of generalists in the freelance world: people who can work with multiple languages, turn their hand to many different things ... and there are those who specialise in a niche and get work from being specialists in the one subject.

Finding a niche is quite valuable, but there are hazards too. If your niche dries up, then you’re going to have to pivot rather quickly onto another skill. But some niches can be very lucrative, for example I recently read that some government / industrial projects are still hiring FORTRAN developers. That blew my mind.

If you maintain a generalist approach, you can turn your hand to most things. And that means you might be able to afford to be more selective in who you work with.

What tech stack should I use? #

There's no right answer to this one either, if I were to freelance and manage my own clients today, I’d probably do the following:

Make a standard frontend that I can fork and iterate on quickly, probably leaning on an existing library to do the heavy lifting. I’d probably use Gatsby because I’m familiar with it and it has a great plugin ecosystem.
Set up a multi-tenant backend so that I have only 1 codebase to maintain and keep my clients happy. I can’t decide how to do this ... it needs to be something that supports multiple tenants, so that might be a bit complex. I’d likely use Strapi or Webiny, or perhaps even a paid platform such as DatoCMS or Sanity to start with, until I’m confident in my own platform
Set up monitoring and analytics tools on my own platforms using open source software so I can keep an eye on what’s going on without paying too much for it.

This is because I'm a JavaScript developer and I love open source and indie web. You'd likely make a different decision if you knew PHP or another language.

Contracting or freelancing? #

I guess there’s broadly 2 types of freelancing open to developers:

a) Building websites for clients directlyb) Contracting

With contracting, you’re able to keep your hands on the tools, so to speak. But if you are building websites for clients, your primary concern is marketing, building your reputation, and keeping them happy. That involves certain sacrifices, though there’s more chance of “money on a sunday” from that one.

What I mean by “money on a sunday” is that you can charge monthly fees for maintenance and hosting, and therefore you’ll be earning money when you’re not working (on Sundays!). That can become very lucrative. I know someone who makes 40% of their business income from those regular fees, and can afford to not take any more new clients on if they wanted to.

So there’s no clear cut answer … and you could do both: have a few clients and contract in between. It could mean that sometimes you’re working all hours because you need to keep clients happy as well as keep your obligations to your contract firm.

Other tips #

Build your network quickly. Start as soon as possible. Use a small business network to get a foothold, connect with businesses on LinkedIn, canvass among businesses locally. Anything to get you connected.
Get in touch with some good designers who are in need of developers. Quite often they are better at marketing and can pass you referrals if you do them favours in return.
Advertise, advertise, advertise! Don't stop writing or producing content like podcasts, blog posts, vLogs ... be careful that doesn't take up too much of your time but you'll be more respected and likely get more work because of that.
Build in a hosting and maintenance model for residual income. All technology needs to be maintained, if it isn't it'll break down sooner than it should. Sell that idea to your customers. If you can aim to get 20-40% of revenue this way, your life will be much easier.
Build in a support model. Clients will often come to you for small updates or content updates (even if they have a cms!), because either they don't have the skill or the time to do it. Arrange a regular fee for those who come to you on this basis. And charge a high minimum 1-hour fee for those who don't (to push them towards the regular maintenance).
Hire an accountant as soon as you can afford to do so. They can keep on top of things like taxes, late invoices and they'll find out where you can claim or save money."

From Wordpress to Developing in React — Starting to See It

Thu, 12 Oct 2017 00:00:00 GMT

As a frontend designer I've always prided myself on being an HTML and CSS specialist. My use of JavaScript has been limited to animations and DOM manipulation. However, at the agency I work we're branching out from our staple of WordPress and trying different methods of creating websites with functionality that our clients require, whilst maximizing their budget.

We're also bracing ourselves for what we’re anticipating to be the stormfront of Gutenbergfor our WordPress projects. Once it's included in the WordPress Core, writing components in JavaScript for the Gutenberg editor will be necessary for every developer on a project.

So I decided to dive into learning React with a few courses and some experimentation. I had the aim of building a site in Gatsby.js as a test for doing projects built entirely in React.

Letting Go

The first thing I had to do while investigating React was to let go of some preconceived ideas I had – that JSX is just plain weird, that JSS (JavaScript Stylesheets) was an unnecessary abstraction. So I tried not to be bothered by the templating weirdness too much and embrace the differences. My classes soon became classNamess, my <a> tags became <Links>, and I quickly saw what people had been enthusing about.

Managing everything in one place is such a cathartic experience.

With WordPress templating, I used to have function files, filter files, template files, and Sass partials, often with similar names, depending on the framework I'd be using. To build a site meant needing to buffer the entire project in my short-term memory. Not an easy feat to manage.

And then you would complete a project, and forget it all entirely. Until it went to the client…

Every time a client returned with a last-minute change, I would open up the code (which I might not have touched for weeks or months), and try to grok what function lived where, or what I had called that Sass partial. Even with the help of browser dev tools, this can be really annoying.

Now with Gatsby and React, I have my logic in one language, and in a way that makes groking easier. I have my layout, template and config modules at hand in the folder structure, without duplication.

import React from "react"
// Template for blog page
export default function BlogPost({ data }) {
  const post = data.markdownRemark
  return (
    <div>
      <h1>{post.data.title}</h1>
      <div dangerouslySetInnerHTML={{ __html: post.html }} />
    </div>
  )
}
// The data query
export const query = graphql`
  query BlogPostQuery($slug: String!) {
    markdownRemark(fields: { slug: { eq: $slug } }) {
      html
      frontmatter {
        title
        type
      }
    }
  }
`

Whilst this might look weird, it actually makes it much easier to understand what’s going on. You know you’re dealing with this data, using this HTML, and with CSS-in-JS strategies such as Styled Components, you can see exactly what CSS is going to be implemented too. In one file.

Beautiful.

GraphQL: SQL-like data queries

One thing I particularly love about Gatsby.js is its inclusion of GraphQL. Like SQL, you use GraphQL to query your data, whether from the WordPress API, Contentful or Markdown, and extract a dataset to display in your template.

This approach to data is really adaptable. I love the fact that you can install a plugin and query your API endpoints with such ease.

Gatsby.js comes with GraphiQL, which is a simple web-based IDE so you can query your data and get back examples of content immediately. You can then copy this query into your React module and get back the information you need, whether it's the title, content, featured image, categories or any other frontmatter you might have set up.

export const query = graphql`
  query BlogPostQuery($slug: String!) {
    markdownRemark(fields: { slug: { eq: $slug } }) {
      html
      frontmatter {
        title
        type
      }
    }
  }
`

Aside from those pesky tick characters, which are sometimes hard to spot for a newbie, I think this is a great tool, and has sped up my development a significant amount.

CSS in JS

This is the thing that I found hardest about React. I've tried 3 methods so far, and don't really love any of them.

However, since I'm managing my HTML with JavaScript, why not CSS as well? As above, having everything in one place simplifies the workflow and allows you to focus on context without having to grok SCSS again, reducing mental friction.

My Project

Following the tutorial on Gatsbyjs I built up my project from scratch, breaking things profusely at first, but it honestly didn’t take long to gain confidence enough so that I launched my first site at https://freebabylon5.com recently.

Be warned: the tutorial isn’t quite finished yet, you might be better off starting with one of the starter kits already available, so that you get react-helmet and active links implemented, the 2 things I had to learn independently.

The Way Forward?

My initial concerns around using a JavaScript framework such as React seem to have all been allayed. With server-side React, we no longer have a dependency on frontend JavaScript, so progressive enhancement is not just a possibility but a standard. There's momentum towards better accessibility, and for me as a developer, the tools are there (such as Babel, Chrome Dev Tools React extension, and others) for a faster, more efficient development experience.

I'm glad to say I'm sold on the idea and methods of developing with JavaScript, and React in particular. The site I've re-built from WordPress into Gatsby.js is now live at https://freebabylon5.com.

Together with my colleagues at Indigo Tree we’re now looking at using WordPress as a backend, where clients can edit their content without the worries of insecure plugins or other methods of being hacked.

Using GatsbyJS with its “Bring Your Own Data” strategy makes perfect sense, and we’re about to start building our first Gatsby client site using the plugin gatsby-source-wordpress to pull in our data and build a totally secure website with some pretty impressive gains on loading time. We’ll also sleep better at night knowing insecurities in WordPress are no longer putting our clients at risk.

The web is always changing. And the way forward isn't always easy to see. Now, with GatsbyJS, we're beginning to feel like we can visualize where things are going."

Fun with Callbacks

Sun, 26 Feb 2023 00:00:00 GMT

I had the opportunity to integrate a callback function with a module I was building recently. But what is a callback? How do you use them? Let's take a closer look.

I was building a module that could be pulled into a PHP application to upload videos using Uppy. But once the video was uploaded, there was another step that needed to run: the uploaded file needed to be sent to another application to be processed.

The original API I envisaged was something like this. A script could be placed on the page (wherever the backend team wanted the uploader to appear). They could pass a DOM element and a message to be displayed on the upload screen:

<script type="text/javascript">
      largeFileUploader(
        ".large-file-uploader", // target DOM element
        "message", // Message to be displayed
    </script>

The application itself was rendered with Uppy, and included some form validation and other things to help the users upload their videos easily:

async function largeFileUploader(
  targetDomElement,
  onScreenMessage,
  callback
) {
  const uppy = new Uppy({
    .use(Dashboard, {
      target: targetDomElement,
      inline: true,
      note: onScreenMessage,
      height: 470,
      theme: "auto",
      showProgressDetails: true,
    })
    // ... other configuration
   Uppy.on("complete", () => {
   // do something when Uppy has uploaded all files
   };
});

To provide functionality for the callback, I passed an extra parameter to my function:

<script type="text/javascript">
      largeFileUploader(
        // target DOM element
        ".large-file-uploader",
        // Message to be displayed
        "message",
        () => {
          // this code runs inside the context of the uploader
        }
    </script>

With this extra parameter, you can run the function inside the context of your application and do something with the result, for example:

<script type="text/javascript">
      largeFileUploader(
        ".large-file-uploader", 
        "message",
        async (formData) => {
          try {
            const getStuff = await fetch("https://jsonplaceholder.typicode.com/posts/1", {
            method: "GET",
            headers: {
              "Content-Type": "application/json",
            },
          })
          const getStuffJson = await getStuff.json()
          return { success: 'message' }
          } catch (error) {
            return error?.message
          }
        });
    </script>

This allows us to provide an extra notification, rendered by Uppy, to inform users that the video has been sent for processing, even though this step has happened outside of the context of Uppy. As my friend Chris Geary said, "The callback you pass is just a reference to a function, so in theory, as long as that reference remains in scope, you can call it."

export function largeFileUploader() {
   ... // other code
   Uppy.on("complete", async (result) => {
      const data = await callback()
      if (data?.success) {
        uppy.info(`✅ Video processing initiated`, "info", 15000);
      }
      if (data?.error) {
        uppy.info(
          `⛔️ Upload to processor failed, please contact support.`,
          "error",
          15000
        );
}

Callbacks are super useful for rendering extra functionality provided by the people using your application code.

Predictions for GatsbyConf 2021

Fri, 19 Feb 2021 00:00:00 GMT

It's no secret that NextJS is knocked the socks off Gatsby last year in terms of developer adoption. I think the frameworks will keep learning from each other (read: steal features), and it's a good thing for both frameworks.

Dustin Schau asked in this tweet, "What do you think we're launching at #GatsbyConf on March 2nd?"

Here's my take on what could be announced at Gatsby's forthcoming conference.

1. Hosting on Gatsby Cloud #

Using Gatsby Cloud as part of a preview environment has been a great innovation, especially the work that has gone into the speed and ease of incremental builds.

Building out this existing functionality into a fully-fledged hosting platform to compete with the likes of Vercel and Netlify makes sense, especially if deploys benefit from that same speed.

2. Server functions #

One of the reasons many developers love NextJs is that it isn't restricted to being exported as a static site. Vercel, Netlify and AWS Amplify and some other platforms allow you to run serverless functions which complement the application, so that you don't have to render all of your content on build, instead when a page is requested for the first time, it can be generated then.

Allowing this isomorphic approach is definitely on Gatsby's radar, and it would really be good to see it a first class citizen on Gatsby Cloud, their hosting platform that will compete with Vercel, Netlify, and all the other offerings that have sprung up in the last year or two.

This will probably mean those platforms will have to do some work to ensure they are compatible with this new way of using Gatsby, so expect a bit of a delay before they are able to use this new version of Gatsby.

3. GraphQL Toolkit #

One of my biggest issues with Gatsby at the moment is that data from GraphQL APIs isn't able to be transformed. If you have a CMS that supplies rich text or Markdown, you have to write that transformation in the template, so it gets packaged into your bundle and transformed in the JavaScript runtime.

It's a huge problem, but the GraphQL Toolkit project looks really promising. Here's hoping it's ready to be merged in!

4. Some new LowCode / NoCode features #

Gatsby is aiming for a very different market to NextJS ... it's chief market are newer or less experienced developers who are building marketing sites for smaller organisations. WordPress proved that there's a huge volume of these and plenty of scope to facilitate a world where developers can pick a theme, add content, make some customisations, set up hosting and charge a client a fee for doing so.

To what extent Gatsby wants to enter this market is anybody's guess. I'm hoping they don't get too invested into it; I think it's healthy for Gatsby to still take on the NextJS market (to my mind this is more experienced developers who want a React & Node framework for their applications). However there's good money to be had and its doing a much better job than the current incumbent in this space."

Getting Started with TypeScript

Thu, 22 Aug 2019 00:00:00 GMT

Mention TypeScript to a developer who works with JavaScript and you'll probably get one of two responses: they will either struggle to understand why someone would NOT want to use TypeScript, or they will struggle to see why someone WOULD use it.

I think this sometimes depends on where you started your journey to JavaScript. Often people come to the language from a server-side language such as C or Java, which are statically typed. In these languages, dynamic typing is seen as a real hazard, an unsafe practice.

But if you've started your journey as a developer from HTML and CSS, you've probably been more used to the generally more ... dynamic ... world of the frontend.

If this is the case, but you hate the fact that your application is blowing up in production, and the cause is often a typeError, or you want to improve communication between developers working on a project, TypeScript can be a good solution.

In this article we're going to introduce some of the concepts of TypeScript using Styled Components. I've chosen this route because I think it gives us a good insight into how certain features can be used without introducing more complicated concepts too soon.

Let's Get Started! #

For this tutorial, we're going to take a react template that uses styled components and move it to TypeScript.

The original code we're going to use is here: https://codesandbox.io/s/netmag-react-typescript-4l4cj

Fork this code, and open on the Container.js module. We'll work through it together to convert it to TypeScript.

The project on CodeSandbox already has a compiler set up for TypeScript. So we can start by renaming our file. Instead of .js at the end, change it to .tsx.

Don't forget the x! This tells the compiler to use JSX, the templating language often used with React. If you didn't have an x at the end, you would start to see some very verbose errors very quickly.

Already, TypeScript is working for us. By forcing us to have an x at the end of the file, we can see that this module contains a JSX template. So it's easier to see at a glance what's going on in each file.

What is an Interface? #

TypeScript works by forcing us to tell it what properties are going into, and coming out of, our functions. Since styled components are functions, we have to type these as well.

Let's add our first interface!

export interface ISContainerProps {}

Here we're declaring what will interface with our function. As a convention we should use a capital I at the start of the name so that we can more easily see later that it is an interface, instead of anything else (more about what else it could be in a bit ...)

Let's add our first property:

export interface ISContainerProps {  bgColor: string;}

I've given bgColor the type "string" here. This means that it can only be a string value. If someone comes to use our component and writes bgColor={42}, they'll see a red underline, indicating an invalid type has been used! Usually this means a PR won't pass quality checks or you will be prevented from pushing code to a branch in the repo.

I want to restrict the use of the next property, text colour, a little more than that. I want them to have the choice between only black and gray text. Because of this, I'm going to rename textColour to lighterText:

    lighterText?: boolean;

The question mark (?) tells the compiler that lighterText is an optional property. If nothing is declared, it'll be false by defaut. Within my styled component, I can output grey or black based on this boolean value:

    color: ${props.lighterText => (props.lighterText ? `black` : `gray`)};

This can be shortened a little bit by some destructuring:

    color: ${({ lighterText }) => (lighterText ? `black` : `gray`)};

Introducing Enums: A Type of Types #

For my final property, I want to introduce you to the humble enum. enums, I'm told by reliable sources (wikipedia), have been around for a while. Instead of a normal type, say a boolean, string or number, an enum is it's own type.

My enum is going to declare the container widths I want people to have access to:

export enum EContainerWidth {
  SMALL = "36rem",
  MEDIUM = "48rem",
  LARGE = "64rem"
}

Can you see the name I've given the enum starts with a capital E? This is the equivalent of using a capital I for the interface.

Now I'm going to declare that my containerWidth property is a type of EContainerWidth:

export interface ISContainerProps {
  bgColor: string;
  lighterText: boolean;
  containerWidth: EContainerWidth;
}

This makes sure someone can only use these values for the container width. In this way, how a developer uses this module can be controlled more tightly. So, no more accidentally setting the container width to something that doesn't match the design. It also helps a developer to stop and think more carefully about their choices.

So, now, how do I tell my styled component about my interface? By adding chevrons after the tag declaration, but before the body of the function (inside the backticks), and writing the name of my interface there:

export interface ISContainerProps {
  containerWidth: EContainerWidth;
}

const SContainer =
  styled.div <
  ISContainerProps >
  `
    max-width: ${({ containerWidth }) => containerWidth}
`;

Typing the Template #

Now we need to tell the compiler what the type is for our JSX template. I'm using a React functional component (FC) for this project. The type for this can be imported directly from React at the top of our module:

import { FC } from "react";

export const Container: FC = ({

Now we need to pass in our properties. They're the same as the ones we've used above, with one addition: the children, other react nodes we want to render inside the template.

export interface IContainerProps {}

export const Container: FC<IContainerProps> = ({
    ...
})

Interfaces can be re-used and extended by other interfaces. This is going to be particularly useful when it comes to our JSX template:

export interface IContainerProps extends ISContainerProps {}

Now the template has the same types as our styled component. Now let's add the additional property. We mentioned that this was a React node, and that's another type we can import from react:

import {FC, ReactNode} from 'react'

...

export interface IContainerProps extends ISContainerProps {
    children: ReactNode;
}

We are ready to use our template!

Using the Template #

We've declared an enum for our container width. We need other developers to have access to this when using our template.

Because we have already exported our enum, we can import it along with our component:

import Container, { EContainerWidth } from "./Container";

I have a global file in my src/ folder (usually I call it "tokens") where all of my project's enums are kept. This is much better for helping others discover what properties they have access to, and it's much easier to re-use them.

Now, instead of assigning properties to the template at will, every time you or another developer comes to use it, you or they will have reason to pause and make sure they're using it in the way it was intended:

<Container
  bgColor="red"
  lighterText={true}
  containerWidth={EContainerWidth.MEDIUM}
>
  JavaScript Container
</Container>

If you got this far and followed along, congratulations on making your first TypeScript component!

Start Using TypeScript today #

TypeScript is one of the fastest growing trends in JavaScript. Reinforcing types in your application can help you avoid making fatal errors in production, especially when working with teams of developers. By using TypeScript we can lean on our IDEs more and simplify our workflow, introduce self-documentation and better code hinting.

Boxout: What is TypeScript? #

TypeScript was developed by Google specifically for the frontend framework AngularJS. It's not a separate language to JavaScript, but a superset, it adds to the existing language. You can write in normal JavaScript, but you add special syntax to identify types for functions, variables etc.

TypeScript needs to be compiled to normal JavaScript to be used on a browser. But this is an advantage: errors that are generated can be caught and mitigated before anything goes into production.

Boxout: Static vs Dynamic Typing #

One of the most powerful features of any programming language is the ability to store things in allocated memory, often called a variable, the value of which can be updated or changed by the program.

// define a variable
var a = 1;

console.log(a); // 1

// define a function which will change it's value
function updateAVariable() {
  a = "a is now a string";
}

// call the function
updateAVariable();

console.log(a); // "a is now a string"

This demonstrates how powerful JavaScript's dynamic typing is. The variable a can be anything: a string of letters, a number, whatever you want.

The problem with that is it can easily break stuff:

function addOneToA() {
  a = a + 1;
}
addOneToA();

console.log(a);

// 'a is now a string1'

By contrast, static typing means that you'll only use one type of thing, and if that changes, you will know about it before you even run the code:

var a: number = 1;

function updateAVariable() {
  a = "a is now a string";
  // typeError!
}

Imagine our variable comes from another location, which is written or controlled by another team. Imagine that variable, somehow and without your knowledge, starts returning a string instead of a number.

In that case, we now have a failure in the application. It could be crashing in production. The only way to check it might be to manually go through the checkout process, add something to your cart and try to purchase an item.

Instead, your code editor, terminal or CI tool lets you know, potentially before even running the code that there could be the possibility of a failure like this in your application.

That's one reason why I like using Typescript: my code editor tells me where I've made a mistake, and it won't let me push code to my Git repo unless some TypeScript rules I've given it are followed.

Boxout: Resources #

There are some great resources for TypeScript out there, and they're growing as more people write about their experience with the language. Here are some places to start:

https://www.typescriptlang.org/
https://definitelytyped.org/guides/best-practices.html
https://github.com/dzharii/awesome-typescript

The main problem I have with TypeScript is that people writing about it assume you've got good conceptual programming understanding to begin with, so it's hard to pick up for newer developers.

If you want a good beginner level course on TypeScript, I can really recommend this one by Scott Tolinski. Scott is great at explaining things simply and clearly without assuming too much:

https://www.leveluptutorials.com/tutorials/level-1-typescript"

Hidden Histories

Wed, 01 Apr 2026 00:00:00 GMT

I feel I need to start this article with a brief explanation: Welsh is one of the languages spoken in the British Isles.

It seems incredible to me that I need to explain this, but in my experience a lot of people I've personally encountered don't seem to be aware of its existence.

The Welsh language is not Gaelic. It's distantly related to the language of Ireland, but they went their separate ways many centuries ago.

It's more accurately referred to as Brythonic. In fact, if you go back far enough on maps or books, you'll find it referred to as "The British language".

And there begins our journey of discovery.

An Awakening

I heard recently of a musician who had been dead set against learning Welsh who, when he finally picked it up, has done so enthusiastically: as a musician he appreciates the beautiful lyrical quality of the language, how it naturally lends itself to song, poetry and music.

His experience mirrors my own: I hated learning Welsh when I was young. But now, for me, it has unlocked my own history, one which I was completely unaware of; it's made my experience of living here that much richer and significant.

I think I can only appreciate this as an adult; now I'm older I am more aware of and appreciate more the efforts of my parents, and by extension, my extended family, in shaping who I am.

A Hidden Heritage

Wales has shaped the identity of many people in the British Isles, but many don't even realise it.

For example, the Welsh are often mocked for having few surnames. In one place there are a whole village of people with the surname "Morris". "Jones" and "Davies" are very common too.

What people fail to realise is that many other "English" surnames have a Welsh heritage too.

"Powell" and "Bevan" include a leftover from the Welsh prefix "ap", which means "son of". What was once "ap Howell" is now "Powell". "ap Evan" simlarly became "Bevan".

Here's another example, and it's probably an even more interesting one: the origins of place names. Wherever you see "tre", "pont" or "caer", you're looking at a Welsh place name.

There are a number of rivers in Britain called "Avon", the Welsh word for "river".

This is because Welsh isn't ... Welsh. The whole of the British mainland once spoke Welsh, before the invasion of first the Romans, then the Anglo Saxon tribes ... which is where we now get the name "England".

Interestingly the Welsh refer to England as "Lloeger", the "lost lands".

For these reasons, I strongly suspect there are many more people in England who will be able to trace at least part of their heritage to a Welsh speaking person as well as Angles, Saxons, Romans and Normans.

I find it unfortunate that the heritage of these other peoples prevail in our culture, whereas Welsh culture is sidelined and sometimes ridiculed.

Ridiculed then valued

The territory of Wales has been reduced even more in recent decades, with more English speaking people moving in who don't see the need to learn the language. "They all speak English anyway" is a common reason for monolingualism.

But so do Native American peoples, Maori and indigenous Australians. I feel it's a shame to miss out on this opportunity to get to know the people and land around you. Also, there are clearly researched academic and life skills benefits to raising children in bilingual households.

There's a lot of water under the bridge between English speaking and Welsh speaking people, and I'm loath to speak about it in any detail. Suffice to say the wounds are still there, scarred over and sometimes mentioned in passing ... and only among Welsh speakers and in academic circles.

But given that history I find myself wanting to defend the language.

"Welsh is a foreign language" is one I heard recently. I find this perhaps more uncomfortable than most, since the meaning of the word "Welsh" is "foreigner". Essentially, Welsh speakers are foreigners even in their own homes!

But the times are changing: I recently spoke to a mother from an almost entirely English speaking part of Wales who wanted to learn Welsh. Why? "My grandkids speak Welsh to each other, and I want to be able to speak to them in the language of their hearts."

I find this fascinating; the demographic of Welsh speakers is changing: in the older generation its more common to find speakers, then there's a dip where Welsh as a language was actively oppressed, and now it's being encouraged again so an increasing number of young people speak it.

I wonder what this will do to the language?

At any rate, in a world where there's a lot of movement of families and individuals, I'm sure it will continue to change.

And I'm ok with that, as long as it doesn't disappear.

Welsh music

If you enjoyed reading about this cultural sidequest of mine, maybe you'd be interested to hear about Welsh music. As I mentioned earlier, the language expressed in song is often mesmerisingly beautiful.

Look up "Mari Matthias", "Bwncath" and "Adwaith" on any music streaming service to see what I mean.

How I Created My Home Web Server

Sat, 18 Feb 2017 00:00:00 GMT

Is it just me or does Apple's iCloud seem like a very simple solution to a problem all of us in a connected world face? That of storing, managing and sharing photos of our loved ones, holidays and interests with others?

Instead of being faced with ever increasing charges from Apple to store my stuff, I decided to get a solution I could be more in control of.

You Will Need #

I could have done this project by setting it up on my own existing home computer, but I already had a spare laptop that had been given to me. I also thought it would give me the freedom to wipe everything and start again if I messed it up irreversibly (which, coincidentally, did end up happening!).

I decided that I'd like to use the current LTS issue of Debian for my hosting platform. I liked the idea of using a bare bones OS, something that I hoped I would be able to customise and learn with. I installed this OS and got to know it, especially how to update and install core and third party packages.

The main commands for this are:

$ sudo apt-get update
$ sudo apt-get upgrade
$ sudo apt-get dist-upgrade

First hitch: Packages not upgrading #

About the first thing I noticed when I went into the Terminal app from the GUI was this message that I could upgrade packages, but when I tried to install stuff, it said that no packages were updated.

After some research, I found out it was because I needed to extend the Sources list to include some other sources, as explained on the wiki.

I was then able to install the AMP stack (Apache, Mysql, Php) and it gave me immense pleasure to see a "hello world" message when I added a php file to the www directory.

Second hitch: Permissions #

At this point, I found that I was having trouble getting the right permissions for PHP and MySQL to operate.

I didn't know at the time that the solution is to create a new group (its usually "www"), add the current user to the group, then add the user used by Apache (www-data) to this group.

Third hitch: Sharing #

At this point, I got stuck trying to see how to see my website from another computer in my local network. It seemed that whatever IP I went to, nothing worked.

I went down a bit of a maze, installing Samba file sharing system before second-guessing myself into the belief that I had taken a serious wrong turn somewhere previously. And now I had Samba, NFS and other file sharing services turned on, potentially exposing my computer to weaknesses.

So in order to progress quickly and avoid having to retrace all my steps, I wiped the computer.

It might seem a little drastic, that, but it did get me out of that situation quickly!!

Second Attempt #

Even though there were a lot of reasons to like Debian I decided that the community support just wasn't there for me. It's not a criticism: those who use Debian really know what they're doing, but for that reason it's hard to find support for it.

Also, I knew that Ubuntu came with a lot of the things I would need preinstalled.

So I downloaded Ubuntu Server, reformatted my install USB, and started again!

This time I discovered the ifconfig command really quickly on support sites, and was very soon able to use SSH to log in to my new server. That was an amazing moment for me!

Soon after that, I installed RSA keys and was able to log in much more easily and safely to my device. I went on to install & configure my AMP stack, and this time view my Hello World app right from my laptop.

I uploaded a website I'd been building that would serve as a replacement for our current iCloud workflow, which required a few tweaks to permissions again (creating a group for www-data user as I mentioned above) and could access the site.

Port Forwarding #

I went into my router's control panel and turned on port forwarding for my web's default port. Almost instantly, I saw my website ... but could no longer access my routers control panel!!

This suits me for now, because I don't have to worry about any other ports being open, and so there's also less vulnerability for my router. But I also don't have SSH access from outside my home network.

I'm going to fix this, but I don't currently have the headspace to go in and configure it and my port forwarding again.

Changing IP Addresses #

At first, I was amazed at how quickly my home IP address would change. I realised that if I wanted to reduce outages, I needed a method to keep my site live enough for it to be useful by our family.

I now have a free plan with No-Ip, which has installed a program on my server to update a DNS server when my IP address changes. If this works successfully, I will likely get a paid pla, unless I can figure out how to do it myself!

Fourth hitch: Uploading files using the CMS #

I how had a working website I could use and refine. But when I tried to upload files I came across an error where the file could be uploaded, but couldn't be moved to the content directory.

Again, this was a permissions issue I had, where I needed to give the "nobody" user the ability to write to disc. I did this by adding this account to the user group I had created earlier.

Conclusions #

Setting up my own home server has been frustrating at times, but its taught me a lot about the underlying technologies of the web.

Given that, I'd really like to try other projects in the future that will help me to better understand how browsers and the internet work."

How I'm doing

Sun, 10 Mar 2024 00:00:00 GMT

I very rarely write personal articles on this blog. But with the demise of social media I've decided to share a little more of my personal thoughts here.

The main motivation is that I've discovered a lovely network of personal blogs, and even a decent directory. The articles I've found myself enjoying more than I thought I would have been the personal ones.

I guess this started happening a lot more when Molly passed away. I enjoyed reading other people's tributes to her, and found how similar they were to my own few experiences with this incredibly inspiring individual.

Career change

I've also reached a bit of an inflection point in my career. I have a stable job with a company that's ... ok, actually. It's never going to set the world alight, but frankly neither is my code. I've long thought that it's not what you do for a job that matters as much as who you do it for. And this company is pretty accommodating and stable.

So along with that, my need to prove myself has diminished. I'm not saying I'm confident, but in terms of my ability or desire to succeed, I think I'm at the apex of this particular career. I'm always planning for the future and have some objectives that will change things up a bit and help me to remain comfortably uncomfortable, as I'm still trying to live by the saying "if you're the smartest person in the room, you're in the wrong room."

Relocating to North Wales

We had an unexpected visit from our landlord about 3 months ago that meant we had to move out. My wife and I were sad about that because we had only just got settled, having been there for only 2 years.

We don't have the circumstances to buy, and rents were going up crazy amounts in Hertfordshire.

So we decided, very quickly, to move up to Flintshire in North Wales.

This was motivated largely by our kids: They've been learning Welsh for around 3 years. This is something they've decided to do of their own volition, but we wanted to support it as well, and give them an opportunity for future careers in the language if they wanted to.

We upped and moved in six weeks. It was terrifying, exhausting, and extremely stressful. We also had to get rid of a lot of personal belongings. But the result has made a huge positive difference to the well being of the whole family.

So we're all learning Welsh now. And enjoying it here immensely.

Writing blogs is hard when you prioritise your family

The other reason I've neglected this blog a little is that writing this blog in Markdown, committing it to the repo and publishing it, even without my revision flow, is slow and cumbersome.

When I'm not working I'm 100% focussed on the family, which means I don't get much time to myself at all. That's a personal choice, but it has consequences. I want to be part of their lives forever, not just for a few years, so I have to take care to maintain my relationships with each of them now.

I do wish there was a better workflow. I want one of those apps that integrates with my workflow so I can write articles on my phone, much like you can do with the WordPress app.

I just tried writing Markdown with the GitHub app. It's not the solution I was hoping it might become sadly. At least not yet.

But the infrastructure is dead simple. I'm not going to beat markdown on a static site. So it looks like there's no good way to beat that friction currently.

Shame. But also, not a big deal really.

Fin

There you go. I have a few more things I wanted to say, but I'll leave it here: it's late, the children are finally asleep so I have ten minutes of doom scrolling before bed.

How are you doing?

PS I'm building a comment section, so when that's live I hope it'll give people an opportunity to reply.

How to resolve Jest issues: tests passing, but code coverage fails!

Mon, 02 Aug 2021 00:00:00 GMT

Today I'm continuing with my trend of making silly mistakes so you don't have to.

The subject today is asynchronous tests in Jest. I've spent waay too much time on this one, and I don't want you to have the same trouble.

I'm testing whether a page renders or not. The page takes some time to contact an API and therefore to render, so I've used the waitFor helper in Jest to assert what should happen.

As I've mentioned the test setup is slightly immaterial, however I'm writing this rather quickly before the kids get hungry.

Here's the test using waitFor:

it("renders the page", () => {
  render(
    <ThemeProvider theme={defaultTheme}>
      <MockedProvider mocks={mocks} addTypename={false}>
        <IndexPage />
      </MockedProvider>
    </ThemeProvider>
  );

  waitFor(() => {
    expect(screen.getByText(/messages/i)).toBeInTheDocument();
  });
});

Anything wrong with this test? No?

Look again. The documentation in fact plainly says this at the top of the page:

"The async methods return Promises, so be sure to use await or .then when calling them."

Source: https://testing-library.com/docs/dom-testing-library/api-async/

What this doesn't do is show up in your tests. It'll look like they've passed!

The only reason I came across it was because when I use --codeCoverage to make sure I've covered all of my code with tests, it shows up as uncovered lines. But also, you'll notice there is an obscure message in the terminal about this too:

ReferenceError: You are trying to access a property or method of the Jest environment after it has been torn down.

Basically the assertion cannot be verified because it's no longer there, the render phase has passed. Although why this results in passing tests is anybody's guess. (Please let me know in the comments if you know!).

Here's an example of a working test:

it("renders the page", async () => {
  render(
    <ThemeProvider theme={defaultTheme}>
      <MockedProvider mocks={mocks} addTypename={false}>
        <IndexPage />
      </MockedProvider>
    </ThemeProvider>
  );

  await waitFor(() => {
    expect(screen.getByText(/messages/i)).toBeInTheDocument();
  });
});

There you go, I've wasted hours of my precious life so you (hopefully!) don't have to!"

How Website Forms Work

Fri, 06 Jul 2018 00:00:00 GMT

HTML Forms are one of the most fundamental elements in the web builder's toolbox. It would be a rare website that doesn't use this element to complete any of the most basic tasks someone might want to achieve. But there's a lot that goes on behind the scenes. Here's a bit more of an in-depth look at what happens when you click 'Send' on a form, and why you need some server-side code (somewhere) to handle that process.

First of all, we need a short history lesson about the form HTML element.

The Origin of Forms #

The <form> element was originally intended to be used to post content directly to web pages. So a person visiting a web page could modify what they found there, adding to it, removing parts or all of it entirely.

This feature never made it out of the lab. But it was a pretty neat, if not idealistic, concept. Imagine how different things would be if anyone could change your website. This kind of idea was eventually tried out by a few intrepid individuals, notably Wikipedia, who were allowed to edit pages without any checks or balances. Unfortunately it led to widespread abuse. The sites were often either shut down or restricted after a short period of time.

But the basic need still remained: as a website visitor I want to send my email address so that the website owner can contact me is the core one. There are many others: making a purchase, submitting a comment for display on the website, purchase an item, and more.

People realised that they needed some method of collecting the data people had entered on their forms. They needed to validate it, making sure fields were filled out and not left blank, that they used a number where that was expected, a date when that was required etc. Then they needed to send that to an email address or other consumer so the person could be responded to, their comment displayed (or not), their order fulfilled, or whatever.

That requirement led to the necessity for server-side code.

Server Side Form Validation #

One of the earliest and most popular languages to tackle this issue was PHP. You could include some server-side code right there on the page. Now when someone clicked "send" or "submit", the PHP would collect the data, format it as an email, then send it.

It did this by the form's method and action attributes. The method was usually a POST request, a new entry to be added. The action attribute could be set as a URL, either the page you were on, or another page, that when sent could be triggered.

Then the PHP would receive that into the global $_POST variable, where it could be processed.

Now people could not only read messages from the website as if the person had merely sent them an email, but they could check the form had been filled out properly too. You didn't have to send it to an email address though. You could accept a payment, submit a comment for approval via a CMS, or connect with a CRM to collect and analyse leads.

But these all required server-side code. Some application or at least a single function that would start running when the form action was triggered by someone pressing "submit".

Great! Unless you have no server-side code.

JAMStack Caveat #

Going "Serverless", using the "JAMStack" means, in effect, going back to the roots of the web. We are approaching websites once more as we did before we had server-side code.

Though this has huge benefits, there are new (and perhaps old) issues to overcome. The primary one being, "how do we validate, process and send form data?"

Processing Forms Using Client-Side Code #

Some people realised that once you submit code to your server, you can gain access to that server remotely and do nefarious things to it. This was entirely dependent on how well people had written their server code, but there were often flaws to be found and exploited.

This resulted in website owners starting to validate code on the client using JavaScript before the form data was submitted. Now form submissions could be more sanitised before they got to the server, and also feedback for the person filling out the form could be much faster. You could tell them it wasn't valid as they were typing, instead of waiting until after they clicked "send".

You might think we can therefore use client-side JavaScript to submit form data. But the trouble with client-side code is that it is all public. And therefore it can be manipulated by someone to perform actions you don't want to happen. For instance, someone could change the URL, and they might get lucky, hitting another API endpoint that exposes a vulnerability. Or they could modify the form data to contain malicious code that takes control of your computer via your email client.

That would be bad.

So we still need some way of submitting code via a private resource. Something that isn't visible or accessible by everyone.

Form Submission Without a Server #

We currently have a few options when it comes to serverless form processing:

Use a 3rd party service
Use a 3rd party function

It's all about how much control you want, budget constraints you have and what you need to achieve.

Popular 3rd Party Services #

Even before the advent of serverless, there were 3rd parties that offered form processing as a service. These are usually paid-for, often use their own form code (with varying standards), often with a version you can modify via their parameters or CSS rules.

Wufoo is probably the oldest of these. But there's also Typeform, Mailchimp and others. If you use Netlify, you can use their Forms system, for which implementation is extremely easy.

However there are often restrictions with these services. Some plans begin to get expensive quickly. Other features you may require, for example, sending a receipt confirmation to the sender, are impossible, or are branded with the providers' details.

Another option is to write your own function.

3rd Party Functions #

With the advent of Serverless, you can now submit your form to a URL that contains a form processor running in isolation from any other code, inside its own "container", or virtual server instance.

Amazon Web Services have their popular version called Lambda. Microsoft has Azure. There's also Heroku, owned by Salesforce. All of these provide an environment of server side code of your choice (PHP, Node, Go etc, with different release versions). This means you can write your own form processor, specify your environment language and any variables you want to utilise, and then deploy it.

This allows you complete control over everything you might want to do with your form. An added bonus is that it is run independently of the website (so no application load time for your users), and can be modified independently at any time.

In this case the form method will still be POST, but the action parameter will be the URL of the serverless function.

Often these services are low-cost, meaning you only pay for the server time you actually use. Processing time is often sandboxed to 1 minute (probably much more than enough to interfere with this use case), and there is no server infrastructure to worry about.

An extra benefit is that if form submissions become extra busy, with concurrent POST requests, the function container will be duplicated enough times to meet demand. Automatically.

It's out of scope for this article how to write a serverless function. However, it's just a normal piece of code that you might want to implement anywhere.

Form The Future #

Form have had a long history on the web. Now we're approaching things from a serverless perspective, we can address the needs of people who use our websites in a much more robust way by abstracting away what holds them back from interacting with our site."

Improving Performance on Deliciousreverie.co.uk

Tue, 06 Oct 2015 00:00:00 GMT

I can't possibly think I could get away with a post about performancewhen my own site wasn't as spot-on as I could get. So I've been spending a bit of time investigating how to reduce my load times. This is what I found out and how I implemented it.

I built this site in December 2014 when I was on paternity leave. Now that my little bairn is finally sleeping through the night I can look at my code again through significantly less zombified eyes. I found a few things I'd like to improve on, namely:

Reduce unused DOM elements
Use non-blocking HTTP requests (at least on the home page)
Streamline my CSS

Let's break down each of these and see what can be done to improve them.

Reducing unused DOM elements #

When I built this site, I originally envisioned a main (central) content area, with sidebars of supplementary content. But at the time, I was beginning to realise that normal people don't understand sidebars.

I have since decided to just focus on delivering good content in a branded experience. That's the core of what I wanted to achieve here.

So the sidebar elements are gone, along with their CSS.

Use non-blocking HTTP requests #

My CSS is pretty important to the experience of the site, but I'm aware that by linking to an external stylesheet in the header was creating an extra round-trip for the browser before the content could be rendered.

Since my CSS is pretty lean on this project (119 lines, 2kb uncompressed or minified), I opted not to use a Taskrunner tool but instead use a PHP include instead:

 <style><?php include 'assets/css/deliciousreverie.min.css'; ?></style>

I have a unique header file for the home page, so on my other pages, the header still renders via 'perch_get_css' as normal.

If my CSS code was much larger, I would find a tool to identify all necessary "above-the-fold" css and abstract that out into a separate CSS file for inclusion.

3. Streamline CSS #

I have tended to use normalize.css to standardise delivery of my site across different browsers, just because I thought that's what everybody else did.

However, after having a chat to someone I admire, I realised that following the crowd in this instance was costing me in terms of performance.

When you include something like normalise.css, you're adding extra work for the browser to parse, often without rendering much of it. And you're potentially adding an extra CSS overwrite to your styles. For instance, if you want a unique checkbox input style, normalise already includes one extra style. So now we have 3 different renders: 1) The User Agent (browser) styles, 2) Normalize styles, 3) Your styles.

I've made a promise with myself to look at the source code of these libraries before I make any assumptions about what I should include in my builds in the future!

Conclusion #

I've gained some great results from these excercises:

Seeing this result on Web Page Test was a big encouragement to me:- I had achieved much of what I wanted to.

I'm especially proud of the fact that I have no images at all on my home page. The render chart is similarly encouraging:

As you can see here, I have made some huge sacrifices for including three Google fonts. On the other hand, I have had a big win with regards to images, it feels nice to splash out a bit on this form of branding.

So, great. For my next challenge I want to see what I can do to streamline Wordpress. Whilst I don't think you can ever get as good results from that CMS as Perch allows you to, I'm sure I can think of some ways of getting better results.

What results have you had from taking a closer look at the performance of your site? I'd love to hear what you come up with - please tweet me on @muzzlehatch_!"

Introducing Webiny Enterprise CMS+

Wed, 12 Oct 2022 00:00:00 GMT

We have struggled for a long time on the question of how to position Webiny as a product. From the start, it was clear that we we were building was a feature-rich product that didn't quite sit firmly in the camp of "Headless CMS", yet that is what the majority of our users focused in on.

As a result, we have doubled down on that particular marketing, building more features and explanation around the concept of Webiny as a Headless CMS. However, as I'm sure you realize, it's an extremely crowded space. And we still didn't quite fit in.

You see, Webiny isn't just one product. It's four fundamental ones:

Content management system
File storage and Management
No-code Web Page Builder
Form builder

In reality, it's actually more than even those products: Not every CMS can scale to tens of thousands of users, serve content for millions of website pages and application interfaces, and be as easily customizable as Webiny is.

We frequently hear about teams using Webiny in unique and different ways: from wealth management portals, to BitCoin exchanges, to online course providers, to customer service desks, to a no-code form builder. It's been amazing to watch different teams do all of this with Webiny and more.

Not every CMS can become all — or any — of these things.

But if we were to look at the terminology around the kind of tools on the market that have those features ... well, that's not what we want to be either.

What if You Need More Than a Headless CMS?

Products that we could compete with that are more than a CMS are varied, but the basic premise is the same: allow large, complex, multinational organizations with content creators in different locations and languages to populate their websites with products, blog posts and other information on a regular basis. This might not just involve writing content, but also creating short-lived product launch pages, gaining feedback using a web-based form, or sharing product literature with vendors and potential contractors.

Popular acronyms in this space are WCM ("Web Content Manager"), and DXP ("Digital Experience Platforms").

These systems are often very clunky, with difficult-to-navigate interfaces. Customization options are very limited, and they often require hiring specialized individuals to manage them. Not to mention the fact that they're nearly always hugely expensive SAAS platforms that exist outside the organization's perimeter.

With the features we have, it would be possible to become a competitor in this space. But Webiny is much more nimble, adaptable and on top of that, is also self-hosted.

By Any Other Name

So we stopped and asked ourselves: do tools to manage content on the web really need more acronyms? CMS is a pretty good fit, and even the "headless" part seems to be on it's way to being generally accepted.

And, we thought, if we adopt that terminology, not only would we be alienating the fact that medium and small businesses can also depend on Webiny for their content, other teams might start to get the impression that we are also a clunky system that's prohibitively expensive, not customizable, and overall difficult to use.

No, we don't want to pigeonhole ourselves like that. Yet, Webiny is more than a CMS.

An All-in-One Solution

So we're sticking with "Headless CMS" as a description of Webiny. But opening a conversation on that front can sometimes present a problem: what if the organization approaching us doesn't have the requirement of a content store. Perhaps instead they're looking for a way to dynamically populating content from the CMS into a no-code page builder? Or Perhaps they need an advanced publishing workflow for their teams to be able to create, edit, update, approve, and publish their content?

Starting off the conversation "We're a Headless CMS" unfortunately leaves out all of the other cool features we've built, or are planning to build in the next few months.

These new features will enable organizations to build dynamic content quickly, ahead of the competition, without the involvement of technical teams past the initial setup, and do so at a fraction of the cost of other providers. And to do it at the scale that enterprise organizations require.

Yes, for those organizations who need that kind of power, Webiny is an all-in-one solution that might be able to replace several costly products that are strung together with digital duct tape, not designed to work together, and are also individually difficult to work with.

For those people, Webiny isn't just a CMS. It's a CMS Plus.

What Is Webiny Enterprise Headless CMS+?

Plus.

It's succinct, minimalistic, but covers a lot of stuff. Webiny is still a Headless CMS. But it's a Headless CMS Plus all of the other features we've talked about.

Adding the word "Plus" to the business and enterprise products opens the discussion around Webiny's other capabilities. It allows us to showcase other features whilst not detracting from the core use that many of our users see value in.

It allows us to start more conversations around the functionality of the Page Builder and Form Builder tools, tools that are going to get some pretty interesting added features in coming months.

We're really hoping that this new initiative will allows us to open more discussions with organizations looking for, not just a Headless CMS, but something that can solve a lot of the issues around producing content in their business.

If this is you, please reach out to us! We'd love to hear about the difficulties you're facing. For our Enterprise customers we provide SLA support and consultancy services tailored to help your teams to build a solution that'll be designed around your workflow and specific needs. And we can build custom SSO implementations with your IDP of choice.

If you're an agency or a smaller business, Webiny is a great fit for you! Use it as a Headless CMS with multi-tenancy built-in, and pay a reasonable cost for each user seat. And add-on optional benefits such as our advanced publishing workflow and headless pages.

Remember, Webiny is an open-source, self-hosted solution: you can use it at no cost under our MIT license today. We hope this will help you to build smaller projects and thoroughly investigate it's capabilities before you make a decision."

Introspection: comparing myself to others, glorifying the past

Sat, 18 Sep 2021 00:00:00 GMT

This post is a little bit of introspection for me. I've recently been given a bit of time to think about certain things, and I'd like a make a note of them for my own personal growth.

If this isn't the kind of article you're looking for from me, feel free to skip this one.

Drawing personal comparisons #

I find that I often spend time thinking about who I compare to other developers, especially those on my team or within my organisation.

This helps me sometimes because once I've identified who I think is a better developer, I tend to watch their work more closely. This is because I'd like to pick up some of the things that make them so effective at their jobs.

However, I've also noticed that sometimes it leads to bumps in the road of my relationships with that person. I sometimes have a tendency to say things like "oh, you're a much better developer than me", which (a) puts them on a pedestal, and (b) sometimes annoys them.

It puts them on a pedestal because really what I'm doing is being a bit envious of that person. I'm never happy with myself, and sometimes value myself very poorly. But sometimes that means I value others too much.

It annoys my colleagues because once I've told them I think they're a better developer than me, they have to find some way of shifting the rather uncomfortable focus either back to me or onto something else. For some people that's a frustration they could do without.

How am I going to tackle this? I'm going to try to remind myself that a team is made up of a spectrum of skill sets and abilities. There are no super chickens, and everyone is of value and creates value for the organisation.

Glorifying the past (or not) #

I used to often imagine myself living in some era of the past. Especially in England during the early 1800s, when John Keats was walking the Lakes or composing his best poetry. I enjoyed thinking of the "simplicity" of the age, in terms of there not being nearly so much commercialism, lack of common good, and general trust between individuals than perhaps there can be in todays' world.

However, somewhere in the past 10 years I've stopped doing that.

I think I finally decided that although those times in the past might have had some advantages, there were no doubt many hardships and disadvantages that caused a huge amount of suffering. Generally much poorer and less available healthcare. Rampant racism and sexism. Even perhaps a sort of brutality that our current age doesn't demonstrate so readily.

But also, I stopped reading. I've simply had no time available to do so.

For both of these reasons, I stopped imagining myself in those past times. But the result of that is that I've felt more adrift.

Having that imagined world to think back on helped me to retreat away from some of the more uncomfortable realities I faced: disappointments, difficult and long commutes, the times I didn't enjoy my home life or work.

I wonder if I should bring back at least some of that imagination. I definitely want to bring reading for pleasure back into my life. It's going to be quite a challenge with my schedule though.

How am I going to tackle this? I am going to try to spend an hour a week reading, hopefully that'll give me a little more "scope for the imagination" back into my life.

Conclusion #"

Is Gatsby in decline?

Sat, 13 Aug 2022 00:00:00 GMT

Recently I encountered a Reddit thread along the lines of "is Gatsby dead?", and even some colleagues I respect were agreeing that yes, Gatsby is in decline, suggesting there are better technologies out there.

Whilst there are a good range of frameworks we can use these days, I don't necessarily agree that Gatsby is in decline and people should avoid using it.

I'd like to explain why.

In decline? #

I feel sure I was one of the earlier users of Gatsby. I was introduced to it and Netlify before v1 was released, and started experimenting with fetching data from a content source pretty soon after that.

I've since had a chance to work extensively with both Gatsby and Next.js, and can see why people like the Next.js lower-level API. I agree that for a lot of use cases, Next.js is a great tool to use. I particularly like how easy getServerSideProps() is to build an isomorphic application that runs in a container.

However, when I need to build an application, there's always going to be some assessment in my head about which tool is better.

Gatsby's strengths #

Where Gatsby's niche might be in the future is one of these, or possibly all of them:

1. Developers who want to leverage the plugin ecosystem #

There are a lot of developers who don't have the luxury of doing things in the best way possible themselves. Usually this is because of tight, perhaps unreasonable, deadlines. The plugin ecosystem doesn't always mean faster development, but it is easy to configure without having to learn APIs deeply first.

This might also benefit newer users to the JavaScript ecosystem, who need a way of learning React and friends.

2. Applications that source from many different APIs #

Where Gatsby does particularly well is with the "content mesh". You might have heard this referred to as "composable commerce" or something similar. But Gatsby had the concept very early on. The idea is that you source data from a variety of APIs which are then compiled into a single GraphQL API which you can then use to build up components, then pages, and built into a website.

Gatsby really excels at this. I can confidently fetch loads data from REST and GraphQL backends, query the rest in pages, components or templates, and chuck away the data I don't want when the application is built.

3. Applications that don't need specialist environments #

Gatsby has the gatsby-ssr.js and gatsby-browser.js files that both wrap the application, one that runs at build time and one on the client, as the names suggest). I think this is far superior separation of concerns than Next.js' _App.js.

It also leverages the open source react router, instead of including it's own. Next.js has admittedly made strides in this direction but until recently I had real issues when trying to render components on nested routes in a Next.js application.

Also, I find the Gatsby <Image /> element superior to Next.js, because it works statically, whereas you again need specialist infrastructure (or a containerized application) for Next.js' to work.

4. Where you need custom data in headers, footers etc #

Ever tried fetching data from an API at build time (or on a server) in a component with Next.js? You either need a context object or prop drilling, which isn't always possible or desireable.

Gatsby's staticQuery() on the other hand works in components and makes this really easy.

Conclusion #

Don't get me wrong, there's a huge amount going for Next.js, and I love building stuff with it. But I'm still not always going to reach for it.

If I need an application that requires React or a heavy amount of client-side JavaScript, I'll assess whether Gatsby is best or Next.js is.

All in all, I think there was an explosion in users when Gatsby first launched, and although there are other great tools like Next.js and Astro, I think it still has a valuable place in a JavaScript developers' toolbox."

Iterate.

Wed, 11 Oct 2023 00:00:00 GMT

Despite better marketing opportunities, the days of big bang releases are gone. In today's software market, iterative releases are the hallmark of good planning, well organised teams, and software stability.

A few times during the past year I've caught myself advocating for iterative changes. Both in software and in terms of our processes I've found that I far prefer smaller, incremental improvements over stressful, complicated and far-reaching changes that may or may not improve fulfilment, delivery and other metrics.

1. A software renovation project

The first time was during a renovation project. I had a code base that was built several years prior by someone who no longer worked at the company. Instead of scrapping the code or making huge rewrites, I decided that my first objective was to make the code more modular and easier to understand. I would also try to use what had, since the original person authored the code, become standard approaches.

So I replaced classes with functions, returned new objects instead of adding to existing ones, added a library of types, and isolated code within clearly defined, modular boundaries.

I knew the code needed some work. But if I were to take it further, I would be taking an extra risk: I didn't fully understand the implications of all of the paths the code could take. I didn't know the business impact of changing something that looked innocuous and yet could affect users.

Later, with these improvements in place, and a better understanding of what the business needed from the code, I could initiate a larger project to rebuild it into a more suitable shape.

There were also a number of other deliverables which the business needed in the interim. I was able to work on these until an opportunity came up to invest more time in improving what I had already standardised.

2. Implementing a team retrospective

I first saw the benefits of retrospectives a few years ago. When they were first implemented in my team, I saw straight away what an opportunity they created.

Quickly, we were able to voice our concerns about what processes had failed during our sprints. Other team members, including project managers, were able to pinpoint issues without having to focus on an individual team member, which led to less accusative and more collaborative environment. Everyone seemed to be happier.

So when I found at my new place of work we didn't have a retrospective, I was keen to get one started. I came up with a simple, straightforward board plan which consisted of electronic sticky notes for each team member, and 4 rows, one each for "were the action points for the last retro done?", "what worked well", "what went badly", "ideas" and "action points".

When someone puts a ticket in the "what went badly" row, I spend a bit more time to identify exactly what caused the issue and try to turn it into an action point.

In the approximately 10 months since we started, some team members have come up with really good ideas which we've implemented. We've eliminated a large number of things that were causing unwanted friction or frustration, and I hope it has also helped the team's wellbeing.

The JavaScript CMS Landscape

Mon, 23 Nov 2020 00:00:00 GMT

I've been keenly interested in a specific subset of CMSes since about 2 years ago, when it became clear that Zopa would soon need to invest in one, and that I would be involved in choosing something that would be a good fit for the company. This post is a roundup of some of the great products available, and is the result of some of the research and proofs-of-concept I have made.

The requirements I had for my search were:

Has to be self-hosted, open source, so we can secure our perimeter
Needs to be maintainable by an internal team of JavaScript developers
Needs to be headless (in order to render the frontend in an existing JavaScript application)

I had a separate search for a hosted platform-as-a-service which I executed in parallel, but my personal interest is more towards the open source community, hence the focus of this blog post.

1. Ghost #

Ghost seems like the grandaddy of JavaScript CMSes. Their version 3, which was launched about a year ago, was fully API-enabled, allowing it to be used as a headless CMS.

Ghost was built out of dissatisfaction with the WordPress technology stagnation if I remember correctly, but it's grown far past that to become a slick, comfortable and beautiful interface for building simple blogs.

Pros: Wonderful editing experienceCons: Not very extendable in terms of custom fields and content typesLink: https://ghost.org

2. Strapi #

I've watched Strapi go from an early Alpha state to a mature product with an international team, and that's been very rewarding to see. The UI is really great with a lot of integrations, and they're constantly working on new plugins and features that enrich your experience.

I was particularly pleased that they continue to improve on their Gatsby integrations, but there's one thing that I hope they're able to resolve in the short term: transforming data in their Gatsby examples is done on the frontend, when it can be done in the Node process.

Pros: Very adaptable and customizableCons: You only get 3 roles on the free tier (unless your OSS or a student), their GatsbyJS examples don't demonstrate how to transform data on the serverLink: https://strapi.io

3: Webiny #

Webiny is a lot more than a CMS, in fact, the CMS aspect is just one of the plugins for this incredible serverless framework. And honestly, Webiny does showcase some of the fancy things that become possible with serverless, and also levels out a lot of that road for newer developers.

If you're interested in serverless architecture, definitely give Webiny a look.

Pros: Serverless, so expect a lot of free hosting. A great way to learn the serverless architecture paradigmCons: Recommends use of gatsby-source-graphql plugin, which doesn't have access to nodes at build time. As a result, data transforming must be done on the frontend :-(Link: https://webiny.com

4: Keystone #

I haven't spent any time with Keystone, but Wes Bos hails it's solid role-based access and easy deployment features. I'll update this post when I've had a chance to play with it.

Pros: Role based access, easy deploymentCons: // TODO: try this app and update this blog postLink: https://www.keystonejs.com

5. Apostrophe #

Apostrophe only just fits onto this list because it's a full-stack CMS with a Headless plugin ... which is just fine, but it's an interesting choice. The presence of plugins like this one gives me confidence this is a mature app that has a solid future. Again, I haven't tried it out, but I will report back when I have.

The marketing talks about "in-context editing", but I'm not sure this refers to headless mode or not (edit: it's not) ... if it is, this could be a killer feature.

Pros: // TODO: try this app and update this blog postCons: // TODO: try this app and update this blog postLink: https://apostrophecms.com

6. Payload #

I heard about this newcomer on the scene only a few days ago, but the value proposition made sense to me. The UI is very minimal, on the basis that you'll want to customize it significantly. Their strongest feature is that there are hooks for every action, so you can extend the functionality very easily, and the initial codebase you see is extremely minimal.

Pros: // TODO: try this app and update this blog postCons: // TODO: try this app and update this blog postLink: https://payloadcms.com

Conclusion #

JavaScript, particularly on the server, is still a new language, but it's incredible that we have so much variety in tooling available to us already."

How We're using JavaScript with Drupal

Thu, 21 Aug 2025 00:00:00 GMT

For the past 3 years I've been working at a company that is heavily invested in Drupal as a platform. The company has several major sites and maintains its own Drupal extensions (plugins, or packages). In fact, it has around 40 of them.

As you might imagine there's a significant amount of JavaScript for some of these, since they manage our in-house player, run live events and provide other functionality such as user dashboards and integrated quizzes.

We've gone through a massive iteration with JavaScript and believe we've arrived at a solution that might help other teams that do similar.

Additionally, this might perhaps provide a basis for further discussion about an integrated bundler for Drupal, something I'm keen to see happen someday. But only if native import maps don't get there first.

History

Historically, JavaScript has been seen merely as a scripting language, meaning it didn't require types and tests, unlike PHP code which we were more keen to guarantee it did what it should be doing. This also meant there was little to no documentation for the JavaScript code.

This was getting more and more difficult to work with especially since some run into thousands of lines. Implementing new business requirements was really slow and often resulted in releases which contained broken code and had to be rolled back.

My task since then has been to strike a balance between releasing safely and still making it possible for our PHP-focussed developers to interact with the codebase.

Background

A Drupal extension helps us to encapsulate code for a specific purpose. For example if we create a new API, it will have its own extension. These extensions can tap into the existing functionality Drupal has.

For that reason, JS code often needs to be written in such a way that it interacts with these APIs.

This frequently means that the JS is called by Drupal, and often that takes the form of an immediately invoked function expression (IIFE), which looks like this:

((Drupal, once) => {
  Drupal.behaviors.myUniqueModuleId = {
    attach: (context, settings) => {
      const [myElement] = once("my-element-id", ".my-element-selector");
      const userName = settings.user.name;
      if(myElement) {
        myElement.innerText = `${userName}`;
      }
    }
  }
})(
  window.Drupal,
  window.once,
)

Drupal behaviours

Drupal behaviors are a part of Drupal's JavaScript API that allow developers to attach functions to be executed at specific times during a web page's lifecycle.

This blog article has a pretty straightforward explanation of why we need to hook into those.

In brief, behaviours are for hooking into the context. For example, Drupal might update the DOM with new elements, so we can run the behaviour in conjunction with those changes, in effect, _attach_ing it to the backend changes.

Once

Once is a jQuery API that has been adopted in Drupal to ensure something runs only ... well, once. It's needed because Drupal might call the code in behaviours multiple times as the page is being set up and executed, and as I mentioned before, DOM elements can be changed by Drupal after the first load. For this reason, window.addEventListener('DOMContentLoaded', () => {}) might not be sufficient.

Using once() no longer means we're using jQuery, it's been forked as a separate, standalone module by the Drupal community.

Sometimes this isn't available to us though, for example our media player is a standalone Custom Element written with JavaScript. To ensure that the DOM element has loaded and been registered in the DOM in that case, we have a custom waitForElement() JavaScript function.

Approach

When it comes to our JavaScript projects, what we have now is two parallel approaches which can be adopted when a new extension is being conceived, or later on when the need arises. If the recommendations below are adhered to the resulting code changes between the two are relatively small.

1. Minimal approach

If only minimal amounts of JS are expected, we can skip the more complex setup mentioned below and add the JS to a /js folder and enqueue it using module.libraries.yml. We ensure it's always defined as an ES Module so that if we need to switch to bundled code we can do so with minimal refactors:

myjs:
  js:
    js/myjs: { attributes: { type: module } }

As I emphasise this is for really small things like setting some text on a DOM element as in the example above. The minimal approach does not support JavaScript unit tests. However because we use JSDoc instead of TypeScript, we can still implement type safety.

2. With bundling

We've avoided tools like Foxy, which allows us to bundle with Vite on the site level and integrates with Composer. Whilst it would be appealing from an organisational perspective to be able to have one JS bundle, it could be wasteful as we'd be unable to customise that for the user. It would also likely be very large on the initial load which could delay interactivity in some cases.

We do use some dependencies from Asset Packagist. These are typically large dependencies such as JSPDF and Chart.js which we use in more than one Drupal extension.

When dependencies are loaded in via Asset Packagist, they are loaded in on the global object, and are accessed directly from that:


const { Chart } = window

const chart = new Chart()

Care must be maintained not to overwrite globals. Also upgrading requires extensive testing since it might be used in several modules, which means that we could easily break something.

We find Asset Packagist to be problematic because packages are often not available until after they have been requested for the first time. This has repeatedly led to failed builds.

It's also a mirror of NPM which has more opaque security implications.

Therefore, When adding external dependencies from NPM we typically switch to using a bundler for that extension.

The downside of this is that we might have several versions of a specific dependency on one page, or across the whole site. But since they are encapsulated in the module they are not globally available so clashes aren't a possibility.

It also means that the JavaScript bundles could be larger as a result which might have been avoided with global dependencies. However the upgrade path is much easier since dependencies are easily identifiable.

If an external dependency is needed, or the complexity of the JS rises above a certain level (we haven't specified where we should switch, instead leave it up to the good judgement of each developer), we have adopted the following setup:

1. Introduction of /src and /dist folders

When someone decides to add this step, the folder structure changes to accommodate the bundling process. This also means that the libraries YML changes to

myjs:
  js:
    js/dist/myjs: { attributes: { type: module } }

Still very similar to above.

2. Package JSON and Vite config

Next, we have a package.json with a "type": "module" declaration and some standarsised libraries such as Vite, our internal JavaScript package, and a dependency of @rollup/plugin-replace, we'll see why in a minute.

We also have settled on the following standardised scripts:

"scripts" : {
    "lint:check": "eslint -c eslint.config.js '**/*.js'",
    "lint:fix": "eslint -c eslint.config.js --fix '**/*.js'",
    "format:check": "prettier --check '**/*.js'",
    "format:fix": "prettier --write '**/*.js'",
    "test:ci": "vitest --run",
    "test:watch": "vitest --coverage",
    "build": "vite build",
    "dev": "vite build --watch"
}

With these scripts a PHP developer can successfully work with JS by running npm run dev, which will re-compile the JS code at every change. They an also format and check linting easily.

Standardising these becomes really handy when there are sub-modules. They also help us use Git Hooks to vaildate the code before we push to our source repository.

We also have a Vite setup as follows:

/// <reference types="vite/client" />
import { defineConfig } from "vite";
import { resolve } from "path";
import fg from "fast-glob";
import replace from "@rollup/plugin-replace";

// Generate entry points dynamically
const inputFiles = fg.sync("src/**/*.js").reduce((entries, file) => {
  const name = file.replace(/^src\//, "").replace(/\.js$/, "");
  entries[name] = resolve(__dirname, file);
  return entries;
}, {});

// https://vitejs.dev/config/
export default defineConfig({
  build: {
    sourcemap: true,
    outDir: resolve(__dirname, "dist"),
    rollupOptions: {
      input: inputFiles,
      output: {
        entryFileNames: "[name].js",
        chunkFileNames: "[name].js",
        assetFileNames: "[name].[ext]",
      },
      plugins: [
        replace({
          "process.env.NODE_ENV": JSON.stringify("production"),
          preventAssignment: true,
        }),
      ],
    },
  },
  test: {
    include: ["tests/**/*.test.js"],
    environment: "jsdom",
  },
});

As you might be able to follow, although the JS files are bundled they're not combined into one JavaScript file. This is because we still want our JS to be managed by Drupal. In some cases, specific JS won't be loaded onto a page. In others as I said above, the JS won't be relevant until certain Drupal behaviours have been run first.

Other than that, we're using the @rollup/plugin-replace to provide an environment variable of NODE_ENV set to production. Vitest by default injects a value of test, which comes in handy for testing as you'll see in a minute.

Test approach

You'll notice in the Vitest set up above we have our test setup configuration as part of vitest.config.js, which saves an additional file.

Test are located in src/test folder exclusively to keep them out of the way of the production code.

We also need to stop the IIFE from executing as soon as it's imported for the test suite, and before any test setups can be performed. This can be handled by wrapping it in this if statement:

if(!process.env.NODE_ENV !== "test") {
  ((Drupal, once) => {
    // initial setup goes here, for example timers
    Drupal.behaviors.myUniqueModuleId = {
      attach: (context, settings) => {
        const [myElement] = once("my-element-id", ".my-element-selector");
        const userName = settings.user.name;
        if(myElement) {
          myElement.innerText = `${userName}`;
        }
      }
    }
  })(
    window.Drupal,
    window.once,
  )
}

But ideally the code is in a separate JS module from the IIFE, since that is much neater than this solution.

Building the JS for Production

We're still in quit a bit of a quandry about how to build JS that uses the bundler approach. A few options we're considering:

Build locally: since we are building the /dist folder in development anyway, we can push the build code into the source repository. This is hazardous because there might be changes depending on the platform. Also if someone forgets to run build before they commit, we could have stale code in production and an unexpected outcome the next time soneone makes a code change
Build in the pipeline: This ensures that the module contains the latest code in the /dist folder. But it doesn't take into account the production environment. It also allows us to build the JS modules upfront, instead of having to do them all at once which could be time consuming
Build it when releasing: Our hosting setup allows us to hook into the build before it's pushed into production. We could build the JS in the target environment at that point. However, we won't be able to automate this since we don't know which extensions need bundling.

We currently use both (1) and (2). Implementing (3) has proved to be problematic and time consuming.

How we handle sub modules

Some Drupal extensions have modules that further extend the functionality over the base implementation. They help with encapsulating other code and allow it to be called only when specifically needed.

So how do we support JavaScript that appears inside these modules?

Each of these sub-modules is treated as a separate JS bundle, so has it's own Vite config, package.json and all of the set up mentioned above. We have experimented with inheriting some of this config but it's proved to be more confusing. Duplicating some files is a small price to pay for simplicity.

The addition of sub-modules could make things awkward when modifying code. To negate this we've adopted an approach that hopefully speeds things up. In the root of the PHP extension we have an additional package.json file that is very minimal. It only has one dependency and includes the following scripts:

  "scripts": {
    "install": "npm-run-all --parallel main:install sub_module:install",
    "main:install": "cd js && npm install",
    "sub_module:install": "cd modules/sub_module/js && npm install",
    
    "dev": "npm-run-all --parallel main:dev sub_module:dev ",
    "main:dev": "cd js && npm run dev",
    "sub_module:dev": "cd modules/sub_module/js && npm run dev",
}

With the addition of this file, developers can run npm run install from the project root and install all JavaScript dependencies for each of the sub modules. This makes development much faster and stil allows for a developer to change directory to a specific sub module if they are only dealing with one.

Conclusion

I hope this has been a useful walkthrough of how we're currently using JavaScript in our Drupal sites. If you have any further questions, spot some errors or want to discuss it further, please feel free to respond using the comments section below.

I launched a YouTube channel

Mon, 30 May 2022 00:00:00 GMT

Time flies, as they say, and as I'm getting older (reached 42 this year) I find I have as much to say but less time to say it. I'm thinking people who are interested in my content are likely to be in the same position.

With that in mind, I opened my own YouTube channel.

Here's some of the things I plan to talk about:

I opened the channel mostly because I wanted to teach my kids and others how to code, for which I've created a playlist.

I like to review and occasionally rant about) open source CMS systems, which I've already created one video about

I'm also mentoring a group of young adults,and although as of today there's only one video in the playlist, I'll be adding more videos as we bump into gaps in their knowledge that I think might be advantageous to share with the wider community."

Learning serverless with Webiny

Fri, 03 Apr 2020 00:00:00 GMT

Webiny is a new CMS in the market, one that seeks to compete with other well-established headless CMS platforms and existing apps. But I've also come to enjoy using it for another reason: it's helping me learn how to apply principles of Serverless application architecture.

I've spent a great deal of time over the past 2 years around the CMS space. I am particularly focused in self-hosted, javascript-based CMSes. I had already created sites with both Ghost and Strapi, two very competent systems with content APIs.

Very recently, I came across Webiny, a hosted platform which recently pivoted to become a self-hosted product, and which has the interesting selling proposition of being a "serverless" CMS.

Why I think serverless matters #

There's a lot of buzz around the word "serverless" in my world recently, and for good reasons. Although some are keen to point out that the term doesn't technically mean you're not using servers, it does have the strong advantage that you only pay for what you use, and if you don't exceed often generous free initial offerings, you don't pay for the product ... at all.

This is how Heroku got so popular. It can afford to give you the space to create an app, knowing that past a certain point it can be destroyed, saving on computing expenses. Your app can then be spun up at some point in the future when called upon at the cost of a minute of two of time.

Companies today spend a huge amount of money running servers constantly, even when there's nobody using their services. Imagine if you didn't have to pay for that downtime?

That's why I think serverless is going to become an increasingly large player in the devops space in the next few years.

Webiny: truly "serverless"? #

Unless your CMS is git-based tool, such as NetlifyCMS, Tina or Forestry, your content needs to be stored on a database somewhere. And therein lies a weakness of any CMS: it depends on writes to one single database, which you could argue doesn't scale hugely well.

Webiny is no exception: it has connectors for different databases, and I've used Mongo's hosted service "Atlas" for mine. However does that mean it doesn't truly fit into the "serverless" paradigm?

I would argue that it does. And this reveals some of the underlying arguments around the semantics of what "serverless" means. To some people, the term only refers to lambda functions, and not to other things like the authentication service, or the file storage system.

However, for me, "serverless" means a disparate collection of interrelated services, tied together by common use. By this definition, the file storage system, the authentication service, the database, and everything else, constitutes a serverless application ... i mean, tool ... I mean, whatever.

How Webiny helped me learn about serverless #

Although you can create serverless applications using the online interfaces given to you by different providers, it's real strength is in allowing you to programmatically create your services as you go.

This is incredibly powerful. Here's my Webiny application code:

name: webiny-apps-xxxxxxx

vars:
  region: ${env.AWS_REGION}

site:
  component: "@webiny/serverless-app"
  inputs:
    description: Webiny Site
    region: ${vars.region}
    memory: 128
    timeout: 30
    code: ./site/build
    env:
      SSR_FUNCTION: ${ssr.name}

ssr:
  component: "@webiny/serverless-function"
  inputs:
    description: Site SSR
    region: ${vars.region}
    hook: yarn build:${cli.env}
    root: ./site
    code: ./site/build-ssr
    handler: handler.handler
    memory: 2048
    timeout: 30

admin:
  component: "@webiny/serverless-app"
  inputs:
    region: ${vars.region}
    description: Webiny Admin
    hook: yarn build:${cli.env}
    root: ./admin

api:
  component: "@webiny/serverless-api-gateway"
  inputs:
    name: Apps Gateway
    binaryMediaTypes: ["*/*"]
    description: Serverless React Apps
    endpoints:
      - path: /admin/{key+}
        method: GET
        function: ${admin}
      - path: /admin
        method: GET
        function: ${admin}
      - path: /{key+}
        method: GET
        function: ${site}
      - path: /
        method: GET
        function: ${site}

cdn:
  component: "@webiny/serverless-aws-cloudfront"
  inputs:
    origins:
      - url: ${api.url}

I'm not going to break down everything, but you might be able to recognise different services for "site", "ssr", "admin", "api" and "cdn", etc, which are variously the API gateway, the admin interface, the frontend static site, and some lambda functions.

They all tie together to make the backend interface work, and to compile a static site hosted on S3.

And if I log into my AWS dashboard, I can see these services there too ... I mention that just because I have a visual kind of brain.

This idea, of "infrastructure as code" means your applications are truly portable: you can destroy it and re-create it from its blueprint using the code you've written.

And with the amount I use my Webiny CMS, I'm probably not going to ever pay a thing for it.

Try it out! #

I highly recommend giving Webiny a spin. The product is in early stages but is already quite promising. It's nice that as JavaScript developers, we have a good range of choice between this, the rising star Strapi, and the very mature Ghost.

What do you think of it? Let me know!"

Let Browsers be Browsers

Mon, 23 Nov 2015 00:00:00 GMT

I've been hearing a lot of different, seemingly contradictory messages from the industry in recent weeks regarding frontend frameworks and progressive enhancement. I got so frustrated that I started a discussion on Web Freelancers Hub about it. I felt the answers that came back were worth documenting.

My original post to the group:

So, I'm really confused.

There's been a huge demand for people who can work with Frontend frameworks especially Angular.js in recent months. For a while, it seemed like I was turning down 5 contracts a week that required this skill.. which I'm now learning.

But now the industry seems to be eating is tail & saying this is bad practice because what if users don't have Javascript enabled on their browser (for whatever reason)... and I agree with them.

I'm a bit confused / frustrated with these mixed messages — I really agree with recent posts from Jeremy Keith and Aaron Gustaffson but don't know the best route to take to

a) stay ahead of the game

b) provide the best experience for users.

Anyone else thinking this / have a take on it?

Although there were a few developers who argued that when developing apps, you have to stipulate to users that JS must be enabled, Nick Bramwell was quick to comment in a way that resonated with me: "I’ve been burnt too many times in the past learning things that quickly vanish."

This is one thing that I've been afraid of, since it takes a lot of time to learn frameworks ... and I have a young baby to take care of!

Rachel Andrew mentioned a use case that I don't think we consider enough: "I travel a lot, end up on terrible wifi. At least once per trip I find myself unable to use a site because bits of their JavaScript haven't made it." This is also what motivated Scott Jehl to write his book "Responsible Responsive Design".

Browsers Are Browsers #

So I guess my position is that there may be certain cases we would justifiably use fronted frameworks to enhance the experience for the user, we shouldn't make it a requirement of our site. Browsers are meant to be used to view and interact with data, and largely cannot be 'exploited' — partly because users need protection from such things.

I guess that if responsive design has taught me one thing, it's that we can't know the conditions under which our site will be served up to someone, and that we shouldn't try to know, or to stipulate conditions unless there is a very good reason for doing so.

I'm grateful to be part of a community that I can go to when issues like this come up. For now, I'm going to stick with what I've learned with regard to fronted frameworks. Perhaps I might start learning server-side (universal, or isomorphic) JavaScript, because I seem to enjoy seeing what's possible with the language."

Let's build a carousel!

Sat, 13 May 2023 00:00:00 GMT

Carousels are a staple of web development life. And thankfully, there are a lot of good off-the-shelf ones ready to build into your project. But it's a good challenge to see if you can build one yourself.

This is somewhat of a personal milestone for me: I remember when I was a new software engineer tinkering with different solutions thinking that I probably would never be able to build one from scratch. But, here it is, I did it in a couple of days.

This helped me to see that I'm still making progress as an engineer and learning new things.

But maybe you just want to build a carousel ... if so, we first need to carefully consider whether a carousel is the best option for what we need to display. Carousels are very often not the solution you want: please check out this site for some statistics. If you have decided a carousel is best for your UI, then read on.

HTML setup

I'll start with explaining the HTML layout here. I'm using Tailwind because that is what I was using at the time, but you can use whatever CSS you like.

First, we have an opening tag:

<div id="animation-carousel" class="tw-relative tw-w-full tw-h-36 md:tw-h-96 tw-my-8" data-carousel>

A few things of note: We're going to use a lot of relative and absolute positioning here. To make sure that is scoped to this container, we're making sure the position is relative. And because each item is going to be positioned in this way, we're setting a height too.

Lastly, we're setting a data attribute. This is what we're going to pick up in the JavaScript. I could just as easily have used a class name.

Next, I've got the inner wrapper. This will contain the images, previous and next buttons, and indicators (the little navigation pips you sometimes see).

  <div class="tw-relative tw-overflow-hidden tw-rounded-lg tw-h-36 md:tw-h-96">

Indicators

Indicators are optional in this setup. If they're not in the DOM, we won't render them. To facilitate that, I've used a template tag so they don't render initially, rather, we can clone this template's inner HTML as many times as we need to render the pips:

      <div id="indicator-container" class="tw-absolute tw-z-30 tw-flex tw-space-x-3 tw--translate-x-1/2 tw-bottom-5 tw-left-1/2">
        <template id="carousel-indicator">
          <button type="button" class="tw-w-3 tw-h-3 tw-rounded-full tw-border tw-border-white" aria-current="false" aria-label="Slide 1" data-carousel-slide-to="0">
            <svg viewBox="0 0 100 100" class="tw-text-white" xmlns="http://www.w3.org/2000/svg">
              <circle cx="50" cy="50" r="50" fill="currentColor" stroke="white" stroke-width="3"/>
            </svg>
          </button>
        </template>
      </div>

Template tags aren't ordinarily rendered by the DOM tree but we can still select them with JavaScript. This is a handy feature as you'll see later. We've got a button with an SVG circle in there so we can click to select a certain slide.

Each of these has an aria label to indicate which item is focused on. I could have used a visually hidden text label.

Next come the items themselves.

Slides

The slides have an opacity of zero to start with - except the item we wish to set as initially active. We do this so that if there's a problem with the JavaScript loading, the user will still see one image and not just an empty part of the screen:

      <div class="tw-opacity-0 tw-transition tw-duration-150 tw-ease-in-out tw-h-36 md:tw-h-96" data-carousel-item="active">
          <img src="https://placehold.co/600x400/orange/blue" alt="..."  class="tw-absolute tw-block tw-w-full tw--translate-x-1/2 tw--translate-y-1/2 tw-top-1/2 tw-left-1/2 tw-h-36 md:tw-h-96">
          <div class="tw-absolute tw-inset-0 tw-flex">
            <blockquote class="tw-m-auto tw-z-40 tw-font-bold tw-text-4xl tw-max-w-xl tw-text-center tw-text-zinc-50">
                The
                <span class="tw-text-primary-400">more I learn</span>, the more I realise how much I don't know
                <p class="tw-mt-6 tw-text-sm tw-font-normal tw-italic">
                  Albert Einstein
                </p>
            </blockquote>
          </div>
      </div>
      <div class="tw-opacity-100 tw-transition tw-duration-150 tw-ease-in-out tw-h-36 md:tw-h-96" data-carousel-item="active">
          <img src="https://placehold.co/600x400/orange/white" alt="..." class="tw-absolute tw-block tw-w-full tw--translate-x-1/2 tw--translate-y-1/2 tw-top-1/2 tw-left-1/2 tw-h-36 md:tw-h-96">
      </div>
      <div class="tw-opacity-0 tw-transition tw-duration-150 tw-ease-in-out tw-h-36 md:tw-h-96" data-carousel-item>
          <img src="https://placehold.co/600x400/red/white" alt="..." class="tw-absolute tw-block tw-w-full tw--translate-x-1/2 tw--translate-y-1/2 tw-top-1/2 tw-left-1/2 tw-h-36 md:tw-h-96">
      </div>
  </div>

As you'll notice, as well as images, each slide can contain a text caption too. I could've used a figure and figcaption element pair for this, but in my case, the image is purely decorative so I opted for a blockquote for the quotation.

We're going to be using the data-carousel-item attribute again in the JavaScript, and attach a value of active to the current slide.

Previous & Next Controls

These controls render to the left and right of the slider and allow users to cycle through the images in order or reverse order. They have icons indicating the direction they're cycling in when they're clicked and a visually hidden label to aid assistive technology users.

  <button type="button" class="tw-absolute tw-top-0 tw-left-0 tw-z-30 tw-flex tw-items-center tw-justify-center tw-h-full tw-px-4 tw-cursor-pointer tw-group focus:tw-outline-none" data-carousel-prev>
      <span class="tw-inline-flex tw-items-center tw-justify-center tw-w-8 tw-h-8 tw-rounded-full sm:tw-w-10 sm:tw-h-10 tw-bg-white/30 tw-dark:bg-gray-800/30 group-hover:tw-bg-white/50 dark:group-hover:tw-bg-gray-800/60 group-focus:tw-ring-4 group-focus:tw-ring-white dark:group-focus:tw-ring-gray-800/70 group-focus:tw-outline-none tw-transition-all tw-ease-in-out">
          <svg aria-hidden="true" class="tw-w-5 tw-h-5 tw-text-zinc-400 hover:tw-text-zinc-800 sm:tw-w-6 sm:tw-h-6 tw-dark:text-gray-800 tw-transition-all tw-ease-in-out" fill="none" stroke="currentColor" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M15 19l-7-7 7-7"></path></svg>
          <span class="tw-sr-only">Previous</span>
      </span>
  </button>
  <button type="button" class="tw-absolute tw-top-0 tw-right-0 tw-z-30 tw-flex tw-items-center tw-justify-center tw-h-full tw-px-4 tw-cursor-pointer tw-group focus:tw-outline-none" data-carousel-next>
      <span class="tw-inline-flex tw-items-center tw-justify-center tw-w-8 tw-h-8 tw-rounded-full sm:tw-w-10 sm:tw-h-10 bg-white/30 dark:tw-bg-gray-800/30 group-hover:tw-bg-white/50 dark:tw-group-hover:tw-bg-gray-800/60 group-focus:tw-ring-4 group-focus:tw-ring-white dark:tw-group-focus:tw-ring-gray-800/70 group-focus:tw-outline-none tw-transition-all tw-ease-in-out">
          <svg aria-hidden="true" class="tw-w-5 tw-h-5 tw-text-zinc-400 hover:tw-text-zinc-800 sm:tw-w-6 sm:tw-h-6 dark:tw-text-gray-800 tw-transition-all tw-ease-in-out" fill="none" stroke="currentColor" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M9 5l7 7-7 7"></path></svg>
          <span class="tw-sr-only">Next</span>
      </span>
  </button>

That's the HTML, now the fun part.

JavaScript

Firstly, we have to ensure that the DOM has fully loaded before trying to select our elements, so we wrap the function like this:

window.addEventListener("DOMContentLoaded", () => {
  // our code goes here
});

More about this here.

Next, let's define our createCarousel function, and just return early in case of some easy to detect errors, for example, if there isn't a window object, or the selector (the HTML element we want to create a carousel with) hasn't been passed to the function:

function createCarousel(selector) {
  if (typeof window === 'undefined') {
    return;
  }
  if(!selector) {
    console.error("cannot find element to create a carousel into");
    return;
  }

Now let's pick up some HTML elements relative to the selector that we're going to need shortly:

  const carouselItems = selector.querySelectorAll('[data-carousel-item]');
  const carouselItemsArray = Array.from(carouselItems);
  const indicatorTemplate = selector.querySelector('#carousel-indicator');
  // define interval timer so we can clear it later
  let intervalInstance = null;

In this bit of code we're getting the slides themselves. But using querySelector() returns a NodeList collection instead of an Array, so I'm making it into an Array that'll be handy later on. We're also getting the indicator template, and also setting an Interval, which is a timer we use to know how long we should wait before scrolling to the next slide. I want to define it here but set it as null so we can assign it and reference it in other functions.

Helper functions

Now for some helper functions:

function getActiveItem() {
    return selector.querySelector('[data-carousel-item="active"]');
  }
  function getPositionOfItem(item) {
    return carouselItemsArray.findIndex((carouselItem) => {
        return carouselItem === item;
    });
  }
  function setItemAsActive(item) {
    item.setAttribute('data-carousel-item', 'active');
    item.classList.remove('tw-opacity-0');
    item.classList.add('tw-opacity-100');

    // update the indicators if available
    const currentItemIndex = getPositionOfItem(item);
    const indicators = selector.querySelectorAll('[data-carousel-indicator]');
    indicators.length > 0 && Array.from(indicators).map((indicator, index) => {
      if (index === currentItemIndex) {
        indicator.setAttribute('aria-current', 'true');
        indicator.querySelector('svg').classList.add('tw-text-primary-600');
        indicator.querySelector('svg').classList.remove('tw-text-white');
      } else {
        indicator.querySelector('svg').classList.add('tw-text-white');
        indicator.setAttribute('aria-current', 'false');
        indicator.querySelector('svg').classList.remove('tw-text-primary-600');
        indicator.querySelector('svg').classList.add('tw-text-white');
      }
    });
  }
  function setItemAsInactive(item) {
    item.setAttribute('data-carousel-item', '');
    item.classList.add('tw-opacity-0');
    item.classList.remove('tw-opacity-100');
  }

1. getActiveItem() (no parameters)

At several points, we're going to need to get the currently active item. And since that changes frequently, I don't want to store it in a variable in the main function's scope since that will mean it will only be called once and hold the same value. Instead I have made a small function for this so I could more easily determine what the intent of this selector is.

2. getPositionOfItem(item)

As it says, this one finds the position of a slide as an index of the array we created earlier.

3. setItemAsActive(item)

This one is a bit more complicated but it's name should inform you of what it does. As well as making an item active, it also updates the indicators (if they exist) to identify the index of the slide.

This was tricky because you have to match the index of a slide with the correct index of the indicators, and you have to remove the active classes from other indicators too.

4. setItemAsInactive(item)

I could possibly have done this in the function above, but that would combine two intents into one function, which would make it harder to understand. It would also mean adding an extra parameter which complicates things somewhat. This way, both functions that set items active and inactive and, if something else needs to happen when slides change, they're easily identifiable and manageable.

That's it for the helpers, now I want to set up some actions that the carousel will perform:

Actions

Actions define either responses to user interactions or things that happen at a set interval

function cycle() {
    intervalInstance = window.setInterval(() => {
        next();
    }, 3_000);
  }
  function pause() {
      clearInterval(intervalInstance);
  }
  function slideTo(nextItem) {
    const activeItem = getActiveItem();
    setItemAsInactive(activeItem);
    setItemAsActive(nextItem);
    pause();
    cycle();
  }
  function next() {
    let nextItem = null;
    const activeItem = getActiveItem();
    const activeItemPosition = getPositionOfItem(activeItem);
    if (activeItemPosition === carouselItems.length - 1) {
        // if it is the last item, set first item as next
        nextItem = carouselItems[0];
    } else {
        nextItem = carouselItems[activeItemPosition + 1];
    }
    slideTo(nextItem);
  }
  function prev() {
    let prevItem = null;
    const activeItem = getActiveItem();
    const activeItemPosition = getPositionOfItem(activeItem);

    if (activeItemPosition === 0) {
        prevItem = carouselItems[carouselItems.length -1];
    } else {
        prevItem = carouselItems[activeItemPosition - 1];
    }
    slideTo(prevItem);
  }

1. cycle() (no parameters)

Cycle sets up the interval that we defined at the start of the createCarousel function to 3 seconds and then shows the next carousel item.

2. pause()

Pause clears the interval so when the next slide shows we start again from 3 seconds instead of whatever was remaining in the timer. I named it pause because I thought I might allow external access to the function so that the slider can be paused. In retrospect, it doesn't do that yet so I should have changed the name to reflect what it does.

3. slideTo(nextItem)

Because this function takes the next item we want to set as active as a parameter, we can call this function any time the user clicks on a prev or next button, or the indicators. It uses the two helper functions to change the active item, resets the interval and then cycles the carousel.

4. next() (no parameters)

There's a little trick to this one which means we have to find out if we're on the last item, and if so, the next slide to show should be the first item. See how converting the NodeList into an Array came in handy?

5. prev() (no parameters)

This is functionally very similar to the next() function except it's only called by someone hitting the "previous" button.

Initialize carousel

I could have put this segment of code into the main closure of the createCarousel() function, but I think having it as a separate body makes it easier to see the intent of what we're doing and it's also clearer to read.

function init() {
    const activeItem = getActiveItem();

    const items = Array.from(carouselItems)
    items.map(item => {
      item.classList.add(
        'tw-absolute',
        'tw-inset-0',
        'tw-transition-transform',
        'tw-transform'
      )
    });
    /**
     * if no active item is set then first position is default
    */
    if(activeItem) {
      slideTo(activeItem);
    } else {
      slideTo(0)
    }
    /**
     * Add event listeners to the buttons if they exist
     */
    const nextButton = selector.querySelector('[data-carousel-next]');
    nextButton && nextButton.addEventListener('click', () => {
      next();
    });
    const prevButton = selector.querySelector('[data-carousel-prev]');
    prevButton && prevButton.addEventListener('click', () => {
      prev();
    });
  }

Here we're leveraging the small helper and action functions we defined above to set some classes on each of the items, making sure the user sees the active item. This also covers the case that the user assigns a slide other than the first slide as the one that should initially be active. It then adds event listeners to the next and previous buttons if they exist in the DOM.

Lastly in the main body of the createCarousel() function we call these functions:

  init();
  // if we have an indicator template, create the indicators
  indicatorTemplate && createIndicators();

Like I said, this last step could've been avoided if I'd put the body of init() and createIndicators() functions in the main function, I merely thought this was better from an organisational point of view.

Create all of the Carousels

Here's the final step, which we'll do outside of our createCarousel() function but still inside the DOMContentLoaded listener:

const allCarousels = document.querySelectorAll('[data-carousel]')
allCarousels.forEach((carouselElement) => {
    createCarousel(carouselElement);
});

Which is to find all of the DOM elements with data-carousel attributes and create a carousel for them. Now each carousel will have it's own indicators, event listeners and intervals which run independently of each other.

And also, if there are no elements found, the createCarousel function won't be called at all.

Further Improvements

Now that the basic carousel functionality is complete, what could I do to further improve the code?

I've seen a lot that I can do.

For example, there's no API. I would like to allow access to some things such as the interval timer so that the speed with which carousel items cycle can be defined on a per-use basis. I'd also like to allow access to other things such as the prev and next buttons. Perhaps analytics need to be added to these listeners in some cases. Or perhaps someone will need to pause the carousel programatically for some reason.

Conclusion

Creating this carousel wasn't just a technical challenge. It was also a communciation challenge: how do I build this in a way that is easy to understand for those needing to modify my code? Does it explain itself easily? Does it allow another engineer to focus on their task and not ask too many mental gymnastics of them?

This is what we must aim for when we're writing modules like this; we must be first and foremost kind to both other engineers who will come along later, and our future selves.

Full code here."

Lowest tech first

Wed, 23 Jun 2021 00:00:00 GMT

Should I use CSS or JavaScript to perform this task? If you're asking yourself this question, I have an answer for you: use the lowest technology available to you which can complete the task.

I did a lot of prop threading the other day. I was updating our navigation section so that the last dropdown in the navigation would be positioned better.

<NavList isLastItem={index.length - 1 ? true : false} />

My colleague pointed out that I was threading this through 3 successive components so I could apply some CSS to the menu, which is OK sometimes, but there could be a better way of doing it.

&:last-of-type {  left: 0;}

Why is this better?

Because it's using the lowest tech first principle. I'm setting the property in CSS which means there's less overhead for the browser to maintain, less prop threading going on, and it's arguably easier to see what's going on in the styles of this component."

Losing Peter

Thu, 14 Sep 2017 00:00:00 GMT

I don't recall how Peter and I became friends. We must've struck up some conversation at school, is the only thing I can think of, and quickly found out we had similar interests. Peter is the guy who I used to call up on a Tuesday evening after the latest episode of Babylon 5 aired, and talk abut the episode, speculate about further plot details, and share our amazement of its then cutting edge FX.

Peter was adopted. He never talked about this situation apart from on one occasion where we were pranking each other that we were both secretly aliens. Of course, being adopted was great backstory material.

As time went on, Peter and I got more intensely interested in writing. We used to meet and flush out story outlines for hours, discussing main plot points, charachters, cultures, technology (most of them were science fiction stories, of course) and story arcs. Peter had a natural talent and I enjoyed listening to him flesh out an idea from nothing.

Quite often our imaginations would be fired by some news of this new technology called The Internet. I had been given a 14.4k Apple modem and would excitedly chime "Let's try to get on the internet!". Hours passed, and we would still be waiting for a beta of NCSA Mosaic or later Netscape Navigator to download, and feel frustrated when all we got was a blue box where a gif header image should be.

14.4k was becoming slow even by standards of those days.

Drifting Apart #

Peter and I didn't stay friends for long after school. We went to different colleges (and were in different year groups - he was a year older than me). We did meet up a few times. He'd gotten into the madcap humour of Monty Python's Flying Circus. This kind of humour was beyond me and I didn't understand why he was rolling around, creased double on the floor at the absurdity of it until much later on in life.

Final Scenes #

I don't know how we came to meet the last time. Perhaps we saw one another in the stret whilst Peter was at Cardiff Uni, and I was working in a recruitment office nearby.

By that point we had not seen each other in a year or two. Peter seemed distracted. There was no absurd laughter, no jollility, and scarcely a mention of the things we used to enjoy.

Looking back, I have to wonder if this wasn't a silent plea for help. He clearly wasn't himself. I think now, perhaps that wasn't due to his occupation with his studies, his involvement with some relationship or whatever else.

Instead, was it an absence of anything that he might have otherwise been occupied with. I know from personal forays into feelings of depression, it's the absence of anything rather than a struggle to find common ground that results in these unhealthy silences.

"As you know, we don't have Peter now", his adoptive mother said to my mother when she met her shopping. Again, a shielding, or an absence of emotion, similar to Peters, I thought.

Final Thoughts #

Peter's gone. But I wonder regularly, if not daily, if I could have done anything to change his mind. I didn't speak to him about my faith after he expressed contentment with his Catholic heritage. Would that have made any difference? I didn't breach many personal subjects with him. I didn't even keep in touch. Was that neglectful of me?

I wsh I could talk to him again, if only just to ask him, "Why did you choose that path?", to try to understand what it was that made him take his life.

My father thinks that individuals tend to be more compassionate after a bereavement. That the harrowing experience of losing someone makes you see our temporary situation for what it is, and are more appreciative of friendships and relationships we have because of that."

Managing Quality across Different Codebases

Tue, 05 Dec 2023 00:00:00 GMT

One of my roles as the sole JavaScript Engineer is that of planning for the future. I hope that the company will grow to the point where other JavaScript engineers will be able to join. But even if it doesn't, I have outlined a definitive plan to continuously improve the code here.

Having a structured plan for the codebases in your project will help extend their lifecycles. It will also help those using the code to more easily manipulate them to fulfil new business requirements whatever these might be.

To that end, I wrote this document as a guideline for my company, feel free to adopt, adapt it or use it as a basis your own.

Index

Lists the sections in this document, hyperlinking them for easy navigation.

Code quality

Type Safety

Unit Tests

End to end Testing

Deployment mechanisms

Error reporting

Observability

Standardisation

Scalability

Education

Code safety

Modularisation

Appendix 1: Background to Projects

Appendix 2: How we write JavaScript Code

Rationale

I have tried to answer the question "Why does this document exist?" and clarified why this is a helpful document to maintain.

This should be a living document that helps chart the progress and objectives for JavaScript concerns at [company name].

Introduction

Outlines the nature of the code in use at the company from a very high level. For my document, this highlights the 3 main different types of JavaScript codebases that exist in the company.

Sections

For each of the sections mentioned in the index, give a high level overview or history, outline what objectives you have for the future, show the current status of those objectives and explain what has been done so far:

Example of Section 1:

1. Code Quality

All of the JavaScript code repositories are working well and producing the needed outcomes for the business. There are some things that could yet be done to improve and standardise the quality of code to ensure that we are efficiently using developer time and improving deliverability and confidence.

Objectives:

Use the same linting rules for all packages where possible

Use language-specific coding styles (eg. functional over classes)

Use a single compiler where possible and follow the latest ECMAScript standards

Status:

Not started / In Progress / Completed / Cancelled

Details:

We will use Vite as a bundler and transpile from ESM where necessary. Each major concern of JavaScript code will be renovated and receive maintenance when possible.

Appendices

At the end of the sections, I have two appendices: one which has an overview of each codebase, and another that builds on the section on "Standardization of Code".

Example of Appendix 1 (Background to Projects):

Project name

This application provides a service to ...

Repo: https://github.com/dummy
Hosting: https://link.to/hosting-platform-backend
Pipeline: Github Actions
Code quality: Good/ Bad / Reasonable
Tests: None / Some / E2E only / Unit only
Status: In development / In production / Being renovated / Up to date

Example of Appendix 2 (How we write JavaScript Code):

Many thinks to my friend Josh Farley for this one. It's a table which hilights decisions on code quality and standardisation that aren't covered by Prettier or the AirBnB Style Guide.


Concern	Decision	Rational
File naming	Files should be snake, eg.Good:`detect-provider.js`Bad:`detectProvider.js`	This makes files easier to scan quickly in the source view
File structure	Files should not be nested beyond 2 levels.Good:root/src/component.jsBad:root/src/components/utils/frontend/component.js	This makes it much easier to discover components and avoids duplication
Spelling	Use American english spelling for variables and function names	Simplifies code for non-native speakers. One standard is better than none.

</div>

monorepos.

Sun, 04 Jul 2021 00:00:00 GMT

I've recently been introduced to the use of monorepos to manage projects, and have a few thoughts on the idea.

First of all, I'd better explain what a monorepo is. The idea is that instead of having many different Git repositories for a group of related projects, you have just one, containing all of your different projects that come under a common umbrella.

Many projects that I am interested, and to some extent involved in, use monorepo tools to manage their work.

Advantages #

A few of the advantages are that you can more easily get a "birds eye" view of the project, so if you're a new starter you can easily see what the coding styles are in other projects that might be similar to yours. This helped me when I started working at Purple Bricks: I could use some of the code that was already there in other applications to call APIs, build UIs and other things without having to ask my team mates hundreds of questions.

Another advantage is that you can also share resources such as NPM packages in a more uniform manner. There's often just one node_modules folder instead of one for each project, and you can have one package.json, meaning every project uses the same version of each module you have installed or built. If you publish a new version of a shared library, the updates are pulled into the project on the next release, which can save a lot of laborious manual updates.

Necessities #

There's a flipside to that though: because shared modules are updated instantaneously, thorough testing is needed to make sure and changes aren't going to detrimentally affect some projects.

There's also quite a bit of configuration involved, since all of your unit, integration and end-to-end tests are typically managed through the monorepo. I'm currently using the Nx monorepo management tool from Nrwl, which has done a lot to streamline the development process.

Nx has a command-line tool, so you can run nx affected:test to run tests for all of the applications you've affected, or altered, with the code you're writing. Nrwl claims to be very smart about this, so it can tell what's been updated and only test those affected apps.

In reality this means that if you change something in a shared library, you have to run all of the tests for that library, as well as all of the tests for each of the affected applications. This can take quite a considerable amount of time as you add different applications to your workflow, and in my experience it soon gets really slow.

We have a common shared UI library which is in active development, as most shared UI libraries tend to be. Each time we touch anything to do with these, it takes up to an hour for all of the CI checks to complete, and there can be timeout issues depending on the availability of the pipeline.

Lerna, an alternative, older monorepo management tool, has a concept of independent projects, which works similarly to if you have separate repositories. But that does negate some of the streamlining that shared projects have.

Verdict #

So far, I've benefited quite a bit by using Nx to manage our projects. development is somewhat simpler, it's learner friendly, and streamlines some of the issues you may have experienced if you've worked on projects with interdependent code.

However, it comes at quite a high price: configuration can be a pain, and build times do become extremely frustrating."

The Music of Mike Oldfield

Sun, 09 Jul 2017 00:00:00 GMT

I've enjoyed the music of Mike Oldfield since I was about 15. Wow, I've just realised that's about 20 years. What makes me keep coming back to this artist? What tracks would I recommend if you wanted to get an overview of his work?

I used to hang out with a group of kids who loved all kinds of music. I admit that I didn't have much of an idea myself, but took on board their recommendations, and quite early on one of them gave me a cassette tape of "Tubular Bells 2".

I had never heard of the artist before, but that wasn't unusual. What impressed me initially were the titles of some of the tracks. "Weightless", "Blue Dawn" and "Sentinel" sounded awesome to this hapless, sci-fi loving word-nerd. But when I stuck it in my stereo, it was the sound that really amazed me.

Looking back now on the albums released around that time, I can see this one represented a clear shift in Mike's style. For years he'd been releasing albums of rock music, with tracks typically around 5 minutes, lots of prominent vocals by mostly unknown artists, and some outstanding music and guitar solos.

Over the years, my favourite albums from this era have become "Islands", "Discovery" and "Crises", with the latter being the most challenging and rewarding of the three.

Paradigm Shift #

But this album, Tubular Bells 2, was a paradigm shift. When "Songs of Distant Earth" was released a short time later, I instantly bought the CD (for £18 I might add). I listened to it constantly for years.

This album still is my absolute favourite. It's sublime arrangements, for me at least, defy any genre classification. They instill in me a sense of wonder that only the TV series "Babylon 5" has managed to compete with.

At the time, I was getting into James Joyce. His idea of the "literary epiphany", a moment when you might percieve some kind of absolute beauty, I think is expressed by this album. Loved it.

Over the next few years I soaked up everything Mike published, which morphed into a dance / trance influenced style, I am told which came from living on the island of Ibiza.

I think of this era of ending with the very left-field classical album "Music of the Spheres". Mike doesn't talk about this album much, and I secretly wonder if was a studio-ordered collaboration between Mike and Karl Jenkins, which may have been difficult for such an independent and determined artist.

The album is nonetheless fantastic and one I listen to very frequently.

Others that I would recommend from this era are "Tubular Bells 3" and "Tr3s Lunas".

But there are two albums near the beginning and end of this group that are totally different: "Voyager", which consists mostly of arrangements of folk music, and "Man on the Rocks", a heavily vocal-led collection which seems to have been a bit of a catharthis from the solely classical "Music of the Spheres".

Origin Story #

Mike had originally started out bringing, as I like to call it, classical structure to rock music. His original few albums had very few track splits, each being around the 20-minute mark.

I was very pleased to hear that Mike was going to go back to this style with "Return to Ommadawn", for a few reasons.

Primarily, it seems this album was "one for the fans" - Mike was rewarding the people who had stuck with him through the years, even if it seemed as if that style of his was a thing of the past. But listening to interviews, it's clear that this is Mike's "home style" - that he's most comfortable with this structure and form of composition.

I must be honest, it took me years to understand and appreciate these albums.

I think my first mistake was believing that I had to enjoy the original Tubular Bells in order to do so. I didn't - and still don't - enjoy the one single album that people generally remember Mike for!

Yes it was a groundbreaking album that displayed a certain genius. But for me, it doesn't resonate for some reason.

Now, going through that early set, I genuinely can say I enjoy these albums as much as the others. Something that helped a great deal was the Elements album, which showcases shortened excerpts from these longer works.

The Future #

Mike Oldfield has produced a body of work that I constantly go back to. I eagerly anticipate his next work, and appreciate that even if I don't love it, at least I know I will enjoy and appreciate it.

Mike might not be one of those artists who get a vast amount of attention. However, his work on the whole has really resonated with me. I hope this short tour gives you some background and insight into an artist who is multi-layered, complex and deeply moving."

My Workflow Today

Tue, 17 Nov 2015 00:00:00 GMT

Workflows and tools are constantly changing. The tools we use and the workflow we have means we've simplified our build process and cut out some of the repetitive tasks, so that we can concentrate on simply doing the stuff we love as well as we can. This is my current set up:

MAMP — for developing locally #

Despite the learning curve, developing locally before you push your changes live avoids so many potential problems, and allows me to use some great pre-build tools that just don't work on servers that I have available.

I use MAMP because it was orginally built for the Mac and was the simplest one to use — and I tried a few. I recently bought the Pro version because it allows you to create virtual hosts. Virtual hosts make it much easier to go live (a lot less absolute URLs to change).

Sublime Text — for coding #

I've tried various tools for witing code, but keep coming back to Sublime Text. I really wanted to like Atom, but it was just too slow and buggy at the time. Sublime Text 3's extensions make a really great workflow that I've got really comfortable with (for the moment...)

Sublime Text Extensions #

Material theme #

Material Theme — because I love dark backgrounds, I find it helps me to concentrate. Plus it has folder icons that help me to see what kind of file I'm looking at at a glance. I find it's quite common for a theme to have great code hinting in either JS or SASS, but rarely both. Material seems to get a good balance between the two.

GitGutter #

This plugin allows you to see what lines have been changed since your last git commit. Really handy if you can't remember what you've done, or your working with a colleague who doesn't use git (it happens!).

Grunt — for repetitive task running #

I'm feeling a bit of pressure to move to Gulp for task running, but I have a well established workflow for Grunt which I don't want to rebuild at the moment ... so I'm going to stick with what I have!

Grunt-sass #

Grunt-sass uses the libSass version of SASS compiler, which is now at feature parity with Ruby Sass. Although it does mean that compiling is much faster, it's not the reason I went with Grunt Sass. Not having Ruby as a dependency for my project is cleaner and less to maintain and could mean less potential conflicts in the future.

PostCSS / Autoprefixer #

I moved to PostCSS for the use of Autoprefixer, which, as the name suggests, allows you to add browser-specific prefixes to your CSS. You can even specify how many previous browser versions you want to support. No more trawling through caniuse.com to figure out what prefixes you need. Want to clean up your code and remove old prefixes? You can do that really quickly with this tool.

PostCSS's creator seems to want PostCSS to do much more, but I find the documentation scant and folder structure (multiple nested 'node_modules' folders) a little jarring, so I haven't used it for anything else, although it shows great potential.

GitHub Desktop — for version control #

GitHub Desktop simplifies Git for me, something which I'm still not comfortable to use solely in the command line (although I do occaionally).

It does bother me that I'm only able to use GitHub accounts with GithubDesktop, so I've been investigating using something like Tower. I don't think I've got a good enough use case for this purchase at the moment really, and I do tend to think I should get used to the command line interface when it comes to GIT.

Cyberduck — for file upload / download #

I love the simplicity of Cyberduck, plus the whimsical yellow rubber duck sitting in your task bar raises a few eyebrows occasionally...

Trello — for project management #

Trello is a great project management tool. It's so flexible; for example, I've set it up as a Kanban board but if you prefer you can set up lists for today, tomorrow future etc or any way you like.

I will usually have 2 main boards: my current project board, and Project Overview board so I can track different projects and easily get a birds-eye view of where I am with each of my projects.

BugHerd — for feedback and bug tracking #

I love BugHerd. With this tool, you can include a script in your dev site and get users / clients / colleagues to click on the site, make annotations and collate these onto a project board. Then you can assign tasks out to different people or open a discussion about why you've built it this way...

Affinity Photo & Designer — for wire framing & designing #

Affinity is not as easy or as professional as using Photoshop and Illustrator — yet. I have become disillusioned with Adobe lately.

Affinity are providing an attractive solution which does work well despite the learning curve, and I'm really looking forward to their direct competitor to InDesign which should be in public beta soon.

What about you? #"

Working with styled components: multiple nested properties

Sat, 01 May 2021 00:00:00 GMT

I've seen this fairly frequently when it comes to using shared media queries. sometimes you want to set the width of an element based on one property, but you need to access the theme too.

const StyledContainer = styled.div`
  ${(props) =>
    props.narrow
      ? css`
          @media (min-width: ${props.theme.desktopMin}) {
            max-width: 48rem;
          }
        `
      : css`
          @media (min-width: ${props.theme.desktopMin}) {
            max-width: 32rem;
          }
        `}
`;

You could destructure them like this however it can become difficult to read, particularly the position of the closing bracket, which becomes difficult to see with the template literal syntax:

const StyledContainer = styled.div`
    ${({ narrow, theme })) => narrow ? css `
        @media (min-width: ${theme.desktopMin}) {
            max-width: 48rem;
        }
      ` : css `
          @media (min-width: ${theme.desktopMin}) {
              max-width: 32rem;
          }
      `
    }
`

I think it could be better not to destructure. You could easily start to think about what would happen when you have nested ternaries, which in my time I have spent too long trying to unpick."

Unit Test your React apps

Fri, 28 Feb 2020 00:00:00 GMT

Originally published in print for Net Magazine.

Do you really know what your code does what it is supposed to do? Have you tested it in the browser? What if you haven't, or you can't test everything, and it breaks in production?

A testing library is a group of utilties that developers use to write individual tests on our application components. Some of the principle parts of a test are:

Description: describe what the test is about
Use / Render: uses the component in an environment where it can be tested
Mocking: create pretend functions, so that you can check your assumptions

In this article I'm going to show some examples from React Testing Library to help you get started with this valuable way of improving the robustness of your code output.

Getting Started with React Testing Library #

I'm going to use create-react-app for this demo, because it already comes preconfigured with the testing library. If you're using Gatsby or a custom setup, there might be some configuration you need to run through before you start using the testing library.

To start let's create a new app. If you have a recent version of Node.js already, you can run the following command without installing anything else globally:

npx create-react-app netmag-javascript-testing

Now open the folder in your code editor.

Create-react-app is great because you can already run yarn test and see that one test is already passing. This command will also keep watching for file changes, so you can see straight away whether or not the tests you write are passing.

Let's start writing some tests!

What to Test #

Imagine we have a simple component, say a button with some state. What are some of the things that need testing in a component like this?

The appearance of the component

We don't want anything to change unexpectedly after we've written our component. So we're going to write a snapshot test to capture how it renders. Then, if anything changes we will see it quickly, without a manual or visual test. This is great for components that consist of many smaller components: you can see quickly when (and where) it's appearance has been affected.

The different branches that render

Because we could have 2 or more different outputs, we need to test that it's rendering all of them correctly, not just one. So we need to simulate a click event and have another snapshot test for the way it renders after this branch of code has been run.

That functions get called as expected

We want to ensure that the code we wrote to call another function works as we assume it will. But since that function is an external dependency, we don't want to test that here. Our tests should encapsulate only the functionality we want them to.

Writing our First Test #

Let's write our first test. Create a new file called MyComponent.unit.test.js in the same folder as the component. By adding test.js at the end, it'll be automatically picked by the testing library. The contents of that file are below:

import React from 'react'

import { render } from '@testing-library/react'

import MyComponent from './MyComponent'

describe('the <MyComponent />', () => {

    // tests go here

})

The first thing I want to draw your attention to is the describe() function, which takes 2 arguments: the first is a string that you can use to better describe as a string of text what your test is going to be doing. In our case we've said simply that it should render. This is very useful when someone else looks at your code, or you have to remember what you did at a later stage. Writing good "describe" statements is a form of code documentation, and another good reason for writing tests!

The second argument are your tests. The describe() function will run all of these tests one after the other.

Cleanups #

Let's introduce helper function called beforeEach(). We need to use this because each time we do something with the component, we want a fresh copy without the props we previously had passed to it still existing in the component. Or we might need to re-render the component. beforeEach() does that for us, and we can pass it the cleanup function:

import { render, cleanup } from '@testing-library/react'
...
describe('the component should render', () => {
  beforeEach(cleanup)
 }

Writing a Snapshot Test #

In this step, we're going to "mount" our component (or render it).

describe('the component should render', () => {
  beforeEach(cleanup)
  it('renders with basic props', () => {
    render(<MyComponent />)
  })
}

This render gives us access to all of the rendered properties of the compiled component. It might be good to drop this into a console.log()so you can see more clearly what it does.

If you do, you'll see that there's a few useful properties we can take advantage of here. I'm going to make an assertion (make a testable declaration) and test it by extracting the container. The container "contains" the DOM nodes (all of the HTML) associated with the component.

it('renders with basic props', () => {
    const { container } = render(<MyComponent />)
})

Now we have access to the container, how do I tell that it's rendered according to my assertion? By adding a snapshot test.

A snapshot is like a photograph. It takes a snapshot of our component at a specific point in time. Then, whenever we make alterations to the code, we can see if it still matches the original snapshot. If it does, we can be confident that nothing has changed in the component.

However, if it doesn't we might have uncovered an issue that originated in another component, one that we might not have spotted previously:

  it('renders with basic props', () => {
    const { container } = render(<MyComponent />
    expect(container).toMatchSnapshot()
  )

Testing Properties #

Props, or properties, of a component can be tested with snapshots too. Testing the different props you provide to your component will give you greater coverage and confidence ... you never know when a requirement is going to mean your component's props are refactored and the final output will change:

Add this object to the top of your file:

const lightProperties = {
    backgroundColour: 'white',
    textColour: 'darkblue'
}

We define the properties in an object and then use the spread operator (three dots followed by the object name, ...lightproperties) in this way because we can only pass one argument in when we render in this way. It's also useful to see what properties you're passing in isolation:

    it('renders with basic props', () => {
        const { container } = render(<MyComponent />
      )
     expect(container).toMatchSnapshot()
    })
    it('renders with the light version props', () => {
        const { container } = render(
            <MyComponent { ...lightProperties } />
        )
        expect(container).toMatchSnapshot()
    })

Testing Changes in the UI #

Imagine that our component has a button, and you want to make sure that something happens when the button is clicked.

You might think that you want to test the state of the application, for example, you might be tempted to test that the state has updated. However, that's not the object of these tests.

This introduces us to an important concept in using a testing library: we're not here to test the state, or the way our component works. We're here to test how people are going to use the component, and that it meets their expectations.

So whether or not the state has updated is immaterial; what we want to test is what the outcome of that button press is.

Let's imagine we're testing the outcome of a function that changes the UI from dark mode to light mode. Here's the component:

const modeToggle = () => {
    const [mode, setMode] = useState['light']
   const toggleTheme = () => {
     if (theme === 'light') {
       setTheme('dark')
     } else {
       setTheme('light')
     }
   }
    return (
        <ToggleButton data-testid="mode-toggle" lightMode={mode} onClick={toggleMode}>
          Toggle mode
        </ToggleButton>
    )
}

First, we should add a test id onto the button so that we can find it in the render phase:

return (
    <ToggleButton
      data-testid="mode-toggle"
      lightMode={mode}
      onClick={toggleMode}
     >
      Toggle mode
    </ToggleButton>
)

Did you notice we added the new attribute data-testid to the button? Here's how you might test that.

First, import a new function, fireEvent from the testing library:

import { cleanup,
          fireEvent,
          render
} from '@testing-library/react'

Now we can use that function to test that there are changes in the UI, and that those changes are consistent:

it('renders with basic props', () => {
    const { container } = render(<ToggleButton />
  )
 expect(container).toMatchSnapshot()
})
it('renders the light UI on click', () => {
    const { container, getByTestId } = render(<ToggleButton />)
    fireEvent.click(getByTestId('mode-toggle'))
    expect(container).toMatchSnapshot()
})

This is great, we don't have to manually go to the site and look around, then click the button and look around a second time, which you might admit, you'll likely forget or miss something! Now we can have confidence that, given the same input, we can expect the same output in our component.

A Note about Test IDs #

Personally, I dislike using data-testid to find something in the DOM. After all, the object of tests is to mimic what the user is doing, and to test what happens when they do. data-testid feels like a bit of a cheat -- although data-testids will likely come in handy for your QA Engineer (see the box "The Role of Quality Assurance Engineers").

Instead we could use getByText() and pass in the text of our button. That would be a lot more behaviour specific.

Mocking and Spying #

Sometimes we need to test a call to a function, but that function is outside the scope of the test. For example, I have a separate module that contains a function that calculates the value of pi to a certain number of decimals.

I don't need to test what the result of that module is. I need to test that my function does as expected. For more information about why this is, please see the box "unit and integration tests". In this case, we could "mock" that function:

const getPiValue = jest.fn()
it('calls the function on click', () => {
    const { container, getByTestId } = render(<ToggleButton />)
    fireEvent.click(getByTestId('mode-toggle'))
    expect(getPiValue).toHaveBeenCalledTimes(1)
  )
})

The functiontoHaveBeenCalledTimes() is one of the many helper functions in the testing library that allow us to test the output of functions. This allows us not only to scope our tests only to the module we want to test, but also allow us to "spy" on, or see what our function does when it calls that function.

Start Testing your React Applications Today #

Writing tests can seem a little daunting to start with. I hope this tutorial has given you a little more confidence to try it. Since I started writing tests for my applications, I really can't go back: I can rest easier, knowing I'm leaving behind a much better legacy for those who will use my work in the future.

For more ideas about how to test your components take a look at one of the links below:

https://react-testing-examples.com
https://testing-library.com/docs/react-testing-library/intro

If you're looking for some courses to help you get started, the one by Kent C Dodds (who wrote and maintains React Testing Library) is popular:

https://testingjavascript.com/

I also enjoyed this one on Level Up Tutorials, it's the one that got me started writing tests for my code:

https://www.leveluptutorials.com/tutorials/react-testing-for-beginners

Boxout: Unit and Integration Tests #

There are generally two types of tests we talk about in the context of code tests:

Unit Testing #

This is to test the units of code, or each module in isolation. They shouldn't call other modules in your project, instead you should mock these modules so that the unit test is able to run.

Integration Testing #

When two or more modules are used in conjunction, then it's a good practice to write an integration test. These test how modules work together. Well written integration tests can identify issues when you're using modules as part of a larger piece of work.

These are the two kinds of tests that developers are usually responsible for writing. There are other kinds of tests that QA Engineers write (see the box "The Role of Quality Assurance Engineers").

Boxout: What is Test Driven Development? #

TDD, or test-driven development, is a testing discipline that forces you to think about edge cases when you're in the process of development.

You start out with a hypothesis that forms the basis for your tests. This could be something like "it renders a button", "it renders a button using the dark mode ui on click", then you go ahead and write your tests.

Only after you have at least 3 robust tests that you can use to verify what your component's behaviour should be, then you start coding the actual component.

I like this concept because it forces you to think about the structure of your component from completely different angles. You're not trying to preserve the code you already wrote, so you can find different ideas more easily.

Be careful though: you can write tests that will pass under any circumstances! For this reason, three or more tests are often required that allow you to "triangulate", or effectively define the functionality of your component, before you can start building it.

Boxout: The Role of Quality Assurance Engineers #

Quality Assurance engineers are the secret superpower of web development. A good QA Engineer and a strong QA process can save development time, reduce issues in production, and have been known to save the reputation of entire development teams ... and the companies they work for!

QA Engineers have totally different mindset to developers: Developers naturally build against assumptions about how something should work, and can only test these assumptions. QAs on the other hand, challenge these assumptions, using every trick they can think of to try to break the code in some way.

This ensures that your customers aren't acting as your QAs, causing you to lose money and good will in a very public manner.

A dedicated Quality Assurance engineer will write an automated tests suite that check the expected functionality of the built application, predominantly on a staging domain, and end-to-end tests that check users can carry out goals that you expect them to achieve."

Osip Mandelshtam / Philip Nikolayev

Thu, 02 Apr 2015 00:00:00 GMT

Since reading Boris Pasternak's Doctor Zhivago, I've been deeply moved by his poetry. I've just discovered the wonderful contemporary Russian poet Osip Mandelshtam, who, like Pasternak, evokes some wonderful imagery and delightful reveries.

Philip Nikolayev has a wonderful command of the English language, and his translations of Mandelshtam's poems is really delightful, particularly in his use of alliteration. Here's my favourite one:

<div class="[&>pre]:ml-0! [&>pre]:mr-0! [&>pre]:bg-transparent!">

Insomnia, Homer, taut sails: my lips have lisped
Down to the middle the detailed list of ships,
That long brood and angular train of cranes
That rose above Hellas once on wings of waves.
A wedge of cranes into far foreign lands -
Divine white froth forming upon kings' heads -
Where have you hoist sail to/ If it were not for Helen
What would Troy alone be to you, Achaean men?
The sea and Homer - as all - are moved by love;
Which should I hark? Homer the first to fall
Silent, the black sea's cries still rhapsodize
Reaching to where I lie with sleep-filled eyes.

</div>

If you want to find out more about Osip Mandelshtam, there's a dedicated facebook page. The translator, Philip Nicolayev, is curator of Fulcrum: an annual of poetry and aesthetics."

Normalizing data using array.prototype.reduce

Tue, 18 Jun 2019 00:00:00 GMT

reduce() has terrified me for some time! I hadn't wanted to touch this new array method because I couldn't quite follow what each of it's properties meant. But recently I had an opportunity to deep dive into this function. Here's how I refactored an object to get the results I needed:

Edit: I realize this article doesn't really show you enough about how to use reduce, but it's still coming up in my SEO keywords quite frequently. So here's a more thorough article on the topic: https://dev.to/ra1nbow1/5-ways-to-use-the-reduce-method-in-js-1k8h

I'm beginning to realise how important good API design is. One API I call for a project has only one endpoint: a list of items. Each item is quite large, with loads of tags and categories. Yet I can't query the API by any of those.

This became an issue when I had a requirement to display them by categories. The real trick was that the category was inside a nested object. AND it had to be matched and replaced by a different value before I displayed it.

Here's an example of the object once I'd parsed the JSON:

{
    ---
title: "some title",
    categories: {
        team: "teamone"
    }
}

What I needed for it to be displayed was this:

{
    teamone: {
        [{
            ---
title: "some title",
            categories: {
                team: "teamone"
            }
        }, {
            ...
        }, ]
    }
}

Step 1: Map & Replace values #

The first thing I did was to map the new values of the team using the following method:

itemsTeamsMap {
                "teamone" : "team1",
                "teamtwo" : "team2",
}

const itemsMapped = []

  const filteredItems = edges.map(item => {
    item.node.categories.team = itemsTeamsMap[item.node.categories.team]
    itemsMapped.push(item)
  })

This was especially important because it wasn't a 1:1 match, some teams would be reassigned to other new values.

Step 2: Sort items using Reduce() #

Once I had that, I needed to sort my items by the new team values I'd given them so that grouping would be easier using a bubble sort:

const sortedItems = ItemsMapped.reduce((acc, current) => {
    acc[current.node.categories.team] = acc[current.node.categories.team] || []
    acc[current.node.categories.team].push(current)
    return acc
  }, Object.create(null))

It took me a good while to get my head around the reduce function, something that's been called the most powerful function in JavaScript. The first argument is the accumulator value, or something which accumulates each iteration of the function.

The second value is the current item you're iterating over, which you can modify with any function you put into the body.

Here' I'm pushing the current value into the team item of each item in the accumulator and then creating an object from that value.

The result of this function on my original data is this:

{
    team1: [{
            title: 'some title',
            categories: {
                team: 'team1'
                    ...
            }
            ...
        }, {
            ...
        },
        ...
    ]
}

This was a good deal closer to the final object that I needed. Of course though, I needed an array of objects that I could iterate over in my JSX template...

Step 3: Array of Objects #

  const finalisedItems = Object.entries(sortedItems).reduce(
    (acc, [team, items]) =>
      acc.concat({
        team,
        items,
      }),
    []
  )

I love the new object methods introduced recently to JavaScript. They allow you to do so much more with objects natively ... in this function, for each entry of the object I'm passing in, it will concat the team into an array of objects, where team is the title, and each group of items is an array of objects inside of that:

[{
        team: 'team1',
        items: [{
                title: 'some title',
                categories: {
                    team: 'team1'
                        ...
                }
                ...
            },
            ...
        ]
    }, {
        team: 'team2',
        items: [{
                title: 'some other title',
                categories: {
                    team: 'team2'
                        ...
                }
                ...
            },
            ...
        ]
    },
    ...
]

Step 4: templatingNow I can iterate over each team, then each item in that team, outputting semantic elements (<li>s) for each item:

 {finalisedItems.map(itemsTeam => (
        <div key={itemsTeam.team}>
          <h3>{itemsTeam.team}</h3>
          <ul>
            {itemsTeam.items.map(item => (
              <li key={item.node.id}>
                <a to={item.node.url}>
                  {item.node.text}
                </a>
              </li>
            ))}
          </ul>
        </div>
      ))}

Going the extra mile in normalising this object isn't a challenge I relished, however I have improved a lot as a developer by working through and providing a solution.

I learned that trying to do a faster job didn't yield the correct results, so my fastest solution was by far the best.

Sometimes you've just got to knuckle down, go in for the long haul, and you'll find you've come out the other end similarly restructured (in a good way!)"

I'm Not Proud

Wed, 30 Jan 2019 00:00:00 GMT

Today was a stressful day. I was working to a set deadline on which a lot of other tasks depended. I felt that if I failed to achieve my work, there would be a cascade effect that could lead to delaying the release of a product I have been working on.

I didn't want that to happen. But I'm not proud of what I did to avoid it.

I was working with our best designer, who had begun to refactor a module but needed some help with some finer structure and more complicated CSS that would display the images in the way that the Creative Director had approved.

In order to hit my deadline I hacked the module I was working on. I added props that were not semantically correct. I added images directly into the module itself instead of passing them down from the page. I wrote no tests and defined no types.

After another round of tweaks and some refinements, it was ready for code review. I rushed it past, asking someone I knew would be too busy to tell me what I should've done instead. I'm sure he raised an eyebrow or two when he approved it.

Miraculously, it passed QA and was released that same day.

I still hate myself whenever I think of that code.

Unexpected Consequences #

We started a new sprint today, and one of the backlog items is a refactor of that module. This time I'm going to do it correctly, passing down the image from the page, removing the duplicate CSS and finding a way for our designer to pass specific CSS so they can position the image as they want.

What I didn't expect was to hear that our user engagement has increased significantly. Conversions are up by about 11%, and the project is much closer to launch than it would have been without those hacky changes I made.

So no, I'm not at all proud of the code I wrote today.

But it helped the whole project move forward. It might even release very soon. And now I'm going to re-do it properly."

Observing Element Changes in the Shadow DOM

Tue, 16 Jul 2024 00:00:00 GMT

DOM content isn't always loaded in top-to-bottom. Custom elements sometimes need to wait for API calls, and sometimes you might instantiate some of the DOM elements via JavaScript. Here's a handy function to observe those changes

Working with large Drupal sites is interesting from a management perspective. It does do a lot of AJAX calls to the backend, and sometimes that might result in the DOM structure changing because of that call.

NOTE: I've got an updated article which simplifies this example, and talks about the implementation in detail.

But this happens a lot in complex applications, and isn't limited to apps managed by PHP. For example, sometimes we need to, or want to do something similar to this:

const container = document.querySelector('put-stuff-here');

container.appendChild(`<div>${someVarible}</div>`);

Or sometimes it's more complex, where you have a custom element that has to be registered, and perhaps has to fetch some data before it renders.

I found myself needing to hook into these state changes a few times recently. Quite often, a mutationObserver was enough to do the trick. However it does contain quite a bit of boilerplate:

/**
 * Wait for an element to be added to the DOM.
 * @param {ShadowRoot | Document} root - The root to observe.
 * @param {string} selector - The selector of the element to wait for.
 * @returns {Promise<HTMLElement | any>} - A promise that resolves with the element when it is added to the DOM.
 */
function waitForElement(root, selector) {
  return new Promise((resolve) => {
    let element = root.querySelector(selector);
    if (element) {
      resolve(element);
    } else {
      const observer = new MutationObserver((mutations) => {
        mutations.forEach((mutation) => {
          if (mutation.type === 'childList') {
            element = root.querySelector(selector);
            if (element) {
              observer.disconnect();
              resolve(element);
            }
          }
        });
      });

      observer.observe(root, { childList: true, subtree: true });
    }
  });
}
export default waitForElement;

This is more or less straight from the docs, but 2 things are missing: shadowDOM support (something I use a lot) and some way of resolving the promise to stop it from running indefinitely.

const observer = new MutationObserver((mutations) => {
  mutations.forEach((mutation) => {
    if (mutation.type === "childList") {
      mutation.addedNodes.forEach((node) => {
        if (node.nodeType === Node.ELEMENT_NODE) {
          if (node.matches(selector)) {
            observer.disconnect();
            clearTimeout(timeoutId); // Clear the timeout
            resolve(node);
          } else if (node.shadowRoot) {
            const shadowElement = node.shadowRoot.querySelector(selector);
            if (shadowElement) {
              observer.disconnect();
              clearTimeout(timeoutId); // Clear the timeout
              resolve(shadowElement);
            } else {
              observer.observe(node.shadowRoot, { childList: true, subtree: true });
            }
          }
          element = root.querySelector(selector);
          if (element) {
            observer.disconnect();
            clearTimeout(timeoutId); // Clear the timeout
            resolve(element);
          }
        }
      });
    }
  });
});

My issue with this is there are lots of nested if statements going on here, I might try to tidy that up soon.

But essentially it works the same by checking for the existence of a shadowRoot and resolving the element if it's found there.

There's only one other thing we need to fix: to ensure it doesn't keep running indefinitely if the element doesn't show up in a reasonable amount of time.

Timeout

Ahh good old timeouts. I'm trying to think if there's a better way to do this.

Let's wait for a timeout of 1 second to expire and then reject and disconnect the observer:

observer.observe(root, { childList: true, subtree: true });

// Set a timeout to reject the promise if the element is not found within 1 second
const timeoutId = setTimeout(() => {
  observer.disconnect(); // Stop observing when the timeout expires
  reject(new Error("Element not found"));
}, 1000);

Putting this together we can observe elements across the normal DOM and the shadow DOM, and provide a slim implementation on top of mutationObserver so we can be certain the element exists before we try to do something like attach a listener or modify it in some other way.

Here's how to implement it:

const myElement = waitForElement(parent, 'my-element');

Here's the full code, with an additional parameter to change the timeout if you like. Happy copypasta!

/**
 * Wait for an element to be added to the DOM or shadow DOM.
 * @param {ShadowRoot | Document} root - The root to observe.
 * @param {string} selector - The selector of the element to wait for.
 * @param {number} [timeout] - The time in milliseconds to wait before rejecting the promise.
 * @returns {Promise<HTMLElement | Node>} - A promise that resolves with the element when it is added to the DOM or shadow DOM.
 */
function waitForElement(root, selector, timeout = 1000) {
  return new Promise((resolve, reject) => {
    let element = root.querySelector(selector);
    if (element) {
      resolve(element);
    } else {
      const observer = new MutationObserver((mutations) => {
        mutations.forEach((mutation) => {
          if (mutation.type === "childList") {
            mutation.addedNodes.forEach((node) => {
              if (node.nodeType === Node.ELEMENT_NODE) {
                if (node.matches(selector)) {
                  observer.disconnect();
                  clearTimeout(timeoutId); // Clear the timeout
                  resolve(node);
                } else if (node.shadowRoot) {
                  const shadowElement = node.shadowRoot.querySelector(selector);
                  if (shadowElement) {
                    observer.disconnect();
                    clearTimeout(timeoutId); // Clear the timeout
                    resolve(shadowElement);
                  } else {
                    observer.observe(node.shadowRoot, { childList: true, subtree: true });
                  }
                }
                element = root.querySelector(selector);
                if (element) {
                  observer.disconnect();
                  clearTimeout(timeoutId); // Clear the timeout
                  resolve(element);
                }
              }
            });
          }
        });
      });

      observer.observe(root, { childList: true, subtree: true });

      // Set a timeout to reject the promise if the element is not found within 1 second
      const timeoutId = setTimeout(() => {
        observer.disconnect(); // Stop observing when the timeout expires
        reject(new Error("Element not found"));
      }, timeout);
    }
  });
}
export default waitForElement;

Hopefully this will provide you with a basis for a utility of your own.

On Being An Advocate

Sat, 23 Jan 2016 00:00:00 GMT

I've always been interested in psychology but never went further than college with it. Recently a friend introduced me to the Meyers-Briggs Type Indicator Test. It illuminated some important traits of my personality and hilighted the motivation behind the goals I have. I thought I'd go over a few points that interested me here.

I studied Carl Jung a bit in college, and always found psychology an absorbing and fascinating field. I was particularly drawn to the practice of counselling, and pursued that for a while before realising that's not what I really wanted to achieve in life.

The website Sixteen Personalities has a scarily accurate test which identifies your type based on the theory that there are 16 main personality types.

In my test, it was revealed that I am an INFJ personality type — identified on this website as "the Advocate". So here's a little bit more about him.

"INFJs tend to see helping others as their purpose in life" #

This really struck a cord with me. Whether it's helping to find other developers work, providing training or support for colleagues, or just listening to the personal troubles of my friends, I'm always aiming to help others pursue their goals.

I try not to be motivated by self-interest and find no satisfaction in sales targets or monetary bonuses. I'd like to be in a position where I can encourage and help others in work and in life generally.

"There is a running theme with INFJs, and that is a yearning for authenticity and sincerity – in their activities, their romantic relationships, and their friendships" #

I find I identify quite closely with this statement. I'm always trying to be honest with other people and expect honesty back. At the same time, INFJs have a tendency to be able to spot insincerity and lies. If I found I have been lied to I can react quite badly.

"If there’s anything INFJs avoid, it’s the accumulation of power over others – and the people who are drawn to that type of power" #

I sometimes derail situations when I perceive that I have power over others, and it's one of the reasons I don't like to speak publicly even though I have been told I'm a competent speaker :- it puts me on a pedestal and could lead to others investing confidence in me personally, whereas I want others to focus on the goals they have and the tasks they've been assigned.

I have been known to actively undermine the confidence of others in me, so that I don't feel like I have authority over them.

"INFJs crave creativity, the ability to use their insight to connect events and situations, effecting real change in others’ lives personally" #

Hmm, how interesting this is in conjunction with my chosen career. I am not content as a coder or a designer, but I want to perceive the connection between the two, as well as the role of content writer and information architect. I have always sought a holistic approach to my career.

"INFJs are still Introverts, — they will need to step back and act the lone wolf from time to time, pursuing their own goals in their own ways" #

This was an interesting revelation. Whilst I feel satisfied in contributing to the goals of others, I still love the times I'm working on personal projects, or writing articles for this blog.

"[INFJs] imagination and empathy make [them] someone who not only cherishes their integrity and deeply held principles but, unlike many other idealistic types, is also capable of turning those ideals into plans, and executing them" #

It's always nice to get a bit of a personality massage, but I do have deep-seated principles which I am not prepared to compromise on. Yet when I comprehend the reason these are asked of me, I can usually think of a way around the situation in a mutually beneficial way.

Conclusion #

The trouble with taking this test is that now I want everyone I know to take it so I can discover more about them! There are a few people whom I have no idea where they're coming from, so I'm scheming for them to take the test at some point in the hopes of understanding them a bit better."

On digital estates

Mon, 29 Nov 2021 00:00:00 GMT

I recently heard the term 'Digital estate' to mean the technical property of an organisation. This, and the associations the term provides, have got me thinking about how we contribute to the success of organisations over time...

Let's start with this question: what does the word "estate" conjure up in your mind? I think it might include a variety of things. A house? Probably? The physical things you own? Maybe.

I live in a country where that term can have two very distinct meanings beyond those two things.

In times gone by, the ruling elite would have an estate. It usually consisted of one major, very large property that resembled (or actually was) a castle or manor house. It would also include extensive land around that area, much of it farmland. It could also include villages where people often worked for the estate. It was like the country's economy in miniature.

If you've watched Downton Abbey, you probably know what I'm getting at here.

Then ... there's the council estate.

In this context, an estate is a collection of houses owned by a local council where people of less secure financial means can find accommodation. Although valiant attempts have often been made to increase the opportunities for people who live in these kinds of places, there's often still a lot of crime and violence. For many people it's home, and therefore it's a nice place to live. But given the opportunity, would they choose to live elsewhere?

So what kind of "estate" are we talking about when it comes to a digitalestate?

I guess it could be either really.

Be mindful #

What I'm saying is that we should be mindful of what we're actually building. The code we write on a daily basis contributes to the overall estate of the organisation we're working for. Are we making that estate into something that is easy to move around in, neat, tidy, something we can be proud of? Or is it something that serves a necessary function but is difficult to work with?

It depends on our attitude towards our work ... are we tempted to think that, because our surroundings are not conducive to finishing things off nicely, that we should do a sloppy job? Or do we have the opinion that we should try to improve the quality of life regardless of the situation of those around us?

The organisations view #

The overall organisation has a huge impact on the digital estate. Sometimes in ways they might not imagine. For example, technical debt is likely to always exist. But are teams encouraged to tackle this debt? Or does the organisation skirt around the issue, thinking that they can maintain velocity without paying it off?

for more on this subject, please see this slideshare

The organisation might want to keep producing features under pressure from stakeholders or higher management. Or they might make decisions about launch timelines of new digital products without asking the opinion of those who understand the cost of delivery.

Both of these increase the rate of technical debt. So "shipping faster" is counterintuitive to the ultimate velocity of the organisation. On this subject, I really enjoyed this article on the InfoQ website, which claims that "In the future, technical debt will become less of an engineering problem and more of an important business prerequisite that helps with delivering more value to our customers and the business.""

From PHP to JavaScript: How I Switched Stacks

Mon, 08 Aug 2016 00:00:00 GMT

I've been working in web development for the past 8 years or so. Traditionally, I have worked at agencies, creating sites that have been designed in Photoshop, and built afterwards with a CMS integration such as Perch, WordPress, or similar, with occasional forays into Laravel.

During that time I've had the opportunity to work with JavaScript too. But understanding the language well seemed a bit beyond my grasp. One of the reasons for this was, I think, that I was trying to juggle several languages at once. I spent a lot of time understanding design patterns in PHP. Then, after I interviewed at Google, I decided to focus more deeply on CSS and HTML. And with these PHP projects, a lot was done in that language, leaving JavaScript as a vehicle for presentational enhancement only.

But then, because of concerns about WordPress' future, the agency I was working for invested time for its developers into finding a possible alternative to that popular platform. Lead developer Chris Geary saw an opportunity here. Instead of trying to replace WordPress with another platform in a 1:1 comparison, he realised we could deliver serverless sites using some cutting-edge tools that were emerging.

Investigating Serverless #

This approach had the advantage of allowing a transition: the agency could keep selling and building WordPress sites (they'd been doing this for 10 years and were comfortable with it), but these sites would have a separate front-end using the headless CMS strategy. It also allowed them a marketing opportunity: headless was a buzzword, they could position themselves as a thought leader and gain reputation from that.

So began my journey to serverless. I was introduced to Netlify, and promptly moved my sites from PHP to static, which allowed me to see the benefits of continuous integration, and saving myself a monthly hosting fee in the process.

I experimented with Jekyll and Hugo, but I didn't see a future for me in either of these languages (Ruby and Go, respectively).

Then Chris started talking about Gatsbyjs. So I built a Gatsby site, FreeBabylon5.com. As mentioned, I hadn't done a huge amount of heavy lifting in JavaScript before. I'd only toyed with React. I had no idea what I was doing and that felt awesome.

As I learned Gatsbyjs I noticed a typo in the docs and submitted an issue. I was invited to fix the issue myself and submit a pull request. I had never done anything like this before! Cloning an open-source project, branching off so I could fix the issue, pushing up my code and submitting a Pull Request ... wow. With this new knowledge I felt like I was finally able to give back to some of the amazing tools I'd so often used in the past.

Documenting the Experience #

My personal view is that I should document everything I learn, in the hopes that someone else might be able to benefit from my experience. So I started writing about my explorations here on delicious reverie.

To my amazement the creators of Gatsbyjs read my post and asked me to publish it on the Gatsbyjs blog. This gave me more confidence to progress further along this path, invest in my own learning towards JavaScript and using Gatsbyjs for more projects.

Around the same time, my colleagues and I became more concerned for developers using WordPress. It seemed that many of them could soon be pushed out of that platform for reasons I'll explain in another post. We wanted to help them identify another route that would also encourage best practices in web development.

So I submitted a talk outline for WordCamp London, which is usually attended by 400-500 developers. I also approached print magazine Net on the same subject. Both the talk and the article were accepted. I soon found myself on the main stage at the conference delivering a 40-minute talk, and saw my material published in the print magazine.

I get so terrified about pushing myself forward in these ways, especially because I was new at all of this. But I felt I had something that could potentially help some other people. That motivated me to overcome my abject fear.

Zopa's Serverless Initiative #

Around this time peer-to-peer lender Zopa were looking to replace a Ruby app with a static website. So they approached me directly and called me for several weeks until I finally had time to hear what they were trying to say.

Zopa had already identified Gatsbyjs as their tool of choice because they had lots of in-house JavaScript and React expertise. They had partnered with an agency to design and build the site, and wanted me on board to help develop it because of my experience with the framework. I'll be able to tell you more about the site when it launches, but it's been a great experience working with some excellent and incredibly smart people both at the agency and at Zopa.

I've had to learn a lot in a short space of time, but Zopa have been supportive and patient with my learning experience. In return, I've been able to help identify where Gatsby's APIs can be used to great effect instead of a custom solution that renders content on the frontend. I've been able to help other developers explore the GraphQL syntax and use Gatsbyjs's built-in IDE to query the data structure. And I've helped identify where opportunities to use functions hosted on other platforms enable us improve performance.

What I've most enjoyed is seeing those "wow" moments as developers really get GatsbyJS, and how, because of it's approach to data, it's not just a static site generator, it's much more than that.

So now, instead of working with agencies on predominantly PHP projects I'm a JavaScript dev working in central London. The deadlines are just as tight, the pressure to deliver can sometimes be much more, and learning a complicated custom-built infrastructure has been a challenge. But I've really enjoyed experiencing things from this side of the fence.

I'm continuing to work with internal teams at Zopa to give them the opportunity to contribute to the project. I'm really hoping I can do more with GatsbyJS as a contributor. And I'm glad to say that in spite of missing the close-knit community of developers I've been around up until now, I'm happier being focused on JavaScript than I've ever been."

Poem Review: She Weeps Over Rahoon, James Joyce

Mon, 21 Nov 2016 00:00:00 GMT

A very dear friend introduced me to James Joyce many years ago with his coming-of-age almost-biography, A Portrait of a Young Man, which had a deep impact on me. But it's only now I'm starting to pick up on just what a master of the English language Joyce was—through his poetry.

Joyce for many people, I am sad to say it, might always be an inaccessible bastion of 20th century literature. It's true that I've struggled and failed to maintain enthusiasm for, much less read Ulysses and Finnegan's Wake, as highly regarded as they are.

But Joyce's power over the English language was demonstrated ever so well in his poems, too, and it's these that I treasure above his larger volumes. If you're not convinced, spend a few hours reading chamber music carefully. It's truly beautiful and evocative.

Yet, until very recently, I had totally missed out on this shorter poem:

She Weeps Over Rahoon #

Rain on Rahoon falls softly, softly falling,

Where my dark lover lies.

Sad is his voice that calls me, sadly calling,

At grey moonrise.

Love, hear thou

How soft, how sad his voice is ever calling,

Ever unanswered, and the dark rain falling,

Then as now.

Dark too our hearts, O love, shall lie and cold

As his sad heart has lain

Under the moongrey nettles, the black mould

And muttering rain.

There is some biographical background to this poem, which is that Joyce's lover was visiting the grave of a previous love, which inspired Joyce to write the poem. Doesn't it evoke some of the poignant sadness of that occasion, as well as some of the other sad fates of unmentioned Irish men?

But as with many 20th century authors, although the context is interesting, the text is much more than a sum of its parts. For me the cadence of the poem as well as its structure conjures up the wind, rain and bleakness of Rahoon. But just enough of it is obscure that I start to ruminate over what I don't know, or isn't explicitly stated.

For instance, why is it Rahoon she weeps over, not her lover? Is there some greater tragedy below the surface? The line dark too our hearts makes me think about my own mortality too, which brings the tragedy home with a great sharpness.

Joyce was an absolute master of the English language, and seemed to know just what to say that would reach through the pages into the readers' heart."

Poems about Time

Mon, 02 Jan 2023 00:00:00 GMT

Time is a funny thing. Whilst modern science has helped us to see it's not as constant as we might have imagined, we still rarely are able to see it and our experiences within it from different perspectives. Although I really enjoy these few poems that seem to be able to step outside of the flow in one way or another.

Clouds - Joni Mitchell

Why I like it: Joni has an amazing mind and all of her music shares the immediacy and intimacy of a life of experiences. Yet this one in particular is written from three unique perspectives: the first verse is Joni as a child looking at clouds and noticing, perhaps for the first time, the passing of time. As the verses progress, she gets older and reaches adulthood and then old age, and has time to reflect on the similarities between clouds, and love, and indeed life itself

<div class="[&>pre]:ml-0! [&>pre]:mr-0! [&>pre]:bg-transparent!">

Rows and flows of angel hair
And ice cream castles in the air
And feather canyons everywhere
Looked at clouds that way
But now they only block the sun
They rain and they snow on everyone
So many things I would have done
But clouds got in my way
I've looked at clouds from both sides now
From up and down and still somehow
It's cloud illusions I recall
I really don't know clouds at all
Moons and Junes and Ferris wheels
The dizzy dancing way that you feel
As every fairy tale comes real
I've looked at love that way
But now it's just another show
And you leave 'em laughing when you go
And if you care, don't let them know
Don't give yourself away
I've looked at love from both sides now
From give and take and still somehow
It's love's illusions that I recall
I really don't know love 
Really don't know love at all
Tears and fears and feeling proud
To say, "I love you" right out loud
Dreams and schemes and circus crowds
I've looked at life that way
Oh, but now old friends they're acting strange
And they shake their heads and they tell me that I've changed
Well something's lost, but something's gained
In living every day
I've looked at life from both sides now
From win and lose and still somehow
It's life's illusions I recall
I really don't know life at all
It's life's illusions that I recall
I really don't know life
I really don't know life at all

</div>

Time - Pink Floyd (David Gilmour, Nick Mason, Roger Waters and Richard Wright)

Why I like it: This is one of the songs I grew up with: my dad played it for me when I was a teen. It's bound to a young mans' perspective but is remarkably foresighted; the writer already perceives that life will go by quickly and will inevitably end in regret one way or another.

<div class="[&>pre]:ml-0! [&>pre]:mr-0! [&>pre]:bg-transparent!">

Ticking away the moments that make up a dull day
Fritter and waste the hours in an offhand way
Kicking around on a piece of ground in your hometown
Waiting for someone or something to show you the way

Tired of lying in the sunshine, staying home to watch the rain
You are young and life is long, and there is time to kill today
And then one day you find ten years have got behind you
No one told you when to run, you missed the starting gun

And you run, and you run to catch up with the sun but it's sinking
Racing around to come up behind you again
The sun is the same in a relative way but you're older
Shorter of breath and one day closer to death
Every year is getting shorter, never seem to find the time
Plans that either come to naught or half a page of scribbled lines

Hanging on in quiet desperation is the English way
The time is gone, the song is over, thought I'd something more to say

</div>

Shakespeare - Sonnet 19

Why I like it: Maybe Dylan Thomas was thinking of this sonnet when he was writing "do not go gentle", another poem I admire but for different reasons. In this poem, the protagonist (it could be Shakespeare; but he hardly ever wrote from his own perspective) rages against time, almost sticking a thumb up, because he believes he has found a way to outwit time: by writing this very verse.

<div class="[&>pre]:ml-0! [&>pre]:mr-0! [&>pre]:bg-transparent!">

Devouring Time,  blunt thou the lion’s paws,
And make the earth devour her own sweet brood;
Pluck the keen teeth from the fierce tiger’s jaws,
And burn the long-liv’d phœnix in her blood;
Make glad and sorry seasons as thou fleets,
And do whate’er thou wilt, swift-footed Time,
To the wide world and all her fading sweets;
But I forbid thee one most heinous crime:
O, carve not with thy hours my love’s fair brow,
Nor draw no lines there with thine antique pen;
Him in thy course untainted do allow
For beauty’s pattern to succeeding men.
Yet do thy worst, old Time: despite thy wrong,
My love shall in my verse ever live young.

</div>

Keats - Ode on a Grecian Urn

Why I like it: This ode remains my favourite verse on the subject of time. In it, time is frozen in one single moment by the creation of art. It's almost a response to Sonnet 19 above, where someone future is able to appreciate that time is beaten; but Keats' luscious and very living words betray a sad fact: the impression is real, but the people, as Keats himself, are gone.

<div class="[&>pre]:ml-0! [&>pre]:mr-0! [&>pre]:bg-transparent!">

Thou still unravish'd bride of quietness,
  Thou foster-child of silence and slow time,
Sylvan historian, who canst thus express
  A flowery tale more sweetly than our rhyme:
What leaf-fring'd legend haunts about thy shape 
Of deities or mortals, or of both,
    In Tempe or the dales of Arcady?
  What men or gods are these? What maidens loth?
What mad pursuit? What struggle to escape?
    What pipes and timbrels? What wild ecstasy?

Heard melodies are sweet, but those unheard
  Are sweeter; therefore, ye soft pipes, play on;
Not to the sensual ear, but, more endear'd,
  Pipe to the spirit ditties of no tone:
Fair youth, beneath the trees, thou canst not leave
  Thy song, nor ever can those trees be bare;

    Bold Lover, never, never canst thou kiss,
Though winning near the goal yet, do not grieve;
  She cannot fade, though thou hast not thy bliss,
    For ever wilt thou love, and she be fair!
Ah, happy, happy boughs! that cannot shed
      Your leaves, nor ever bid the Spring adieu;
And, happy melodist, unwearied,
      For ever piping songs for ever new;

More happy love! more happy, happy love!
      For ever warm and still to be enjoy'd,
       For ever panting, and for ever young;
All breathing human passion far above,
      That leaves a heart high-sorrowful and cloy'd,
       A burning forehead, and a parching tongue.

Who are these coming to the sacrifice?
      To what green altar, O mysterious priest,
Lead'st thou that heifer lowing at the skies,
      And all her silken flanks with garlands drest?

What little town by river or sea shore,
      Or mountain-built with peaceful citadel,
       Is emptied of this folk, this pious morn?
And, little town, thy streets for evermore
      Will silent be; and not a soul to tell
       Why thou art desolate, can e'er return.
       
O Attic shape! Fair attitude! with brede
      Of marble men and maidens overwrought,
With forest branches and the trodden weed;
      Thou, silent form, dost tease us out of thought
As doth eternity: Cold Pastoral!
      When old age shall this generation waste,
       Thou shalt remain, in midst of other woe
Than ours, a friend to man, to whom thou say'st,
      "Beauty is truth, truth beauty,—that is all
       Ye know on earth, and all ye need to know."

</div>

I find time fascinating because its so intrinsic to our nature as humans; if we were to not exist within the stream of time we would hardly be called such. Yet I can also appreciate that there exists a perspective that is outside its bounds."

Polymorphic Elements in Astro

Wed, 22 Feb 2023 00:00:00 GMT

I'm building a component library in Astro, and one of the things I needed to do was to build a component that could render as either a button or an anchor tag. Here's an example of the outcome I wanted to achieve:


 // renders an <a> tag
<Button href="https://some-link">This is an anchor tag</Button>
 // renders a <button> tag
<Button type="button">Submit</Button>

With Astro you're writing the best parts of JSX there could be; for example, there are no abstraction leaks like having to add `key`s to iterables. But also what you're writing is very obviously much closer to HTML than JSX is, there not being a virtual DOM, or styled components, so this puzzled me for a while. I could either have 2 components and duplicate my styles, or find a way to render either depending on the use case.

Thankfully with some clever prop handling, we can have the best of both worlds:

// button.astro

const { href } = Astro.props

---
<>
  {
    href ? (
      <a href={href}><slot /></a>
    ) : (
      <button
        type="button"
      >
        <slot />
      </button>
    )
  }
</>

With this code, the default output is a <button> element, but if you pass a href, it will instead render an <a> tag. What I love about this is that we are also using another trick first implemented by React, the Fragment. By wrapping the component in fragments (empty tags, or <> </>), you're still returning one element, which is a requirement for Astro components, but eliminating the wrapper in the compiled code. This way you don't have an extra `div` which could cause styling issues later on.

But what if you wanted more polymorphic elements? Say you had a container that could sometimes be a section, and sometimes you'd want to render it as a plain ol' div element. Here's a solution for that:

---
interface Props {
  as?: "section";
}

const { as } = Astro.props;

---

<>
  {as === 'section' && (
      <section>
        <slot />
      </section>
  )}
  {!as && (
    <div>
      <slot />
    </div>
  )}
</>

By specifying the types, we give whoever is using this component the hints to show them what elements they can render this component to.

It's nicely extensible too: if you wanted to have another tag, say an <aside>, you could just add that to the types and to the return:

<>
  {as === 'aside' && (
      <aside>
        <slot />
      </aside>
  )}
  {as === 'section' && (
      <section class:list={classList}>
        <slot />
      </section>
  )}
  {!as && (
    <div>
      <slot />
    </div>
  )}
</>

I really like this because the person using this code doesn't have to worry about polymorphism at all, the default return is a div, which they don't have to specify. But what about passing styles to all of these different elements?

Styling polymorphic elements

For my component I wanted to pass in a default set of Tailwind classes, but also allow the person using the component to be able to override them by passing in an arbitrary string. So I used Astro's class:list helper:

const { overrideClasses, as } = Astro.props;

const classList = new Set([
  "tw-mx-auto tw-max-w-7xl tw-px-2 sm:px-6 tw-lg:px-8",
  overrideClasses,
]);
---
<>
  {as === 'section' && (
      <section class:list={classList}>
        <slot />
      </section>
  )}
  {!as && (
    <div class:list={classList}>
      <slot />
    </div>
  )}
</>

By using polymorphic elements I'm keeping code dry, making it reusable, and providing the best possible experience for my colleagues."

Project Documentation Templates

Sun, 01 Nov 2015 00:00:00 GMT

Writing documentation for new team members - or just to provide a referenceable standard - is hard. I've provided these documents with the aim of helping this process.

I'll be honest, I'm not naturally an organised person. But I think that gives me an advantage. It means I have to try harder to be organised. And because I hate the chaos that can result, I'm pretty determined to get things organised and keep them that way.

Consequences #

I suppose I don't really have to revisit what happens when a project is disorganised, do I? We've all been there, and have perhaps been guilty of it ourselves at one time or another. But it has some pretty drastic effects:

Loss of profit
Developer disillusionment
Project can fail to meet client expectations

All three of these can have a really bad effect on the atmosphere in any team or organisation.

Help is Available! #

I've just set up a repo of documents that I helped develop whilst I was at Blaze Communication. These practices allowed us to increase profitability, better meet client expectations, and they certainly helped developers feel more informed, empowered and invested in their projects.

Contractor Guidelines #

The Contractor Guidelines document sets out some principles for contractors. It was chiefly written for off-site developers. I have tried to avoid stipulating rules where possible but set down some best practices — "guidelines" that developers can hopefully use to work smarter instead of harder.

GitHub Link to Contractor Guidelines

Prelaunch Checklist #

It's possibly the worst feeling in the world when a project has gone live and you realise you missed that key security feature, or forgot to activate this piece of functionality. It's even worse to get a call from the client several weeks down the line asking for analytics data ... then realising you forgot to add the code snippet.

This Prelaunch Checklist, from an original by friend and colleague Adam Maltpress can be used as a template for your projects.

GitHb Link to Prelaunch Checklist

Technical Manual #

When I was leaving Blaze I wanted to leave behind more knowledge to the replacement developer than I had the opportunity to say in a handover meeting. This "Technical Manual" sets out some broad strokes for a new developer, and can be used as a handbook for new developers to refer to.

GitHb Link to Technical Manual

Project Scope #

This is the most barebones of templates, and shouldn't be too extensive. It should only outline some key deliverables, specifications and user stories that have been discussed and agreed by the client and the development team. As well as affording deniability, this can help give more direction and focus to a project.

GitHb Link to Project Scope

Missing Stuff? #"

Publishing both JS and non-JS sites

Fri, 18 Sep 2020 00:00:00 GMT

It's always bothered me that the majority of internet users spend a lot of money downloading and running JavaScript, yet I enjoy building things with JavaScript, and want to provide an enhanced experience using JS without sacrificing their needs to do so. Here's one way we can have the best of both worlds.

I'm not going to argue JS vs no JS use here, I don't think it's even an argument. In my opinion we need to provide the best experience we can for all of our users, and that includes both those on slow connections and those on fast ones.

Now, there's an easier way to do both.

Why this site? #

I don't really need any JavaScript on this site. But I used GatsbyJS to build it, which by generates HTML but then overlays that with a React app (I already switched from React to Preact for better performance benefits). I had a site search and an animation, which wasn't part of the core experience but were nice enhancements.

So I decided that I should by default provide an experience which was friendlier to those with CPU or battery restrictions, and then have another domain which had the fancy stuff.

Setup using Gatsby Plugins #

I used two plugins to render an HTML & CSS site with Gatsby: gatsby-plugin-no-javascript and gatsby-plugin-no-javascript-utils. The first builds the site as normal, but then unlinks the JavaScript, so that all you get is the HTML version of the site that Gatsby already builds. The second allows you to do some other finessing so that you can disable inline styles and remove sourcemaps too.

After installing and configuring these plugins, the next step was to setup an environment variable:

In a file called .env in the root of the project, I added:

JS_DISABLED=false

then in my gatsby-config.js I access this variable by adding the following to the top of the file:

require("dotenv").config()

However, I wanted to keep everything else about the configurations the same. Since the export in this file is a single JavaScript object, I could compose it up from separate elements. I could have one array for the plugins, another array for my noJS plugins, like this:

const defaultPlugins = [
  // everything else
];

const noJsPlugins = [
  ...defaultPlugins,
  `gatsby-plugin-no-javascript`,
  {
    resolve: `gatsby-plugin-no-javascript-utils`,
    options: {
      removeGeneratorTag: false,
      noInlineStyles: true,
    },
  },
];

And now combine them into one object, conditionally choosing which array of plugins to choose, and export it as a module:

module.exports = {
  siteMetadata: {
    pathPrefix: "/",
    title: siteTitle,
    siteUrl: `https://deliciousreverie.co.uk`,
    description: `blog of developer & bookworm benjamin read`,
  },
  plugins:
    process.env.JS_DISABLED === "true" ? [...noJsPlugins] : [...defaultPlugins],
};

Setting up on Netlify #

Netlify is very smart. I only needed to do a few things: I initiated a new site and chose the same base repo. I had to make sure I was using Netlify's DNS so that I could use my subdomain with the new site. Then I set the environment variables on both sites, merged the code and it was live!

Updating the UI #

When I checked the noJS version of my site, the search bar was still visible, although it didn't work. So I had to stop it from showing on the noJS version.

I tried using process.env.JS_DISABLED in the JSX, but Gatsby separates environment variables. If I wanted to access this variable on the frontend, I could rename it GATSBY_JS_DISABLED, but I thought of another way of doing it that proved just as effective...

{typeof window !== 'undefined' && <SearchForm />}

Now, the JSX that gets built on the server doesn't render the searchform because there's no Window object. Josh W Comeau has a more robust way of handling this use case on his blog post

I used this again to display a message in the footer to let users know which version of the site they are on:

{
  typeof window === "undefined" ? (
    <p>
      You're currently on the <i>javascript disabled</i> version of the site. To
      enable the site search and some pretty animations,{" "}
      <a href="https://jsenabled.deliciousreverie.co.uk">
        view the javascript enabled react app
      </a>
      .
    </p>
  ) : (
    <p>
      You're currently on the <i>javascript enabled</i> version of the site. if
      you need to conserve your battery or CPU,{" "}
      <a href="https://deliciousreverie.co.uk">view the HTML & CSS only site</a>
      .
    </p>
  );
}

I used this method because I wanted to toggle between elements that were inside a single parent element.

Discouraging search engines #

Finally, I needed to discourage robots from indexing the JS enabled version of the site: the last thing I need is duplicate content warnings or to hurt my own SEO.

I could have used the same trick I've already used above to add this meta tag to the Layout file, but I opted instead to use Netlify's postprocessing tool to add it to the appropriate site:

<meta name="robots" content="noindex" />

I thought this would be more robust since the first pass robots make is done without JavaScript enabled, so it needed to be added to the pre-rendered HTML that's under the React app.

Conclusion #

What's stopping us building everything like this? It didn't take long to set up, and with some UI tweaks I got the best of both worlds: a fancy site with all the whistles and bangs of a JavaScript app, and a barebones (but still pretty) site that doesn't cost the earth to run.

Post-Script: "This should be done by [browser, hosting platform, a serverless function]" #

One of the things I hear often is that we developers shouldn't be responsible for this kind of thing, that it should be the responsibility of some other part of the infrastructure of the web to handle request loads, but I don't think that's the case.

Here's a few reasons I don't think it can be the responsibility any of those things:

Browser? #

Browsers start fetching small packets of information, then scale up until they've expended the bandwidth. It's only once they reach that point that they can tell what the users' connection is like. This is probably too late to decide what to send down the wire.

Hosting platform? #

Hosting platforms don't have the information required to know in detail what the users' connection is like. At best this is an approximation. Could we implement an API so that we can get that data? Maybe, but take a look at what happened with the Battery status API. We don't want to go there again.

Serverless function? #

Hmm ... maybe ... but see above. Potentially an edge function could do it ... though I don't know enough about this to see how.

Nobody? #

Erm no. Someone has to take responsibility here, and it's not the user. Since we've exhausted all other options, I think that we must rise to the challenge and provide people the online experience ... not that they need, but that they deserve."

Publishing Poetry with Webiny CMS and Astro

Fri, 10 Mar 2023 00:00:00 GMT

Publishing poetry on the web is ... tricky.

I admit, some poetry is relatively straightforward: it's organised similarly to normal text, where on the left side, everything lines up nicely to the left, and right lines are "ragged", or not justified to any alignment. This is quite commonly how paragraphs appear.

Some poems, and some poets, choose to organise their work in more ... unusual ways. Take Keat's Ode on a Grecian Urn, which has meaningful indentations which I think really helps you to appreciate the lyrical cadence of the poem:

<div class="[&>pre]:ml-0! [&>pre]:mr-0! [&>pre]:bg-transparent!">

Thou still unravish'd bride of quietness,
       Thou foster-child of silence and slow time,
Sylvan historian, who canst thus express
       A flowery tale more sweetly than our rhyme:
What leaf-fring'd legend haunts about thy shape
       Of deities or mortals, or of both,
               In Tempe or the dales of Arcady?
       What men or gods are these? What maidens loth?
What mad pursuit? What struggle to escape?
               What pipes and timbrels? What wild ecstasy?

</div>

There are much more ... adventurous, shall we say? layouts for poems, but this was the main use case I wanted to meet: lines which respect the indentation of the author. But there was something else I wanted to achieve as well:

One of the things I hate seeing, in printed poetry as well as electronic, is wrapping lines. My reasoning is this: if the author so carefully indented their poem, and broke lines in certain specific places, then that is as important to the work as the left indentation is.

I really hate it when because of the limitations in width or type size, the authors' poems are butchered in this way. It's not as bad as justifying everything centrally (that is a crime I will never bring myself to forgive), but at least on the web we do have the opportunity to scroll left and right if the need arises. Looking at my 2 requirements I realised that there is another common written format that satisfies both of these needs.

A sudden moment of clarity

So to recap my requirements were:

1. No wrapping lines unless the author intended it
2. Indentation as the author intended

I realised that the way we represent code is quite similar. Along with indented lines that don't wrap, we also have syntax hilighting, so it's easier to see what the code was doing. But I realised that was simply an implementation detail, and I could use some things that I already had in order to render poetry correctly on my blog.

The Backend

I use Webiny, the Serverless CMS, for the backend. I'd already configured an Editor.js extension to render code blocks. I forked this code and added to my Webiny admin app's HeadlessCMS config (full PR here), and after a few minor modifications I had the ability to add a block of poetry:

So far so good. I don't mind that the display looks like a code block, for me it's a minor consideration and actually the monospace helps me to see where the indentation occurs.

Rendering on the Frontend

Now that I have my code block coming through with an appropriate field type identifier, I could modify my frontend Astro app to render blocks of poetry as part of the rich text renderer, which I'd already built.

      {field.type === 'poetry' && (
        <RenderPoetry poetry={field.data.poetry} />
      )}

I admit this solution was much hackier than it needed to be, but it works for now. I hope I will have time to create a proper theme for Shiki so I don't have to forcibly override the inline CSS with some !important declarations.

You can see both poetry blocks and code rendered in this article. And I hope it'll mean I'm finally more free to add more articles about poetry here. Time, and children, willing of course!"

A recipe for quick and dirty ramen

Tue, 28 Jan 2020 00:00:00 GMT

I really like asian food. I'll go for gyoza / jiaozi any day over a "traditional" sunday roast. One thing I love most is ramen, but it's taken me ages to invent my own recipe.

So this is a very quick and loose recipe, please don't stick to it, I'm sure it can be improved upon!

Ingredients #

1 pack miso soup (one of the paste varieties are best)
Soy sauce (light or dark, I can't decide)
Small amount of sesame oil
1 spring onion, finely chopped
I egg, fried or boiled
A little meat (optional), pork or chicken
Small quantity of chopped pepper
Pinch of garlic (to taste)
Small quantity of ginger (to taste)
dried chilli
Rice noodles
Boiled water

Method #

Put the noodle into a bowl. Pour in the boiled water. Whilst they're getting ready, fry your egg. I like to keep it a little runny, so it mixes into the soup a little.

You could fry the other ingredients (except the spring onion) together beforehand, but that's just a little nicer if you have time. Otherwise, throw them into the hot water.

Once all the ingredients are in, add the spring onion.

If the water has cooled down or the noodles haven't absorbed water enough, put the bowl in a microwave for a minute or so.

Add the egg, and serve with chopsticks.

Variations #

I've sometimes used mackerel fried in Lao Gan Ma's crunchy chilli paste, instead of the meat. That stuff is awesome and goes with almost anything."

React Accessibility

Mon, 24 Dec 2018 00:00:00 GMT

Originally published in print for Net Magazine.

In the recent past, I've heard React being criticised for not being accessible, or for allowing practices that don't help developers think about accessibility.

I have to disagree.

Just like any other frontend code, working with React in an accessible way requires some forethought.

One of the common mistakes I've seen is that we sometimes put click events on non-semantic elements. It's true, React makes it very easy to put a click event on any dom element.

I think one of the reasons for this is that with React, our JavaScript is our single source of truth, so we tend to think in JavaScript. One DOM element is like any other DOM element, and they all can be manipulated in much the same way.

However, if we're thinking about accessibility, we're committing ourselves before we start coding, that we're going to do things the right way, including using semantic elements. If we do this, we'll ensure we assign a click event to a semantic element to start with - a <button> or an <a> tag - or we'll do all the extra legwork that's required to make up for it.

Encourage Accessibility in Components #

Another way to encourage others using your components to think in an accessible way is to compose components to include accessibility features. For example, this component passes down a prop that's an image source.

<img src={this.prop.imgurl} />

What are we missing here? The alt attribute. How do we encourage accessibility in this instance? Wrap the component in a conditional, so that it doesn't render without the alt tag:

{this.props.imgalt ? (    
    <img src={this.prop.imgurl} alt={this.props.imgalt} />  
) : null}

You could go a bit further with this, and check that the length of imgaltis greater than 0.

It Starts & Ends With Accessibility #

Just as our thinking process starts with the commitment to code with accessibility in mind, our launched product should finish with accessibility. So test your assumptions:- did you really build an accessible site or app?

Use software to test what you can. But better than that: connect with people you know. Take your product to your friends and relatives or your audience. How do they get on using it?

React has plenty of built-in support for building things in an accessible way. Check out the documentation at https://reactjs.org/docs/accessibility.html.

The question is, will you make that commitment to yourself, and the people who will use your site?"

Build a countdown timer using Temporal with React

Sun, 04 Jan 2026 00:00:00 GMT

As the Temporal API is now stable at Stage 3, and a specification compatible polyfill is available, I thought I would use Temporal to build a countdown timer in a React project.

I had previously build this countdown timer without Temporal, but since the API I was using exposed durations in ISO 8601 compatible strings it meant I was already pulling in another library to handle those. So I thought I might as well utilise the Temporal polyfill instead.

The API is pretty straightforward as you might imagine

import countdown from 'countdown'
import { parse } from 'tinyduration'

const CountdownTimer = ({ startedAt: Date, duration: string }) => {}

You can probably guess where we're going with this, we'll use a useEffect() hook to run a setInterval() which refreshes the calculated remainingTime and sets it in useState(). We need to memoise the calculated ending time to persist across rerenders too:

const [timespan, setTimespan] = useState<Timespan | null>(null)

const parsedDuration = parse(duration)

const {
  years = 0,
  months = 0,
  weeks = 0,
  days = 0,
  hours = 0,
  minutes = 0,
  seconds = 0,
} = parsedDuration

const durationMs =
  years * 365 * 24 * 60 * 60 * 1000 +
  months * 30 * 24 * 60 * 60 * 1000 +
  weeks * 7 * 24 * 60 * 60 * 1000 +
  days * 24 * 60 * 60 * 1000 +
  hours * 60 * 60 * 1000 +
  minutes * 60 * 1000 +
  seconds * 1000

const endTime = useMemo(
  () => new Date(startedAt.getTime() + durationMs),
  [startedAt.getTime(), durationMs],
)

useEffect(() => {
  const updateCountdown = () => {
    const now = new Date()
    const result = countdown(now, endTime) as Timespan
    setTimespan(result)
  }

  updateCountdown()
  const interval = setInterval(updateCountdown, 1000)

  return () => clearInterval(interval)
}, [endTime])

const countdownHours = String(timespan?.hours ?? 0).padStart(2, '0')
const countdownMinutes = String(timespan?.minutes ?? 0).padStart(2, '0')
const countdownSeconds = String(timespan?.seconds ?? 0).padStart(2, '0')

return (
  <time 
    dateTime={`PT${countdownHours}H${countdownMinutes}M${countdownSeconds}S`} 
  >
    {countdownHours}:{countdownMinutes}:{countdownSeconds}
  </time>
)

If this seems a little verbose, it is. tinyduration only returns units of time if the parsed data has any, meaning you have to calculate around it. Also the library is not typed, meaning I had to implement my own.

If you're reading this you're probably familiar with the countdown library. In all honesty it doesn't do much at all apart from provide a duration until the end time you provide it.

This can all be achieved much more succinctly in the Temporal API.

Switching to Temporal

The first change is to the types. Instead of a string or my own implementation of the result of the tinyduration library, I can use a type from Temporal:

const [remainingTime, setRemainingTime] = useState<Temporal.Duration | null>(
  null,
)

The endTime memoised calculation is fairly similar, only using Temporal is a bit more clear about what's happening:

const endTime = useMemo(() => {
  const startInstant = Temporal.Instant.fromEpochMilliseconds(
    startedAt.getTime(),
  )
  const durationObj = Temporal.Duration.from(duration)
  const totalMs = durationObj.total({ unit: 'millisecond' })
  return startInstant.add({ milliseconds: totalMs })
}, [startedAt, duration])

The first change here is to convert the start time to a Temporal Instant object.

Temporal.Duration natively supports the ISO 8601 format, therefore calculating the total is a matter of converting the duration to milliseconds, and then using .add() to calculate the sum.

So although endTime itself slightly more verbose, we didn't have all the setup and its easier to see what's going on. I much prefer my code that way since it makes it much easier for other developers to pick up later on.

Now for the useEffect() hook:

useEffect(() => {
  const updateCountdown = () => {
    const now = Temporal.Now.instant()

    const remaining = now.until(endTime, {
      largestUnit: 'hour',
      smallestUnit: 'second',
      roundingMode: 'floor',
    })

    setRemainingTime(remaining)
  }

  updateCountdown()
  const interval = setInterval(updateCountdown, 1000)

  return () => clearInterval(interval)
}, [endTime])

This is much clearer as well: we instantiate a new Temporal Instant then use the .until() utility to represent the duration from the current instant until the end time.

The second argument allows us to format the remaining duration in place.

The final step is formatting and the render function:

const countdownHours = Math.max(0, remainingTime?.hours ?? 0)
const countdownMinutes = Math.max(0, remainingTime?.minutes ?? 0)
const countdownSeconds = Math.max(0, remainingTime?.seconds ?? 0)

return (
  <time
    dateTime={`PT${countdownHours}H${countdownMinutes}M${countdownSeconds}S`}
  >
    {String(countdownHours).padStart(2, '0')}:
    {String(countdownMinutes).padStart(2, '0')}:
    {String(countdownSeconds).padStart(2, '0')}
  </time>
)

Temporal doesn't provide formatting operations suitable for this situation so we need padStart() and then Math.max() to ensure we don't go into negative values.

I'm using the raw data to set the datetime attribute as ISO8601 formatted string, then formatting it again to render the content of the element.

Wrapping up

The function is now much shorter (56 to 83 lines) but that isn't the biggest win here.I'm only using one library, and that's a polyfill which can be removed eventually.

It's also much clearer. We're not using mathematical operations on milliseconds, instead treating times as their own separate entities. And I think the code is much more readable too.

My only concern with this component is it doesn't announce to screen reader users when the time counts down. However we wouldn't want to annoy the user more by announcing every second.

An improvement could be that we add logic to announce the time left at meaningful intervals such as "10 minutes left", "1 minute left" etc.

All in all, the Temporal API is going to improve JavaScript a huge amount. I'm really looking forward to seeing how the landscape changes over the next few years.

React: data fetching results in type error: object is not a function

Mon, 12 Jul 2021 00:00:00 GMT

Sometimes I get frustrated with React, there can still sometimes be obscure errors that are a result of it's component lifecycle methods. But at least it means I can tell you about how to avoid them ...

Take a look at this code a second. There's nothing wrong with the fetch() resolution, the API works fine. There's nothing wrong with the render method either.

But if you try to run this code you'll get some error like Error: Objects are not valid as a React child (found: [object Promise]).

async function getFacts() {
  const response = await fetch("https://cat-fact.herokuapp.com/facts/random?amount=10")
  if (!response.ok) {
    throw new Error(‘error’)
  }
  const facts = await response.json()
  return facts
}
export default async function Home() {
  const facts = await getFacts();
  return (
    <div>
    {facts && facts.map(fact => (
      <div key={fact.id}>test</div>
    ))}
    </div>
  )
}

The problem is that the data you fetched isn't consumed by the components' lifecycle. Yet there's nothing to tell you that in the error. Here's what you need to do to resolve it:

import { useEffect, useState } from 'react'

async function getFacts() {
  const response = await fetch("https://cat-fact.herokuapp.com/facts/random?amount=10")
  if (!response.ok) {
    throw new Error(‘error’)
  }
  const facts = await response.json()
  return facts
}
export default async function Home() {
  const [facts, setFacts] = useState()

  useEffect(() => {
      getFacts().then(facts => setFacts(facts))
  }, [])

  return (
    <div>
    {facts && facts.map(fact => (
      <div key={fact.id}>test</div>
    ))}
    </div>
  )
}

You see that we imported the useEffect and useState hooks, then when the component mounts, we fetch the data, and pass it to state using setFacts.

Now, once the data has been retrieved, you won't get that error any more.

It's a simple fix for an issue that wouldn't have occurred except for the fact that we're in React, and we need to respect the React component lifecycle, and it's way of handling data. We can't fetch data and assume we can just use it.

When in Rome ..."

A React performance case study

Wed, 12 Dec 2018 00:00:00 GMT

Part of the way through a project to replace an app with a static site generated by Gatsbyjs, initial tests revealed that we there was a lot we could do to improve the performance. Here's how we turned the project around, cutting load times to under 3 seconds and reducing percieved performance bottlenecks.

Here's what the initial test looked like in web page test (webpagetest.org):

Identifying the Problem #

The initial issue we had was how do we identify what problems we had that we needed to solve. Without a clear understanding of what was going wrong, we had no way of focusing on tasks that could help us improve the current situation.

One of the first things we did was to find out how our site was rendering across different browsers. Straight away, we noticed that browers that didn't have the benefit of a fast JavaScript parser were significantly slower. Sometimes it would take up to about 14 seconds to render a page. Interacting with key conversion-linked elements on the page was janky or stuttered a lot, which further put users off.

We also had a popup module for cookies which took over a large part of the screen, and which couldn't be dismissed until all of the JS was downloaded and parsed. This was a major issue to overcome.

Over later iterations we also implemented a few other strategies to help us pinpoint issues:

Webpagetest #

Web page test has got better & better over the years. It provides a teriffic overview as well as a deep dive into the waterfall and has tonnes of settings for different locations and devices.

It's still my favourite performance tool.

Lighthouse #

I use the Lighthouse plugin for Google Chrome (instead of the native version) because it contains newer features that we wanted to incorporate into our results. Lighthouse is typically better for pinpointing different overall issues such as PWA status and accessibility.

HotJar #

I was skeptical about using HotJar to begin with. I felt it would add to our JavaScript burden instead of reduce it. However, we managed to integrate HotJar with our analytics tracking script, reducing the overall load. The script was loaded asynchronously which helped. However, downloading and parsing still took up processing power, reducing the interactivity of the site until it was completed. So reducing the overall JS burden was still a necessity.

HotJar later became instrumental in helping us identify a severe issue which resulted in a white screen, so it became really useful in our testing and iterating process.

Splunk #

Splunk allows you to collect errors in your application and log them for later use offsite. I particularly love Splunk, because it allowed us to send any JavaScript errors we wanted that occured on the client off to be examined and quantified later. Once we had implemented Splunk (using React's ErrorBoundary API) we could see what was really going on with our app in the wild.

Having set up these two, we were ready to dive into prioritising and fixing some of the significant issues we were seeing.

Issue 1: Large JavaScript Burden #

Even though GatsbyJS does a huge amount to reduce the amount of JavaScript (rendering what it can on the server, tree shaking and minification to name a few) We had 2 issues with our JavaScript:

1. Pulling Too Much onto the Frontend #

GatsbyJS' compiler looks for code that includes references to the windowobject and renders those on the client. The rest, it renders on the server. We had a lot of extra JavaScript, including page configuration objects, that ran on the frontend instead of using the Gatsbyjs' data layer.

By refactoring back into gatsby, we were able to reduce our bundle size.

We also noticed that somehow larger libraries like Lodash were used to render client-side content. By refactoring this library out of what was rendered on the client side, we further streamlined it.

2. Using WebPack 3 #

We moved the site from GatsbyJS v1 to v2 (it took a day to complete). this led to a reduction in JS bundle size from about 700kb to around 530kb.

Issue 2: JavaScript Caching #

Using Webpack 3 was a huge step forward. Previously, in Gatsby v1, we could cache HTML really well. But page components couldn't be cached. Webpack 3 allowed us to cache JS more aggresively, leading to much better performance.

After the work with Webpack we saw the following results:

Issue 3: Obtrusive Cookie Module #

As I mentioned above, we noticed that on mid-range phones where users didn't use a fast JavaScript parser, it could take up to 14s for the site to be interactive. Users could still get some functionality and scroll around ... however, they were prevented from doing so by the cookie module. This was a particular issue on smaller screens but affected everyone.

To combat the issue we redesigned the module to be much smaller. We also set a timeout on the module so that it wouldn't try to check for cookies or even render until after the dom was loaded.

As a result, we didn't stop people from using the site until the cookie module was dismissed.

Issue 4: Large Dataset #

On a high-converting page we made a call on the frontend to get a large dataset that was over 1.3MB. This was a stop-gap: we ultimately needed to get this dataset not from a static JSON file but from a server that could take 35ms to compile the data we needed and make it available as a JSON object.

Instead of querying this API directly we built an intermediary service that cached the data set we needed as a static JSON file. The API also reduced the data down by eliminating extra data that we didn't use.

This reduced our file to around 650KB.

Issue 5: Long-running API Calls #

The data we requested on the API endpoint mentioned above was mission-critical. It had to be available to our users, otherwise conversions would not be possible. Therefore we hadn't set a timeout on the API call.

We were able to find a workaround to this issue by caching the data in the project on build. That allowed us to set a timeout so that if the connection dropped suddenly, a default data set that was fetched on build earlier could be displayed.

Issue 6: Large Number of JavaScript Math Functions #

Having an intermediary service allowed us to further reduce the JavaScript on the site.

The large dataset contained raw numbers down do several decimal places. But we only needed to provide 2 decimal places to visitors. Although it reduced the adaptability of the intermediary service we had built, we decided to perform rounding there instead. This freed up much more processing power for the main thread and further reduced times.

Issue 7: Avoiding Re-rendering #

I'm going to cover this more in-depth in another post, but we also managed to reduce page re-renders massively. We did this using the Context API to share data across several modules via State. Most of our functions were used in State as well. This meant that only the components consuming that data re-rendered when they were used.

Conclusion #

A recent performance test shows we have achieved the following:

I recognise these performance results are under optimal conditions and actual, not to mention percieved, results aren't captured by these figures. But we also need some achievable goals that help us to stay motivated and focused.

We haven't finished yet either ... more work will continue to be done during the lifetime of this project.

This has been a great adventure, and it's been exciting to see how we've gone to fully-loaded times of around 10s to 4, then down to 2.5, where it currently stands. I hope this post gives you some clues about how you can use new tricks and old to make your apps more performant.

Here's what the initial test looked like in web page test (webpagetest.org):

Doc-CompleteFully--LoadedLoad timeFirst ByteStart RenderSpeed IndexFirst InteractiveTimeRequestsBytes InTimeRequestsBytes InCostFirst view4.733s0.901s2.1002266> 8.179s4.733s211,536kb8.055s232.691kb$$$$

Identifying the Problem #

We also had a popup module for cookies which took over a large part of the screen, and which couldn't be dismissed until all of the JS was downloaded and parsed. This was a major issue to overcome.

Over later iterations we also implemented a few other strategies to help us pinpoint issues:

Webpagetest #

Web page test has got better & better over the years. It provides a teriffic overview as well as a deep dive into the waterfall and has tonnes of settings for different locations and devices.

It's still my favourite performance tool.

Lighthouse #

HotJar #

HotJar later became instrumental in helping us identify a severe issue which resulted in a white screen, so it became really useful in our testing and iterating process.

Splunk #

Having set up these two, we were ready to dive into prioritising and fixing some of the significant issues we were seeing.

Issue 1: Large JavaScript Burden #

Even though GatsbyJS does a huge amount to reduce the amount of JavaScript (rendering what it can on the server, tree shaking and minification to name a few) We had 2 issues with our JavaScript:

1. Pulling Too Much onto the Frontend #

By refactoring back into gatsby, we were able to reduce our bundle size.

We also noticed that somehow larger libraries like Lodash were used to render client-side content. By refactoring this library out of what was rendered on the client side, we further streamlined it.

2. Using WebPack 3 #

We moved the site from GatsbyJS v1 to v2 (it took a day to complete). this led to a reduction in JS bundle size from about 700kb to around 530kb.

Issue 2: JavaScript Caching #

After the work with Webpack we saw the following results:

Doc-CompleteFully--LoadedLoad timeFirst ByteStart RenderSpeed IndexFirst InteractiveTimeRequestsBytes InTimeRequestsBytes InCostFirst view3.993s0.344s1.3002098> 4.663s3.993s21740kb5.609s331,069kb$$

Issue 3: Obtrusive Cookie Module #

To combat the issue we redesigned the module to be much smaller. We also set a timeout on the module so that it wouldn't try to check for cookies or even render until after the dom was loaded.

As a result, we didn't stop people from using the site until the cookie module was dismissed.

Issue 4: Large Dataset #

This reduced our file to around 650KB.

Issue 5: Long-running API Calls #

Issue 6: Large Number of JavaScript Math Functions #

Having an intermediary service allowed us to further reduce the JavaScript on the site.

Issue 7: Avoiding Re-rendering #

Conclusion #

A recent performance test shows we have achieved the following:

Doc-CompleteFully--LoadedLoad timeFirst ByteStart RenderSpeed IndexFirst InteractiveTimeRequestsBytes InTimeRequestsBytes InCostFirst view1.787s0.223s1.2001.384s> 2.698s1.787s21301kb2.571s33438kbq$

We haven't finished yet either ... more work will continue to be done during the lifetime of this project.

Reading micro expressions

Thu, 07 Aug 2025 00:00:00 GMT

I find it fascinating that in spite of many advances in modern psychology certain myths and downright errors still persist. We've become wonderfully aware of neurodiversity and the abilities and limitations that has placed on individuals. Yet there is still much to understand about our fellow humans.

Micro expressions are one of these, and learning to notice them has helped me to understand people, draw out ideas and comprehend situations more fully numerous times in recent years.

What are micro expressions?

It's funny, in researching this article I could find a number of articles that help identify the emotions you might see in micro expressions, but not what they are or what opportunities they open up.

Micro expressions occur when you're talking to someone, perhaps you ask them a question or they reach an important juncture in the conversation. There might be a very short pause, a very slight change in the persons demeanour, and then it's gone.

That change can be for the briefest of moments. Literally milliseconds, just enough to be noticeable. I quite often notice them when someone else I'm with is doing the talking. Suddenly I'll see the person glance my way, or at the floor, or even skywards for a fraction of a second, and then resume the conversation.

As I said above, a lot has been done to identify the kinds of emotion a person might be experiencing when these happen. However I've learned a pattern of response that is broader, and which can foster openness in communication.

How to utilise the opportunity

When I notice someone displaying a micro expression my immediate thought is that in some way, I'm not hearing the whole story. The person might be projecting; it could be that they're telling me something because they think I want to hear it. Or because they're not emotionally ready to tell me about some detail of it yet. Or more seriously because they're concealing something that they don't want me to know for some other reason.

This is an opportunity for further discussion, and a lot could be lost by ignoring it or pretending it didn't happen.

I typically try to ask a viewpoint question, to try to draw out the person and help them explore the subject a bit more. For example, if we're talking about a conversation they've previously had, I might say to them, "is that all they said?" because it might be a good way of opening the door to further understanding.

What not to do

We mustn't fixate on micro expressions. Calling someone out because you've noticed they've displayed one is only going to shut them down, and could very quickly erode your relationship with them.

Instead use them as a platform for further understanding, and your relationship with them, more deeply.

If they're concealing something, it might be good to think of why. Are we a person in their life they don't want to open up to fully? How could we improve that relationship?

Concluding Thoughts

There's a lot more resource on this subject out there, so if you're interested in this sort of thing, you can go and look it up.

Also, am I going to regret diluting my blog's SEO by adding another, entirely new subject to it?

No.

Do I care about SEO?

Also no.

Lessons from refactoring a large dashboard application

Sun, 16 Nov 2025 00:00:00 GMT

Over the past few months I've been refactoring a large application written in vanilla JavaScript. I learned a few things from doing it, and thought they might help someone else.

Lesson 1: Don't do it all at once

One of the first things I did was to add bundling so I can type check and set up a test suite. I also moved to ECMAScript modules which would also help me split my JavaScript into smaller modules later on.

Of course, once you add things like that it becomes apparent that there are a lot of changes you need to make.

That first PR I made quickly led to over 300 files changed. That's going to be extremely difficult for someone to review.

After this we decided to make a feature branch. Then I would incrementally branch off that and open PRs against the feature.

This goes a bit against the grain, I dislike trunk-based development. However for a project this large and a rewrite this extensive, it was the only practical way it could be segmented and reviewed.

Having a feature branch allowed me to reduce the change size. I'm not saying it was always manageable for the code reviewers, but it was at least manageable on some level.

Lesson 2: Don't be afraid to be brutal

I had a clear philosophy in mind with this project: it should be modular.

It seems the original concept for the project was built that way: it consisted of a series of components that would render charts, table data and other statistics (text percentages, counters and date information).

However this philosophy seemed to have been abandoned in favour of a single, massive function that was responsible for fetching all of the data, transforming it, then passing it to components.

I could see the intent of this function: the developer wanted to reduce requests that were made to the API.

What was less clear was what data went where in the application. In fact, sometimes it was impossible to figure it out.

All data resulting from the API requests were first transformed into other structures, then channeled through a CustomEvent that bundled up the resulting data into an array of Promises which was then attached to the event.data.

This made it very difficult to see on a component level what was API data, what was modified data, and even where it had come from.

But the core concept, a CustomEvent, which could be triggered by a user input, I thought was a good way of handling state refreshes, so I kept that.

I made each of my components responsible for their own data, and implemented two things on the fetch that would reduce load on the API: caching in LocalStorage and a queue that would, my means of a key, identify duplicate requests and return data from LocalStorage if they existed there.

This led to deleting a major chunk of the old data fetching strategy. I was so glad to see it go. But it did leave holes in my application that I had to reimplement later.

It was a brutal change but reduced the applications' complexity by an order of magnitude.

Lesson 3: Types are your friends

I had to build all of the types from the ground up for this application. It was a struggle to figure out what on earth was going on, and sometimes even where the types would change in the original application. There were type collisions everywhere, no consistent naming convention, and variables were being overwritten all over the place.

But had I not made the effort to collect those original types it would have been much more challenging to refactor later on. That refactor implemented a different API with significant type changes, which led me to delete all of the types I had collected on this first pass. But without them I think it would have taken longer. At least I could see more easily where the differences were and knew more quickly what to change.

Additionally, previous developers on this project were very fond of using the global scope. They would import a JavaScript library and attach it to the window. This meant doing things like

const chart = new Chart()

Depending on your experience you might not recognise that this is in actual fact

const chart = new window.Chart()

Aha now we can type it.

declare global {
  interface Window {
    Chart: new(ctx: ChartContext, config: ChartConfig) => void
  }
}

What is not fun at all is doing that with JQuery. If you find yourself in that unfortunate position, JQueryStatic is your friend. And don't ask about mocking jQuery for unit tests ... wow that was awful.

Lesson 4: Know when to bend the rules

In the original implementation there were 2 separate codebases, one for a dashboard for individuals, another dashboard for teams.

I can see why this was done: the teams module was somewhat independent of the main dashboard and I guess the original author decided it couldn't be counted on to be present.

However with bundling we could now import from the main codebase and reduce the technical debt by a significant amount.

Many of the components went down from a few hundred lines to an import and a function call that passed down the team data that I needed.

I thought this was a tradeoff that kind of broke the rules of separation of concerns, but due to the fact that we were calling the same APIs, and rendering data with a lot of shared implementations, I judged that it would reduce the work that needed to be done to maintain the modules over time.

Lesson 5: Don't rely on AI to write your tests

I'm going to try not to get salty here. AI did help me get started with a test suite. I managed to get it to mock my data and functions that were called in the implementation, and often it was quite close to what I wanted.

But it never implemented data structures that even remotely matched the implementation. This was bad because the unit tests are documentation. For me one of the main reasons I put a test suite in there was to show other people coming across this project later how the components were being used. If the data structure used in the test isn't representative of the real set, it undermines it's usefulness.

Plus it really wasn't that smart and often I had to debug really difficult issues. Sometimes it even just put code comments around stuff that was already there, as if it thought I was lying or something.

Use AI by all means. But only with careful guardrails. And don't let it loose on your codebase without carefully reviewing what it's changed.

Browsers have a lot of what you need

Even though I implemented bundling I really didn't import that many packages. The main one was a polyfill for the Temporal API since that's not out everywhere yet and I wanted to lean on that instead of implementing Moment or some other large external library.

As this was a vanilla JS application I had to think carefully about state management. I split this between the LocalStorage API and the URL. Using the URL as a state manager is really good by the way. It also meant that if users have a problem later they can share their URL which will give us a lot of information about their dashboard.

Also, utilising CustomEvents was great. I didn't choose to pass any data down using these events, but I used them as triggers: when a user updated some filters, changed dates, or updated their data, I could tie into the save event, do a dispatchEvent(new CustomEvent("refreshDashboardState")), and each of my components, because they were listening for it, would reload nicely:

function renderMyDashboardComponent() {}

window.addEventListener("refreshDashboardState", () => renderMyDashboardComponent())

Simples.

Conclusion

This project has been my focus for a long time and it feels somewhat like the Ship of Theseus now that it's all done.

What hasn't changed ironically is the way users will experience it. Aside from a few helpful updates for developers (caching can skipped by appending a query string to the URL, which will really help our Helpdesk team), it looks the same.

But it uses an entirely new backend data source. And in future, any further change requests (of which we receive many) can be implemented swiftly without me running for the hills.

Reset Webiny State Without Redeploying

Tue, 01 Nov 2022 00:00:00 GMT

It’s not always feasible to destroy your Webiny instance once you’ve started to create entries. If you need to delete entries in the Headless CMS and effectively restore your instance to a previous state, you can follow along here.

This technical documentation involved a deep dive into DynamoDB backup and restore, as well as ElasticSearch (OpenSearch) snapshots.

View the full article"

Reusable Components with Atomic Design

Wed, 09 Jan 2019 00:00:00 GMT

Structuring components in a reusable, discoverable way is a challenge across teams with diverse requirements. It can be difficult for developers unfamiliar with your library. They need to learn quickly how adopt, use and discover, features it has. How do you help them do that without over-burdening your components with too much documentation? I'd like to introduce the idea of Atomic Development using React and styled-components.

Atomic Design is an approach to development that Brad Frost came up with a few years ago. He's published an excellent book about it which you can buy, or read online at https://atomicdesign.bradfrost.com/

This post originally appeared in Net Magazine issue 314 under the title "Put Atomic Design into Practice"

The Atomic Design Approach #

Brad Frost's book outlined a way to structure the design of a project based on building things up from the smallest component part.

For example, this could be an input field. Input fields should never be used on their own. Instead, you would make sure for accessibility you have a label associated with it. In our analogy, the input field would be the Atom, and the atom and label together would be a Molecule.

You could keep building upwards in this way so that you eventually had a fully designed page using not only Atoms and Molecules, but also Organisms (for example, a complete form) and a Template which contained your complete form, and perhaps other Molecules.

Atomic Development #

- components/
  - Atoms
  - Molecules/
  - Organisms/
  - Templates/

Atoms #

Atoms I designated as constants which could receive props. For example, a paragraph that had a prop for the text colour.

Molecules #

Molecules consist of stateless functional components, often comprising of several Atomic elements and can contain some custom Atoms that relate to that specific Molecule.

In the example below, we have a "popout" section, which has a custom MainContainer element that's only used in this Molecule. It can accept multiple props and can wrap other components.

Organisms #

As we get to organisms, we start to see larger elements, bringing in several Molecules and atoms, often with components that render on the frontend as well as server, and data that might come in from an external source.

Templates #

Templates are layouts for pages that are programmatically created from other data sources, such as the one I made below for a side project about my favourite musician:

Atomic at Zopa #

Coincidentally, Gerard Brull, a colleague of mine at Zopa.com who works out of our office in Barcelona had also come across this idea. We discussed it as a team and decided to try to implement it in a future iteration of our shared component library.We hope that it will help our team collaboration and assist the onboarding experience for newer developers that join us.

Modularised Application Development #

At Zopa we build apps and services that start small but often scale up to millions of users. Organizing our code in a way that's easy to maintain is something we must therefore take into careful consideration before we begin development.

Senior Developer Alexis Mangin has shared his experience of what could be the best approach when it comes to scalable applications which could have many component parts:

"Categorizing files based on what they represent is an easy way of partitioning your project and has became a popular practice with developers who use patterns such as MVC. In our experience, that’s okay when you work on small applications, but it can have a tremendous effect on the team’s velocity as the application grows. One way to avoid poor-planning headaches is to structure your codebase around the concept of modules, each with its own responsibility. Creating a module means you will group a set of related components, methods and assets together, providing a public interface to be used by other modules."

Please see this post for the full article."

Sad songs say so much

Fri, 21 Feb 2025 00:00:00 GMT

I enjoy times when I'm gleefully happy. But I also enjoy melancholy, which quite often comes out in my musical tastes. Here are some of my favourite melancholy songs, and why I like them.

I'm not going to include any instrumental songs here, though I could. I find instrumental songs so abstract that what I consider to be melancholic someone else could experience a totally different way. This is also mostly a study of the lyrics of my favourite sad songs.

Honestly, I think people either prefer lyrics or melody in music. For me, lyrics is the main thing, though you can hardly have one without the other.

Sad songs say so much by Elton John

Had to include this, since I lifted the title straight from this song. I love the irony of the line "when all hope is gone / tune in and turn 'em on", like that's going to help at all.

But, as the line goes, misery does like company.

Sweetheart Like You by Bob Dylan

Sweetheart like you is one of my favourite down-and-out songs. I say that because the narrator (it's not Bob, it's never Bob) seems to be in a bad place. I get the idea that the setting is some greasy spoon restaurant on the wrong side of town, and that he is facing a crippling loss of some kind.

The trauma of that kind of loss can sometimes make us reach out and focus on others as a kind of coping mechanism. We don't want to think about ourselves so we invest ourselves in others.

The narrator is trying to give hope to someone else who perhaps is living with a challenging situation not unlike his own albeit with slightly more stability, enough that they can hold down a job at least.

Key West by Bob Dylan

I can't go without mentioning Key West here. In my mind's eye I see the narrator (Bob?) spending his final days looking out to sea from this location. He's singing about finding immortality but in reality knowing it beyond his grasp.

Bob is getting on a bit now, and that makes this song even more poignant.

Aberdaron by Bwncath

The Welsh language music scene is incredible. I have purchased so many tracks and full albums from Welsh artists in the last year, not just as an exercise for language learning, but just because I enjoy them so much. Aberdaron is one I find myself thinking about again and again.

The lead singer's striking vocals really lean into this song, accompanied by the thought provoking lyrics which focus on a time in the narrator's future where they can rest in a homely cottage by the sea, take their ease "beyond all praise and judgement" and gaze out to the rocks of Aberdaron.

The real stinger in the story is that, if you know the town in question, the best view of the rocks is from the graveyard just up from the beach.

Fin

That's it I guess. Thought I had a few more to talk about, but it seems I don't.

The search service landscape

Fri, 28 Aug 2020 00:00:00 GMT

Search is a massive growth market but there is only one market leader and many inadequate contenders. Here's why I think we should be building more search providers to expand the market.

When did you last use a site search tool? What engine powered that tool? I can pretty much guess what most of you would reply. And whilst they are by far the market leaders, I think that's an indication that search as a market is about to explode.

History of search #

Building a basic search implementation was often something that was baked into monolithic apps. For example, WordPress comes with a no-frills search built in.

However we've gotten past monolithic do-everything-in-one-codebase apps now. Search becomes it's own concern, which is better for the search tool, and for the person using it, because we have the option to swap tools that aren't doing their job well enough.

Whilst Apache Solr and Elastic Search are the granddaddies of search, they are in themselves extremely powerful, out-of-the box platforms. I am going to exclude them from the criteria because they're more used for logging, processing and analysing hefty amounts of data such as event logs, application errors and things like that.

The use cases I'm specifically thinking about are site search, app search and searches for online documentation.

The contenders #

Search is an emerging market and not nearly as well developed as the CMS market. But there are a few up and coming options.

1. Algolia #

Algolia is the shining beacon here. It is clear from their implementation that they saw a gap in the market and have exploited it masterfully. They've also got the most unbelievable marketing department, so good that you might be pursuaded to believe there are no contenders for the throne (which is almost true ...)

Pros: Lovely analytics dashboard, delightful SDKCons: Closed source platform that can get really expensive ... just mention "single sign-on" and watch the dollar signs roll...Link: https://www.algolia.com

2. Cludo #

Cludo are an up and coming SaaS platform not unlike Algolia, with a nice dashboard and machine learning backed features. I really want to like Cludo ... but they currently only offer a web scraper as a tool. Having seen the interface, I can say it's a really good scraper with a good set of features, but it's still a content scraper. In the use case I had, I needed to upload indexes at the time we built them, and we couldn't rely on that kind of tool. If they extended their product to include API uploads, it would be a much stronger product in my opinion.

UPDATE: Cludo team have told me they do have a data push API, so you don't need to use their site crawler. More info: https://docs.cludo.com/#data-indexing_push

Pros: Good contender for the marketCons: No SDK.Link: https://www.cludo.com

3. SwiftType #

SwiftType have recently been bought out by Elastic, and is now called "Elastic Site Search". Again, a good tool with two strong products: app search and site search. But again this product falls into the trap of only allowing content in via web scraper.

Pros: Backing of Elastic, a leader in the landscapeCons: No content uploads API. No SDK.Link: https://swiftype.com

4. Meili #

Meili is an outlier in this space, and an entirely new product that isn't built on any existing technologies (most of the other platforms rely on Elastic under the hood). Meili is an open-source, self-hosted solution (I hope they'll offer it as a Saas in the future too), written in Rust. It is in active development, and possibly pre-alpha as I write this. There's no dashboard which seems a shame, but there is a good set of SDKs and integration guides on their GitHub repo.

Pros: Self hosted, fast as heck. Upload content via APICons: No analytics dashboardLink: https://www.meilisearch.com

5. TypeSense #

I'm adding Typesense about a month after I published this original article, since I've only just found it.

Typesense is, like Meili, open source, with a "cloud" version that you can sign up to as well. Typesense seems to have its positioning right, as a search API that you can install from a Docker image or on a Linux server via apt or yum package managers. There's also API libraries in JavaScript, PHP, Python and Ruby. The JS package in particular seems to come at a reasonable bundle size.

The interesting thing about this is it's written in C++, so it's likely to be pretty fast too.

Shoutout to Christopher Geary for letting me know about this one.

Pros: Self hosted and cloud optionCons: No analytics dashboardLink: https://typesense.org

Conclusion #

If I was an indie hacker or an investor with some developer punch, I'd be building an Algolia clone right now. Whilst they are clearly market leaders there's plenty of room for growth here, only one or two key features hold back the expansion into a mature market with a good amount of choice."

Selecting parents of a nested element with the css :has selector

Wed, 20 Dec 2023 00:00:00 GMT

I was playing around with the :has selector recently and noticed it's got a pretty awesome feature: relative selectors. They're defined as:

a selector representing an element relative to one or more anchor elements preceded by a combinator. Relative selectors that don't begin with an explicit combinator have an implied descendant combinator.

That means that :has() is really smart, anything in the brackets behaves like it does in normal CSS.

This really helped me because my DOM structure looks something like:

<div class="element">
  <div>
    <div>
      <div id="element-i-want-to-target">

According to this stackoverflow answer, I would need to write something like:

.element:has(div > div > div#element-i-want-to-target) {}

(This is Drupal by the way, which seems to love nested <div>s even more than React)

The trouble with that is that it didn't work as I had expected. It never targeted the element correctly, and after playing around I couldn't get it to respond correctly.

By chance experimentation I came up with this:

.element:has(#element-i-want-to-target) {}

And it worked! Even though the targeted element was 3 <div>s deep, it's relative to the parent .element, and so :has() evaluated it as I desired.

This makes sense given the MD docs, but it's still not all that clear, I wish that MDN would have shown the HTML structure along with its examples.

But it makes sense if you think about it: by using :has() you're scoping your CSS to the parent you're using it on; anything appearing inside that pseudoselector works just like normal CSS does.

I do love these recent additions to CSS, it's made it so awesome to work with.

Sending Emails via SendGrid with Cloudflare Functions

Thu, 18 Jan 2024 00:00:00 GMT

SendGrid remains a popular tool for routing emails in today's applications. But it isn't without pitfalls. Here's my setup.

I am very grateful to Aman for his article, which this article updates to reflect changes in the Cloudflare Functions API, some issues I found with SendGrid, and some other useful stuff like local development with Wrangler.

First of all, don't try to use SendGrid's own library with CloudFlare functions. It has a lot of cross-dependencies with Node.js, and CF functions are decidedly not Node.JS. At some point it's going to try to call crypto and fs, neither of which exist in the V8 engine.

Part 1: Semantics

The first thing to confuse me was the terminology Cloudflare uses for its products. It took me a while to realise that Workers were separate from Functions. I think the terminology is sometimes interchangeable in their marketing which doesn't help.

The APIs relating to Functions (not workers) are on their docs site. The Workers API is different (although I am still confused about the delineation of these products because they both seem to use the V8 engine).

The setup for a new worker is as follows:

export async function onRequest(context) {
  const { request, env } = context;
}

Definitely don't use export default here, it works fine locally with Wrangler but when you deploy it your logs will say that it couldn't find any valid function entry point.

I passed the request object around a fair bit in other functions, but do be careful about cloneing the object. At the time of writing, Cloudflare functions have a very minimal memory allocation and cloning objects leads to a warning message, which can lead to functions crashing.

Be careful about how you utilise request in your functions. If I could be more specific about that particular iceberg, I would. It led to quite a long period of debugging for me.

This seems to work OK though. As Aman did in his article, I'm first catching all other requests that are not POSTs and rendering a "not found" message:

export async function onRequest(context) {
  const { request, env } = context;

  if (request.method !== "POST") {
    return await handleDisallowedMethod(request, env);
  }
  return await handlePostRequest(request, env);
}

async function handleDisallowedMethod() {
  return new Response("Object Not Found", {
    statusText: "Object Not Found",
    status: 404,
  });
}

Environment Variables

You might notice that I'm also anticipating that there are some env variables being sent with this request. This wasn't trivial to set up in Wrangler. It's totally non standard. And because Wrangler is built for Cloudflare Workers as well as Cloudflare Functions, there are 2 ways of supplying variables. I got confused by this and tried to do it the wrong way. Cue 4 more hours of painful debugging. Here's the code that should be in your .dev.vars file locally, which will get picked up in the build step:

SENDGRID_API_KEY=
SENDGRID_EMAIL_RECIPIENT=me@example.com
SENDGRID_EMAIL_SENDER=mywebsite@example.com

Make sure that's excluded from your Git history of course. And be kind to your teammates and make sure you include a .dev.vars.example, which is checked in to Git.

Handling the Request

Now let's handle our request. First we need to get the referrer URL so we can redirect the user back to the website.

async function handlePostRequest(request, env) {
  const returnUrl = request.headers.get("referer");

  let formData = await readRequestBody(request);
  const requestBody = composeRequest(formData, env);

  // ...
}

I'll pause there because I need to tell you what's happening in readRequestBody and composeRequest.

readRequestBody is mostly unchanged from Aman's implementation, except I didn't think it was necessary to go too deep; I am only anticipating requests that contain JSON and formData.

async function readRequestBody(request) {
  const { headers } = request;
  const contentType = headers.get("content-type");
  if (contentType.includes("application/json")) {
    const body = await request.json();
    return body;
  } else if (contentType.includes("form")) {
    const formData = await request.formData();
    let body = {};
    for (let entry of formData.entries()) {
      body[entry[0]] = entry[1];
    }
    return body;
  }
}

Composing the API Request Body

composeRequest is a little more structured than originally. I wanted to compose this in place rather than passing things around too much, so I ended up with this implementation:

function composeRequest(formData, env) {
  const { email, firstname, surname, textarea } = formData;
  return {
    from: {
      email: env.SENDGRID_EMAIL_SENDER,
      name: "my website",
    },
    replyTo: {
      email: `${email}`,
      name: `${firstname} ${surname}`,
    },
    subject: "New message from my website",
    content: [
      {
        type: "text/plain",
        value: `New message from ${firstname} ${surname} (${email}): "${textarea}"`,
      },
    ],
    personalizations: [
      {
        from: {
          email: env.SENDGRID_EMAIL_SENDER,
          name: "my website (example.com)",
        },
        to: [
          {
            email: env.SENDGRID_EMAIL_RECIPIENT,
            name: "Recipient",
          },
        ],
      },
    ],
  };
}

Here's where I got tripped up for about a day and a half. If there's something wrong with the request body here, SendGrid API will return a 302 error. That's right. 302. This error is very vague but I have been told (by StackOverflow) that it's meant for GET requests only because you cannot re-try POST requests.

Don't go looking for this response in the SendGrid documentation, it's not there. Basically make sure you have submitted strings and that they are populated. String interpolation is a good idea here.

there's a long SO post about this which has some other suggestions, like don't break the content up by using \ns. If you are stuck on this one, and you've tried the above without success, good luck.

Okay, let's finish our handlePostRequest function:

if (!env ?? env.SENDGRID_API_KEY) {
  return Response.redirect(`${returnUrl}?success=false&reason=no-api-key`);
}

let emailResponse = await sendEmail(requestBody, env);

// continued below

Submitting to the API

Some basic things here, the sendEmail is a fetch request, pretty straightforward:

async function sendEmail(messageBody, env) {
  try {
    const email = await fetch("https://api.sendgrid.com/v3/mail/send", {
      method: "POST",
      headers: {
        Authorization: `Bearer ${env.SENDGRID_API_KEY}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify(messageBody),
    });
    return email;
  } catch (error) {
    return { status: 500, statusText: error };
  }
}

Whatever the response is, we want to provide valuable feedback, and I haven't set up logging on CF functions yet so I want to pass those back to the client:

// continued from above
let emailResponse = await sendEmail(requestBody, env);

if (emailResponse.status > 299) {
  return Response.redirect(
    `${returnUrl}?success=false&reason=SendGrid%20API%20returned%20${emailResponse.statusText}%20(statusCode: ${emailResponse.status}))`
  );
}
return Response.redirect(`${returnUrl}?success=true`);

I enjoyed coming up with this flow. We're redirecting the user back to the sender URL with some query parameters that will allow us both to render a success/fail message, and to debug what's going on (as long as the user hasn't navigated away from the form).

CopyPasta Here

Putting that all together and you get this. Happy copypasting!

export async function onRequest(context) {
  const { request, env } = context;
  if (request.method === "POST") {
    return await handlePostRequest(request, env);
  } else {
    return await handleDisallowedMethod();
  }
}

function composeRequest(formData, env) {
  const { email, firstname, surname, textarea } = formData;
  return {
    from: {
      email: env.SENDGRID_EMAIL_SENDER,
      name: "my website",
    },
    replyTo: {
      email: `${email}`,
      name: `${firstname} ${surname}`,
    },
    subject: "New message from my website",
    content: [
      {
        type: "text/plain",
        value: `New message from ${firstname} ${surname} (${email}): "${textarea}"`,
      },
    ],
    personalizations: [
      {
        from: {
          email: env.SENDGRID_EMAIL_SENDER,
          name: "my website (example.com)",
        },
        to: [
          {
            email: env.SENDGRID_EMAIL_RECIPIENT,
            name: "Recipient",
          },
        ],
      },
    ],
  };
}

async function sendEmail(messageBody, env) {
  try {
    const email = await fetch("https://api.sendgrid.com/v3/mail/send", {
      method: "POST",
      headers: {
        Authorization: `Bearer ${env.SENDGRID_API_KEY}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify(messageBody),
    });
    return email;
  } catch (error) {
    return { status: 500, statusText: error };
  }
}

async function handleDisallowedMethod() {
  return new Response("Object Not Found", {
    statusText: "Object Not Found",
    status: 404,
  });
}

async function handlePostRequest(request, env) {
  const returnUrl = request.headers.get("referer");

  let formData = await readRequestBody(request);
  const requestBody = composeRequest(formData, env);f

  if (!env ?? env.SENDGRID_API_KEY) {
    return Response.redirect(`${returnUrl}?success=false&reason=no-api-key`);
  }

  let emailResponse = await sendEmail(requestBody, env);

  if (emailResponse.status > 299) {
    return Response.redirect(
      `${returnUrl}?success=false&reason=SendGrid%20API%20returned%20${emailResponse.statusText}%20(statusCode: ${emailResponse.status}))`
    );
  }
  return Response.redirect(`${returnUrl}?success=true`);
}

async function readRequestBody(request) {
  const { headers } = request;
  const contentType = headers.get("content-type");
  if (contentType.includes("application/json")) {
    const body = await request.json();
    return body;
  } else if (contentType.includes("form")) {
    const formData = await request.formData();
    let body = {};
    for (let entry of formData.entries()) {
      body[entry[0]] = entry[1];
    }
    return body;
  }
}

Rendering the success / fail message

As a bit of icing on the cake, I have the following minimal JS in my Astro site to render success or fail messages to the user:

document.addEventListener("DOMContentLoaded", () => {
  const formHandlerResponse = new URLSearchParams(window.location.search).get(
    "success"
  );
  const formHandlerReason = new URLSearchParams(window.location.search).get(
    "reason"
  );
  // return early
  if (!formHandlerResponse) return;

  // was it successful as a boolean instead of a string
  const isSuccessful = formHandlerResponse === "true";

  // grab the original form
  const form = document.getElementById("contact-form");

  // ahh HTML templates. What joy.
  const submissionSuccessTemplate = document.querySelector(
    'template[name="submission-success"]'
  );
  const submissionFailureTemplate = document.querySelector(
    'template[name="submission-failure"]'
  );

  const formParent = form?.parentElement;

  formParent?.removeChild(form);

  formParent?.appendChild(
    isSuccessful
      ? submissionSuccessTemplate.content
      : submissionFailureTemplate.content
  );
});

HTML Templates

With HTML templates I don't need to construct HTML out of JavaScript. Whilst these do have some limitations (good luck rendering the error message into a <slot/> or whatever), they are a lot nicer to use in my opinion.

<template name="submission-success">
  <div>
    <p>Thank you for your message. We will be in touch shortly.</p>
  </div>
</template>
<template name="submission-failure">
  <div>
    <p>
      Sorry, something went wrong when we tried to submit your request. Please
      try again later.
    </p>
  </div>
</template>

The Form Markup

For completion, here's the form markup:

<form
  id="contact-form"
  method="POST"
  action="/send-email"
>
  <input
    required
    type="text"
    name="firstname"
    id="firstname"
    placeholder="First name"
  />
  <input
    required
    type="text"
    name="surname"
    id="surname"
    placeholder="Surname"
  />
  <input
    required
    type="email"
    name="email"
    id="email"
    placeholder="Email address"
  />
  <textarea
    id="textarea"
    placeholder="Message"
    name="textarea"
    rows="3"
  ></textarea>
  <input
    type="submit"
    value="Send message"
    />
  </div>
</form>

Conclusion

Despite some challenges setting up Wrangler to accept my variables, crashing because of some unfathomable clone problem, and sendGrid's insane 302 status code, this was a good project. I appreciate the more lightweight feel to CF functions, and it has forced me to get to know SendGrid API a little better.

Serverless as a Competitive Advantage for your Business

Sun, 21 Aug 2022 00:00:00 GMT

This week I gave a talk at Moar Serverless Conf 2022. The subject was aimed at helping people who are reticent about serverless, or who love it but struggle to convince other people to try it.

Watch the video here

The 4 benefits I talked about are

No server management
Inherent automatic scaling
Consumption based pricing
Faster product iteration

No Server Management #

I once had an error at a place I worked that stopped all of our applications from deploying, and which took around a week for four people to resolve. It was that the containers were running on Ubuntu 16, which became unsupported by the infrastructure.

If we had used Serverless, the problem would have been minimised because we don't care what OS the application is running on. So we can be more flexible about dependency upgrades.

Automatic scaling #

I talked about how organisations will usually have around 2 replicasets of their application running continuously, and have autoscaling to maybe 4. But that traffic spikes could easily go beyond this, leading to Error 429messages.

In a serverless environment, you just watch the numbers roll up, knowing that your infrastructure will scale to the amount of requests.

Consumption based pricing #

I had two numbers on the screen: $2,000 and $0.86. The first figure is what I guessed one company I worked at once was paying for just one of it's applications in one environment every month. They had 4 environments, and 13 applications, and so the monthly cost would be huge.

In contrast, I shared that I got my first AWS bill last month and it was $0.86. This is after 2 years of use.

I admitted that it wasn't a fair comparison; I mostly have small blogs which don't get a lot of requests, but I was just amazed at the difference in costs.

Faster product iteration #

I said that since developers don't have to worry about every part of the stack, they can focus on business deliverables. That means the business can go faster and beat the competition to market, instead of being late to the party.

I concluded by saying Serverless isn't a rose garden. It requires that the application is specifically built for such an environment. This makes adopting serverless difficult.

So developers need to start with serverless, not try to adopt it later.

One way of getting started with Serverless quickly is to take an existing open source application and deploy it. That allows you to see the potential and how certain problems have already been solved.

I advised that developers take Webiny for a spin, because it has all of the benefits of Serverless without the cost of building something yourself.

It was great to speak at an event with so many other good people I know and respect."

Should I move from WordPress to code?

Thu, 24 Oct 2024 00:00:00 GMT

I'm getting asked this question a lot lately, so I made a few charts that I hope explains the choices people face when they are thinking about switching, and the effort that might take. I hope it helps someone.

Web development is a funny term isn't it? It encompasses a wide section of people who build things with the web, and they may or may not be technical.

But then I suppose something similar could be said of people who speak a language: I speak a few, but none of which I could confidently say I'm fluent.

So too web development is a ~spectrum~ gradient. Along the gradient are people who code considerably well in JavaScript, and those who build things with off-the-shelf themes.

So I thought I'd draw up some comparisons for you.

Language difficulty charts

You may have seen something similar to these graphs before: for each language you learn, there's a learning curve: sometimes it's relatively easy to begin with and gets harder as you learn more about a language.

This one below is for an English speaker learning Chinese. As you see, it starts quite difficult. Tonal languages are challenging for English speakers. Then there's a dip in the middle, which shows that once you've got the idea of tones, you realise the grammar is simpler when compared to English: there's no tenses which mutate word endings, there's no masculine / feminine dichotomy to contend with. But then as you learn those things you realise there's a lot of challenges with context, and communicating time. There are more than 10 different definitions for the partial le (了). So towards the end of your journey learning Chinese, you have a steep rise in difficulty again.

Code learning difficulty chart

I think learning to code can be charted similarly. It starts off rather difficult, but then gets easier with time. You have trouble figuring out what all those chevrons mean, what attributes different elements have, and when to self-close and when not to.

Then you pick up CSS and realise it's just a selector, a key and a value. Not to say there aren't challenges: learning how the cascade works is likely going to be a major "eureka!" moment for everyone.

At least, that has been my experience.

But where does WordPress sit on this gradient?

Here.

To start with, it's easy peasy. Run the installer, grab your pagebuilder plugin, and start dropping boxes onto the page. Add more plugins for forms and other functionality, for Google Analytics, etc etc etc.

Sooner or later youre going to want something that all of the plugins you have access to don't do. So you do a web search. You find this one dodgy plugin from seven years ago that you have to pay for in Bitcoin.

Two weeks later you find that someone has defaced your website. You delete the defacement. It comes back.

You've been hacked.

How do you un-hack a WordPress website? Good luck finding a plugin for that, if you can even install one by this point. Two days later the client calls you saying people can't access the website because it's behind a red screen. You've been blacklisted from search engines.

Now the learning curve is so steep it's almost impossible. And people are shouting at you to fix it.

This is the point that people often start questioning their technology choices.

Imagine if you'd questioned them sooner.

Should I use ternary or the logical and operator?

Thu, 06 May 2021 00:00:00 GMT

Templating with JSX in React is easy ... until it's not. Recently a colleague recommended I use the logical and operator instead of a ternary. However once we'd dug into it a little, we found these operators do very different things ...

Quite often we follow this pattern for brevity, and there's good value doing it if there's only one variable at play:

{  isLoggedIn && <SomeComponent />;}

This avoids us from having to write something like

{  isLoggedIn ? <SomeComponent /> : null;}

which is totally redundant here, since if it's value is false, it won't return the component.

However, when there's a couple of things going on you might find it doing something unexpected:

{
  formErrors.likesPotatoes ||
  formErrors.likesBananas ||
  formErrors.likesCake ? (
    <NotificationMessage icon="alert" status="error">
      <p>
        Please ensure that all the required questions have been answered before
        proceeding.
      </p>
    </NotificationMessage>
  ) : null;
}

is not equivalent to:

{
  formErrors.likesPotatoes ||
    formErrors.likesBananas ||
    (formErrors.likesCake && (
      <NotificationMessage icon="alert" status="error">
        <p>
          Please ensure that all the required questions have been answered
          before proceeding.
        </p>
      </NotificationMessage>
    ));
}

With the ternary operator (isTrue ? dothis : dothat), our <NotificationMessage/> will show when any of the conditions are met. The logical AND (isTrue && dothat) will only show the component if all of the conditions are met.

Why?

The Difference between the logical AND and ternaries #

Ternaries work similar to the if operator. So it short circuits (closes off before any other variable is assessed), and returns true if any of the values are true.

On the other hand, the logical AND operator returns true only if and only if all of its operands are true.

In our case, when checking for form errors, we want to show a notification if any of the fields have an error. So the ternary is the way to go.

Alternative 1: abstract it #

There is another way of handling this situation where you could use the logical AND: chain those errors in a variable before returning the JSX:

const hasError =
  formErrors.likesPotatoes || formErrors.likesBananas || formErrors.likesCake;
return (
  <>
    {hasError && (
      <NotificationMessage icon="alert" status="error">
        <p>
          Please ensure that all the required questions have been answered
          before proceeding.
        </p>
      </NotificationMessage>
    )}
  </>
);

Alternative 2: Wrap it #

My friend Warrick Hill mentioned that you could also wrap the options in brackets to ensure they get evaluated together and therefore don't short circuit. This is how mathematical bracket operators work, where everything inside the brackets gets evaluated first, for example (2 \* 3) + 2 = 8 but 2 * (3 + 2) = 10:

return (
  <>
    {(formErrors.likesPotatoes ||
      formErrors.likesBananas ||
      formErrors.likesCake) && (
      <NotificationMessage icon="alert" status="error">
        <p>
          Please ensure that all the required questions have been answered
          before proceeding.
        </p>
      </NotificationMessage>
    )}
  </>
);

Although Warrick points out that this is harder to spot than the alternative #1 above.

Thanks #

Thanks to Aimable N and Chris Geary (as well as Warrick) for their help with this article."

Should I Use Gatsby or Next.js For My Next Project?

Fri, 23 Sep 2022 00:00:00 GMT

Developers tend to get very passionate about their tools. And rightly so: without the best tools for the job at hand, we couldn't achieve our objectives, make something awesome, or build a better world.

React is still a major player in terms of frontend development. There's currently a competition of sorts going on between the two React heavyweight frameworks Next.js and Gatsby. Which one is better? Which should you learn? Is Gatsby better, or Next.js, for building your next project?

At Webiny, we're interested in providing developers a backend Headless CMS to compliment your frontend, so it's of deep interest to us which tools are popular. Knowing which ones you're using ... even which ones you are preferring, influences our product decisions. We think about it when we talk about marketing. And we consider it when we're planning what we build into our open source CMS.

So we decided to do a bit of critical thinking about whether Next.js or Gatsby is better. We came up with these criteria to match them against. Even so, we're going to leave it up to you as to which one you're going to use!

Server Generation and Configuration

Generating React code, or any JavaScript, when an application is built, is a critical metric when it comes to frontend frameworks like Gatsby and Next.js. But we don't think it's speed that's critical. It's certainly important, but it's one of those metrics that the developers of these frameworks are acutely aware of, so they're going to make sure they keep making advancements in this area.

What's more interesting is what APIs you have access to on server generation. Does the tool make it easier or more difficult to do what you need to? That's what we're assessing here.

Whilst Next.js allows access to customize headers, page initialization, add rewrites and redirects with it's config file and custom _App.js component, Gatsby has a config file, a custom html.js document (which is similar to Next.js). It also has files to directly access the server-rendered application and the hydrated application in the browser.

Gatsby certainly has a wider variety of configuration open to developers here. But that can also be daunting or unnecessarily complex for someone who just wants to build with React but also have static generation.

Also, sometimes it's not clear what the files are for. For example, the gatsby-node.js file isn't specifically for APIs that run in the server environment (which would be using Node.js), instead, this is where the internal node transformation happens, allowing you to create dynamic pages and custom GraphQL resolvers to your application.

Admittedly, once you know that, it's great to have access to all of these APIs to customize your data. But we'll get to that a bit more later.

Documentation

Whether Gatsby or Next.js documentation is good or bad seems highly subjective. But sometimes documentation can inadvertently hide things developers need to know under generalizations or by accidentally missing things out. Sometimes an API badly explained is worse than not having it documented at all.

We all know this, which is why we looked at documentation as a marker on whether you should choose Next.js or Gatsby to build your next project.

Whilst Next.js documentation has a cleaner, focused look, Gatsby's is busier with a 3-column layout. Though sometimes that's a good thing. Developers probably won't get as lost contextually in the Gatsby docs as they might in Next.js.

We generally prefer the darker code blocks and higher contrast on the Next.js website. But it could be because we're just getting old and our eyes aren't as good as they used to be.

Next.js' strengths come out clearly in it's focused approach towards lower-level APIs such as the getStaticProps() and getServerSideProps() functions, and these are explained very well. But there are also some things that are not mentioned except succinctly in an "unsupported features" section. This can be frustrating when you find out you've built your application to use a certain feature, only to realize it's not supported in the environment you intended to deploy to.

Routing

Both frameworks take advantage of file-based routing. Next.js and Gatsby have the ability to define dynamic routes as well, but Gatsby doesn't clearly explain how this works, perhaps assuming you're only looking for this feature if you've already used Next.js.

What is great about Next.js is the ability to create API Routes. This could be really useful in a lot of situations, especially if you have a sprawling REST API endpoint and want a minimal set of data returned from it, you could massage that data down to only the content you need on your frontend. In fact, Next.js has it's own express-like server-side framework called Micro.

This turns Next.js into a truly isomorphic application, which is meant to have components server side and on the client, working in tandem to produce a single application.

On the other hand, Gatsby seems to keep these concerns clearly separated, even with their own Gatsby Functions, the emphasis is on a separate backend that communicates with the hydrated frontend application.

Do you like to mix these concerns? Or keep them separate? It depends slightly on what kind of developer you are, and what kind of project you're building, as to which you'll prefer from this point of view.

Conclusion

Gatsby and Next.js are two very different tools with unique value propositions: either low-level tools to provide fundamental APIs and free developers from unnecessary tooling, or an ecosystem full of plugins of varying quality, together with a data layer to stitch your content together.

As we said at the start, this isn't a full roundup of all of their features. Maybe what you're building needs something more specific like Astro, Nuxt or SvelteKit. We only know what we've encountered when building our starters for Next.js and Gatsby.

Oh we didn't tell you? We've built 1-click deploy starters for Webiny headless CMS.

👉 1-click starter for Next.js and Webiny Headless CMS on Vercel

👉 1-click starter for Gatsby and Webiny Headless CMS on Gatsby Cloud

Make sure you check those out. Maybe it'll help you make your mind up ... should you use Gatsby or Next.js? The choice is ultimately yours."

Start and end with the people

Sat, 19 Oct 2024 00:00:00 GMT

Professional software development at an enterprise scale is inherently, and at every level, social. How does comprehension of this fact help teams to perform better?

Who are the people who matter most to the product your team produces? I'm sure you're probably thinking about this in terms of end users: the people who spend time on the platform, working their way through carefully crafted workflows you've been crafting.

The people who use our products are the ones we obsess over the most. After all the chances are that they play the most significant role in generating the revenue our organisations need.

However, in any organisation there are other groups of people we ought to give significant consideration to. When we acknowledge and seek to meet the needs of these other groups I think we are the most effective we can be as a software team.

Two of the most significant of these are those higher up the chain (stakeholders) and those who are closer to the product (engineers). There are others in the wider team too (power users) that we should also give consideration to.

Stakeholders

Are stakeholders users of our product? It would be a mistake to see them as such, even if they use it themselves. Their unique perspective puts them in a responsible position towards users. However, their recommendations, suggestions and feature requests need to be validated and researched if the actual needs of stakeholders are to be met.

What do I mean by that?

Imagine a stakeholder requests a feature, which is added straight to the backlog. The feature is built and shipped, the stakeholder's requirements are satisfied. But the feature proves to be a waste of time.

Why?

Without due diligence is done beforehand, we are not meeting the real needs of stakeholders. After all, we didn't ask the most important question before any further work was done: "why do you want this feature?". Anything other than "I think it would be good" is an acceptable answer here.

Without that thought process, we could build a feature that nobody will use. Or we could build something that doesn't solve the problem the stakeholder was trying to address.

So ask them, what is it you're trying to achieve?

Even after we're clear about what is being requested, that's not the point it should reach the backlog; we would only be satisfying the real needs of stakeholders by doing proper due diligence.

We might need to do further research about whether it's a good fit for our product, whether competitors have already implemented it or not, and why, whether users think it's a feature they could benefit from, and how much time might it take to build?

The main idea behind all of these is principally, "is it something that will add value to the product" and then, "is it worth the cost of implementation"?

Stakeholders exist to improve the value of the product. If they're not achieving that, they're not going to be fulfilled. That means that knowing the implications of what they've asked for is a key driver to their decision making.

Team members

Ensuring the success of people who use what we're building as well as those who are stakeholders in the product should be as important to us as the success of the individual members of the team involved in constructing it.

How are they doing? Are they having a good day, a good week, a good experience working for you?

It's only when we help facilitate them being able to find their place within the team that we get the most out of them.

That means that if they are disengaged, its our job to find out why that might be the case. Often it boils down to three reasons.

Tasks are too challenging

In this case, the person might benefit from some extra coaching or support to achieve objectives. Their backlog could be cleared to help them focus if they are easily distracted, or tasks can perhaps be broken down to improve focus. The size of a ticket matters less about it's percieved difficulty than it is possible to do for the one who fulfils it. So customise tickets: write smaller tickets for those less able to focus, larger ones for those who can.

Personal problems

How well do we know what's going on in the lives of those we are responsible for? If there is no "idle" conversation, then how will you know if a bereavement has affected them? If they are having issues with a colleague? If they are struggling with impostor syndrome?

Get to know your people. Being an effective leader means being someone who represents the company's consideration for them when they're going through a rough patch. You'll get so much more out of them in terms of years of service as well as productivity if you make a friend instead of just a colleague.

Lack of clear direction

We don't want to be seen as treating people like they're infants, but often what someone perceives as clear direction is much more explicit that we might expect. Have we clarified what is expected of them for this task? Have we asked them to outline their plan verbally to us? (often this is a really good way to check understanding, a simple "how do you think you're going to approach it?" is enough).

More importantly than either of the above questions is "do they feel comfortable approaching us if they don't understand what is expected of them?"

If your response to their questions give even a slight hint that you're expecting them to already comprehend fully your requirements, then you're doing it wrong. There should be no hesitation on their part to ask you for more clarifications, more detail, or to challenge anything about the work you have scheduled for them.

Otherwise it is likely that the delivery will suffer, and so will their confidence, which could lead to other significant issues and underperformance.

Knowing our team personally and making sure individuals are comfortable approaching us for clarification is a big step to creating a team that is as productive as it can be.

The wider business

So far we've looked at individuals within the business. But are the people who are impacted by changes in our product ever involved before we release new features?

I have found there are often key people within a business, whether that's power users or that one technical person in the 1st line support team, who have a deep understanding of the product. Do we consider them?

Even if we just tell them what's coming up, it can empower their decision making and the way they give support to others significantly. If we also ask them for involvement in scoping features, or even suggesting new ones, we'll tap into a potential goldmine of new ideas that can inject more revenue into a business from unexpected angles.

Start and end with the people

The people are what make up everything about our business. Look holistically at our product. It's definition might be a piece of software, but it's the result of interactions between people. When we focus on that instead, zooming out of implementation details, and instead focusing on what can do to fulfil stakeholders, generate effective teams, and involve the wider business, we will build things that are wanted, needed and enjoyed. By all involved.

SVG Animation on Hover with GreenSock

Sun, 21 May 2017 00:00:00 GMT

One thing I've tried to do more of is to use animations on frontend projects I've been involved with. GreenSock animation library is a great way of standardising and improving on animations that otherwise wouldn't be available on all browsers.

Animations add another dimension to your projects that helps users and can provide either an extra bit of feedback, extra character, and even increase conversions.

I really love GreenSock for it's performance and range of animations. I particularly like their modular approach to the library. For example, if you have something relatively straightforward in mind, you can just use the "Tween Lite" plugin, which is 27kb minified. If you need more flexibility with timelines, you can add Timeline Lite, a further 12kb.

This means that you're not tied to a monolithic platform that covers all bases. You can customise and work towards a performance budget whilst still using some great features.

GreenSock also works really well with ScrollMagic to make rich interactive experiences. It's often the combination of these libraries that result in great experiences that are often featured on the Website Awwards site.

Example #

For a recent project, I wanted to add some hover effects to a UI element that made it seem as if it was almost a fluid change of state. The element was contained in an SVG shape with 2 layers, the background and the graphic.

A fluid-feeling interaction #

I found that there are a few caveats when working with SVGs. The first is how to scale from the center of the animation. To do that you can add transformOrigin: 50% 50% to the properties you're applying. That will find the center of the SVG element, and animate from there.

Initially, to animate this I tried to create one timeline that targeted both elements, but this resulted in an effect that looked artificial, since the animations didn't overlap each other. One scale effect was applied to one element. It was only after that that the other scale effect started.

var connectCircles = new TimelineMax();

connectCircles.staggerTo($(this).find(".cloud, .microphone, .handset"), 0.3, {
          scaleX: 1.2,
          scaleY: 1.2,
          ease: Elastic.easeOut,
      transformOrigin:"50% 50%"
          }).staggerTo($(this).find(".disc"), 0.6, {
          scaleX: 1.2,
          scaleY: 1.2,
          ease: Elastic.easeOut,
      transformOrigin:"50% 50%"
          })
        };

Output: https://codepen.io/endymion1818/pen/dWQevw

To resolve this issue, I split the functions into separate timelines that allowed them to run separately:

var connectCircles = new TimelineMax();
var connectShapes = new TimelineMax();

connectCircles.staggerTo($(this).find(".cloud, .microphone, .handset"), 0.3, {
          scaleX: 1.2,
          scaleY: 1.2,
          ease: Elastic.easeOut,
      transformOrigin:"50% 50%"
          });

      // second timeline
      connectShapes.staggerTo($(this).find(".disc"), 0.6, {
          scaleX: 1.2,
          scaleY: 1.2,
          ease: Elastic.easeOut,
      transformOrigin:"50% 50%"
          })
        };

That way, when we run the animation, the two timelines run independently from each other:

https://codepen.io/endymion1818/pen/xgEYqG

Getting the timing right in animations is one of the hardest things, and takes the longest amount of time. It took me quite a bit of experimentation and asking for feedback to arrive at the duration of each transition so that it appeared that the effects were interacting with each other in a natural way.

A lot of animation is fooling the eye into thinking it's real. Our brains recognise natural interactions which follow our experience of physical reality, for example, the way a water droplet responds when you touch it. But if you get it wrong by a fraction, it looks strange or unfamiliar, and it can have an adverse effect on your audience.

Conclusion #

I've had so much fun with this you wouldn't believe. I am really excited about animations and look forward to working with GreenSock and the Web Animations API more in the future."

Content-Author friendly APIs with Strapi

Fri, 08 Jun 2018 00:00:00 GMT

I've worked in a few different capacities around the web, and can see the value of interfaces that those writing content for the web feel comfortable using. I'm also becoming more comfortable with JavaScript, and want to find alternatives to the PHP tools I've been using. On that journey, I discovered Strapi.

When I think of content authors, I'm specifically thinking about non-technical individuals who don't want to get involved in writing HTML. For these people, making or editing a Markdown file might be a disagreeable process. This is simply because they need to be in an envinronment that allows them to focus on their current task — writing an article, updating page content, publishing a job advert - without distraction. Or at least, with minimal distraction.

I am sure this is one reason WordPress has gained the community it has:- it's easy to use, and makes the content author feel informed and empowered without them feeling like the software is getting in the way.

The challenge for me has become to find some software - preferably open-source, definitely JavaScript - that can be a suitable environment for these kind of authors.

And I think Strapi could be a very good fit indeed.

How Strapi Works for Content Authors #

The 3 main requirements I had when analysing this project for a content authors' perspective were:

easy to write & edit posts
easy to restrict and enable access and authorship
content authors feel in control

I'm going to cover these in more detail below.

Writing and Editing Posts #

Logging in and authoring content in Strapi works very much like any other content platform. There's a left sidebar which you can use to navigate different types of content, and fields that you fill out that are clearly labeled, functional and give suitable feedback when you're filling them out, or saving posts, or just navigating around.

Author Roles and Permissions #

New authors can be defined by administrators, and access can be given to specific areas of the API.

This allows us a key feature that I've seen being particularly useful in larger applications: authors can write articles that can be reviewed before publication.

Authors Feel In Control #

The thing that I felt conflicted most about WordPress was that it gave too much control to authors. The ability to add plugins and change the site's appearance made it a minefield for a developer.

But WordPress made a deliberate choice to do that. WordPress' aim is to "democratise publishing" so that anyone can have a blog or personal website.

There's always a balance to be struck here: too restrictive and authors will go elsewhere, too open and you end up with a mess of a site that can cost thousands to fix.

Strapi has a plugin ecosystem, which I hope will grow. Through their marketplace (which you can't miss, it's heavily promoted within the backend), you can add analytics tracking and other functionality, or purchase plugins which extend Strapi.

This allows content authors and business owners to feel more in control of their project. Having these plugins means they're not totally dependent on others when it comes to extending their Strapi site.

It makes authors feel that the site belongs to them and not their developer.

But, of course, it has to work for developers as well...

How Strapi Works For Developers #

For Strapi to work for those with a development background, I am looking for the following:

easy API authorship
extensible code
ease of entry for newer developers

API Authorship #

API authorship can be done in 2 ways with Strapi. Either in the GUI or in the command line. This, I think, is a master stroke.

Independently-minded content authors can define or destroy their own content fields, and entire types as they wish, in a GUI that gives them adequate feedback about what's happening.

Conversely, APIs can be set by the developer in a JavaScript object, without having to interact with the GUI at all.

This is really great. By not forcing us down a single route, Strapi gets the best of both worlds.

Extensible Code #

Calling to the strengths of a robust language that's well known, developers can modify Strapi to their hearts content, as long as they don't want to update the core. And if they do, they can also turn to authoring a plugin, and hook in to Strapi in a way that separates our concerns out and allows us to write code in isolation.

As I cautioned above, this could lead to Strapi being mis-used, and result in bloated or insecure code. However, Strapi's major advantage in this is that it is an API first, and a content authoring platform second.

So whatever plugin authors do, they can never arbitratily affect the frontend of the website. Even if some malicious — or simply bad - code enters Strapi, rendering it unuseable, that needn't affect the frontend of the site at all, especially if it's a static site.

Good For Newer Developers #

Ultimately, developers are going to be the ones selling this product. We're the ones who hear about this new stuff and set expectations to decision makers about whether or not to use it.

So it has to be accessible to a broad base, including (in my view) newer developers.

I particularly frame this for newer developers not just to widen the catchment of a product but because it gives me a window into some other areas not directly related to the code: the helpfulness of the documentation, and a sense of community surrounding the project.

It seems like a lot of effort has been made by the Strapi team to get clearly written and easy-to-follow documentation into the places where it can be found:- not just on the website and GitHub repo, but on 3rd party sites, like Medium, too.

However, I have had difficulty with the documentation because it often refers to older versions of Strapi which are no longer maintained. I found this to be a bit of a minefield when I tried to initially use the Docker image that's available, and found the interface took me to the now-depracated Strapi Studio to create my APIs.

I then deployed to Heroku using the Strapi 1-click deploy, and spent literally days trying to either modify the API or clone the site locally. Admittedly, this was less to do with Strapi than complications with using Heroku. However, it was a roadblock that delayed my using Strapi in production until I can fix that.

To be fair, the community is very responsive and after joining the Slack channel I got answers to my questions very quickly.

Strapi - A Project I hope will grow #

Overall, I really believe the Strapi authors have done it right. It meets the needs of content authors and developers really well. Although the docs leave a little to be desired, the project itself is making good progress.

Personally, I can't wait to see what the future of Strapi means for content creation on the web."

Switching to BEM Syntax for CSS

Fri, 21 Oct 2016 00:00:00 GMT

Like a lot of things in the web world, CSS is a bit of mess. As the web gets more complicated, the tools and languages we use need to develop with us. This is currently happening as the W3C push forward with new functionality like Grid and Flexbox, but until those come into general use, we can aim to limit the risk of our code becoming unfathomable by other developers.

I was a little suspicious of using BEM syntax for describing my CSS, I guess I didn't see the need for it and didn't have to work with other developers that much. But once I saw how it was being used on a project, I must say I was beginning to warm to the idea.

If you take a look at the code on the FutureLearn website, I find I can much more easily scan the code and identify the different elements that appear -- and more importantly, I can clearly follow the hierarchy of elements that are used on the site. Now, this hierarchy is critically important and something that can be very valuable to your projects.

Wait, doesn't SASS nesting give us hierarchy? #

Yes, it's true that you can nest elements inside one another, and therefore more easily scan your CSS. So an element would be described something like this:

.panel {
    ...

    .body {

        .title {
        ...
        }
    }
    .image {

        .caption {
        ...
        }
    }
}

But have you ever looked at the output for that in your CSS file?

.panel {
    ...
}
.panel .body {
    ...
}
.panel .body . title {
    ...
}
.panel .image {
    ...
}
.panel .image .caption {
    ...
}

Although this is still fairly easy for us to scan quickly, the downside is that all of these extra selectors in your entire project up can eventually slow down the parsing of your css file.

BEM allows us to use a much flatter format, which can be even more descriptive:

.panel {}
.panel__body {}
.panel__title {}
.panel__image {}
.panel__caption {}

These double underscores indicate that the selector is referring to an Element (the “E” in “BEM”) of the Block .panel. This keeps the CSS flatter and allows us to scan what the element is doing quickly.

This gets even more useful when you add in Modifiers.

Modifiers #

An elements might have a variant that we would use in a different context:- say, for example, a navbar in the footer: we could re-use the same Block-level code, with a few modifications. Keeping with our example, we could do this:

.panel—-large {}

In the HTML I would then have

<article class=“panel panel—large”>
	…
</article>

In other words, the Modifier class would only contain the modifier code.

This is helping me to think a bit more about the structure of my code: I find that I’m looking for ways of making a class as succinctly as possible, because at some point it could be extended or re-used in some other way.

For that reason, I don’t typically specify things like column widths inside Blocks, my sizes are a separate entity which contain these elements. That way I can specify how wide an item is by using a column, Flexbox, or (in the near future!) Grid syntax.

To Conclude #

Using BEM syntax makes logical sense, it’s not a huge thing to learn, and helps me keep my code clean and easy to follow by other developers. It doesn’t require any extra build tools but has increased my productivity quite a bit.

More info: getbem.com"

Tailwind progressively collapsing menu.

Wed, 18 Oct 2023 00:00:00 GMT

I had the opportunity to rework one of my favourite codepens recently. When I first saw this method of being able to progressively hide individual navigation items instead of all of them, and to do so without the use of JavaScript, I was impressed with the idea. Here's a remix using Tailwind.

The idea is to use flex to display the individual elements. At small breakpoints, the flex-basis is 100%, meaning each item stretches to the full width of the container.

At medium breakpoints the flex-basis reverts to the initial value (auto), meaning they bunch up as blocks, much like they would if you floated them.

If you then set a fixed height and hide the overflow content, you effectively hide the items that don't fit on the top row.

The overflow can then be toggled with a visually hidden checkbox and an accompanying <label> element (this is known as the "checkbox hack") that provides the functionality of a button.

<p class="codepen" data-height="300" data-default-tab="html,result" data-slug-hash="dywEaBY" data-user="endymion1818" style="height: 300px; box-sizing: border-box; display: flex; align-items: center; justify-content: center; border: 2px solid; margin: 1em 0; padding: 1em;"> <span>See the Pen <a href="https://codepen.io/endymion1818/pen/dywEaBY"> Tailwind Progressively Collapsing Navigation CSS Only</a> by Ben Read (<a href="https://codepen.io/endymion1818">@endymion1818</a>) on <a href="https://codepen.io">CodePen</a>.</span> </p> <script async src="https://cpwebassets.codepen.io/assets/embed/ei.js"></script>

Caveats

There's a couple of caveats with this:

1. It's not accessible.

This is probably the primary reason for not using it. We have hidden the checkbox and the label, because for screen reader users, they're not useful. I sincerely hope the navigation list is still useful to people with visual impairments. I've experimented with focus-visible to ensure all elements in the list are visible, but it is still quite jarring.

2. You can't automatically hide the "more" button

Without manually showing and hiding the toggle for the other items in the list, it will always show regardless of whether there are more items in the list or they are all on the page.

3. It's not easy to style the "dropdown"

Since this is an overflow area, it's not even part of a pseudo element. So changing the background only in the overflow area is not possible. And since we cannot know the height the overflow is going to be, we cannot easily make a pseudo element to fit that area.

An interesting experiment

As I said before, probably don't use this in production. There's probably a very small edge case where it would be more useful than a nav that collapses at a certain breakpoint.

All in all, I still love the combination of CSS hacks that lets us do this sort of thing. It's what makes CSS such fun.

Accessibility and you

Tue, 26 Jan 2021 00:00:00 GMT

Published on Tuesday, 26 January 2021

Over a year ago I was invited to give a talk at London dev meetup JS Roundabout on the subject of accessibility. I recently discovered the talk has been put online ...

I was honoured to present this talk at JS Roundabout, who specifically contacted me on the back of some of my articles about accessibility, especially in the React space.

Watch on YouTube

I don't know about you but I always look back on myself with a critical eye. I talk too fast. I stutter sometimes and lose the thread. But my aim was to motivate and generate enthusiasm, and I think I achieved that. I hope I achieved that."

How Were Using WordPress as a Headless CMS

Tue, 24 Apr 2018 00:00:00 GMT

Published on Tuesday, 24 April 2018

I'm not a stranger to public speaking. But speaking on a technical subject? To people who are smarter than me? Not something I relish. Yet, I felt I had something to offer the WordPress community, so I volunteered to speak at one of the UK's biggest WordPress meetups, WordCamp London.

UPDATE: Here's the talk hosted on Wordpress.tv

Ever since starting my web development career I've wanted to be in the place where I could help other developers out with their development dilemmas. It's taken me a long time to get to this stage, and it's only because of some great help from the following people:

Chris Geary
David Hewitt
Steve Woodall
Louise Towler

It's thanks to them imparting their knowledge and expertise (and sometimes their time), as well as others, that I have anything that might be considered worthy of sharing.

The Talk #

My talk covered these main points:

What is a headless (or decoupled) CMS
Why use WordPress as a headless CMS
Tools and process of building a headless site
A basic example and some gotchas we found
The Future is (probably) headless

You can find it on the conference website too.

The experience #

... was terrifying, to say the least! I rattled through it all in less than 30 mins (I had been given 35 minutes), but there were some really great comments & questions from the audience that I found fascinating. I'm currently planning a series of blog posts on some of the questions that were raised."

What should I use to build my new project?

Tue, 20 Jul 2021 00:00:00 GMT

There are so many choices when it comes to frameworks, and that's teriffic! But it could be bad news for people who struggle to make decisions. They might feel paralyzed just by the amount of choice that's on offer.

With that in mind, I've given some deep thought into the tools I've looked at recently, and historically, and tried to imagine how I might build an application, both frontend and backend, if I was just starting out.

I came up with this chart. It's not a complete list of technologies, and it's at least a little biased towards my favourites. But it's hopefully something that can help unblock that paralysis, at least giving you a place to start.

Making decisions about technology is difficult. I hope that this resource eases your pain even if just a little!

Backend #

This isn't mutually exclusive, but you might be spinning up a blog using Markdown or a 3rd party service. In which case, let's just skip this part and go to the frontend.

Also, a lot of the frontends I've mention have some sort of functions API built in. But since the backend isn't the focus of those tools, I've left them out for simplicity's sake.

If you're thinking you aren't going to need a lot of backend, likely the best thing to use would be lambdas. As I mentioned above, a lot of the frontend tools we have access to do have some sort of API for backend services. You could use that, or Netlify / Vercel's implementation, unless you're going directly to a cloud provider for your hosting.

If you need more than this, you might want to consider whether you want to go the serverless route, or use a monolilth.

Serverless tooling is a little less mature but it's probably more cost effective. You're not constantly paying for containers running in whatever service you're hosting on. So if you're needing a lot of backend it might pay dividends later.

As I mentioned, the choices are very broad with monoliths. From straight-up CMS forms like Keystone, Ghost, Strapi, Payload and Apostrophe to full-blown frameworks such as Express, Meteor, Nest or Blitz.

If you're looking at serverless, choices are decidedly more finite. There's a caveat though: some of the monoliths can run as serverless. Nest and Express are the most notable of these.

Otherwise if you want a framework to build on, there's Redwood (which works particularly well on Netlify and soon Vercel) or Webiny (for AWS, Azure and others).

Frontend #

The only JavaScript tool that doesn't have dynamic client-side routing out of the box is Eleventy. Eleventy's focus is on compiling away JavaScript, so you get a fast, browser-native experience out of the box.

If you're focus is performance, and you need client-side routing, check out SvelteKit. Rather than shipping your framework to the frontend, SvelteKit compiles to pure JavaScript. But it still hydrates all of what you can see on your screen, meaning it's not as performant as Astro can be.

Astro, a newcomer on the scene, does client-side routing via it's Collections API, other than that it's HTML all the way baby. I'm particularly fond of Astro because you can use whatever frontend framework you want.

If you're planning for a lot of client side JavaScript, dynamic routing and all, then your choices are on to whether you want a Vue-based or React-based framework.

In the Vue world, there are plenty of choices, from the more low-level Nuxt, to Gridsome, which has a plugin ecosystem and build-time GraphQL API. It'll likely be quicker to use Gridsome because of this, but you may not need it.

Similarly, Gatsby or NextJS are React frameworks. Gatsby has a large plugin ecosystem and GraphQL build-time API, whereas NextJS has a more low-level approach, you source data yourself, and plugins are scarce and likely not officially supported.

I must admit i really enjoy using the build-time data layer approach. It's much easier to see when things are going to be rendered, and you can save API calls for your visitors by doing them upfront at the build stage.

But also Next and Nuxt have some tricks up their sleeves. Next's innovated ISR "incremental static regeneration" means it can keep content more fresh than Gatsby currently can. However I imagine that's going to be a short-lived triumph.

Go forth and build! #

Phew! There's a lot to choose from isn't there?!

If you're still stuck at this point, and don't think you have a clear idea other than you want to build ... just choose one at random!! You will always learn something significant, and it'll be fun!

It's never been more fun to build with JavaScript, the choices we have these days are staggering. I hope this has helped unblock your decision paralysis!!"

Testing with Apollo Client mock provider

Sun, 02 Jan 2022 00:00:00 GMT

Published on Sunday, 2 January 2022

Apollo's MockProvider is a great tool for testing mutations, however it's a little bit magical, making errors a little difficult to find. If your testing your error state, this might come in handy.

I'm currently building a UI for a messages app, but encountered issues when testing sending new messages. Here's my component:

export function SubmitForm() {
    const [message, setMessage] = useState('');
    const [submitMessage, { loading, error }] = useMutation(MESSAGE_MUTATION);

    return (
        <form
            onSubmit={event => {
                event.preventDefault();

                try {
                    submitMessage({
                        variables: {
                            SendMessageInput: {
                                body: message,
                            },
                        },
                    });
                    setMessage('');
                } catch {
                    console.log(error);
                }
            }}
        >
            {error && (
                <div>Sorry, there was a problem submitting your message</div>
            )}
            <fieldset>
                <label htmlFor="input">Compose message</label>
                <input
                    type="text"
                    id="input"
                    value={message}
                    onChange={event => setMessage(event.target.value)}
                />
            </fieldset>
            <button type="submit">Send message {loading && <Spinner />}</button>
        </form>
    );
}

I wrote a test suite for this component, all of which worked correctly, until I got to the stage when I was testing the error state:

it('should render the error state UI', async () => {
        const mockErrorMutation = {
            request: {
                query: MESSAGE_MUTATION,
                variables: {
                    SendMessageInput: {
                        body: 'test',
                    },
                },
            },
            error: new Error('drat'),
        };

        render(
            <ThemeProvider theme={defaultTheme}>
                <MockedProvider mocks={[mockErrorMutation as any]}>
                    <SubmitForm />
                </MockedProvider>
            </ThemeProvider>
        );

        const inputField = screen.getByLabelText(/compose message/i);
        const button = screen.getByText('Send message');

        userEvent.type('test');
        fireEvent.click(button);

        await waitFor(() => {
            expect(
                screen.getByText(
                    /sorry, there was a problem submitting your message/i
                )
            ).toBeInTheDocument();
        });
    });

This test consistently failed, because all we ever got was the loading state. Yet manual testing passed fine.

The solution? Async the submitMessage() function:

            onSubmit={async event => {
                event.preventDefault();

                try {
                    await submitMessage({
                        variables: {
                            SendMessageInput: {
                                body: message,
                            },
                        },
                    });
                    setMessage('');
                } catch {
                    console.log(error);
                }
            }}

Without making this asynchronous it would always fail. Ah well. All's well that ends well."

Testing Packages Locally Using Lerna & Verdaccio

Tue, 01 Nov 2022 00:00:00 GMT

Published on Tuesday, 1 November 2022 , originally at https://www.webiny.com/blog/testing-packages-locally-with-verdaccio-lerna/

We want to provide the easiest way for developers to get started when building a new Webiny project. So we have a create-webiny-project command line interface (CLI) that creates a new user project with their chosen AWS region and some other preferences they need.

This CLI requests over 100 packages which are published to NPM. These all work together to create GraphQL APIs, the Admin interface, and infrastructure-as-code that allow our four interrelated applications to work together.

Modifying these packages and testing them could prove to be a challenge if it wasn't for very useful tools like Lerna, which we use to publish new versions to NPM, and Verdaccio. Verdaccio is a proxy registry. That means when configured it intercepts requests to NPM, and serves packages you've published to it locally.

Here's how we use leverage Verdaccio to test packages at Webiny, using our repository as an example.

Start Verdaccio

First of all, edit something in a package and build it. Then start Verdaccio. We can do this from the webiny-js project root directory using the following command:

yarn verdaccio -c .verdaccio.yml

The flag -c tells it to accept the config file we already have set up in the project.

Next we need to set the registry address to use the port Verdaccio is running on instead of NPM's (https://registry.npmjs.org) By default this is port 4873. Run this command to change the registry address:

npm config set registry http://localhost:4873

Did it work? To check, you can cat ~/.npmrc, you will hopefully see the following:

registry=http://localhost:4873

Tricks With Yarn

If you're running Yarn v2 or greater, Yarn no longer recognizes this config file. Instead you will also need to create a .yarnrc.yml in the user root folder with the following parameters:

npmRegistryServer: "http://localhost:4873"unsafeHttpWhitelist: - localhost

There's more information about this on Verdaccio's docs site.

Version and Release Packages

The final steps before we test our user project is to create a new Beta version and release the packages. This is where our existing setup in the Webiny project with Lerna steps in. You can see details of how this command works by looking at the package.json in our repository.

yarn lerna:version:verdaccio

Under the hood this runs the Lerna version command but doesn't push or create a changelog. It also creates a new tag for the release. If this is the first time you've run the command, the tag will be v{version}-beta.0, then v{version}-beta.1, and so on. Once this has completed you will see a new release commit on the branch which you will need to roll back afterwards. Now we can release the new version to Verdaccio.

yarn lerna:publish:verdaccio

This takes the existing packages with the appropriate tag and publishes them to our local Verdaccio registry. Please note that if you are running Verdaccio on anything other than the default port you'll need to run the full command:

yarn lerna publish from-package --dist-tag next --registry=\"http://localhost:{port}\" --no-verify-access --no-verify-registry --yes

Create a New User Project

The next step is to create a new Webiny user project. You can create it in a subdirectory of the webiny-js repo, or in another folder. We can use NPX to do this using the following command:

npx create-webiny-project@next test-project --tag next --no-interactive "unsafeHttpWhitelist":["localhost"]}' --template-options '{"region":"us-east-1","storageOperations":"ddb"}'

You can of course replace test-project with something more relevant, and set it to deploy to a different AWS region other than us-east-1 if you wish.

Once this has finished you should be able to run yarn webiny --version and see your newly published beta tag.

You can now deploy your project as normal and test any changes you made.

Tear Down and Roll Back

Once you have some changes you need to make, or you're satisfied that your changes won't break anything, you need to roll back your system and the webiny-js repo to it's previous state.

Stop Verdaccio
Point local registry back to npmjs.org:

`npm config set registry https://registry.npmjs.org``

Remove yarn config files

`rm ~/.yarnrc.yml# if you think you might need this file later, you can instead do mv ~/.yarnrc.yml ~/.yarnrc.yml.bkup``

Roll back the release commit

git reset --hard HEAD~

Delete the prerelease tag

git tag -d${tag}

Verdaccio: A Great Solution to a Common Problem

As you can see, Verdaccio is very easy to use, and in fact we have hardly spent any time configuring Verdaccio since it has been set up. For the most part, we've used the default configuration. Leveraging Lerna's ability to publish packages quickly and easy has meant that we can concentrate on delivering great software instead of spending more time on our tooling.

If it helps, we also created a video of this process:"

Testing Web Components

Tue, 23 Apr 2024 00:00:00 GMT

I've been taking a deep dive into Web Components recently: building a complex application. I have really enjoyed it. However, as with any project I want to deliver something that is proven to work by means of static, unit and end-to-end tests. I've run in to some issues with existing test suites.

When I deliver a new application I want it to be as well documented as possible so that any changes to it can be made as easily as possible, and by people who don't know the codebase as well as I would.

To that end, making sure that what I hand over is reasonably tested is paramount to my own sense of professionalism. It's only this that gives me a strong sense of accomplishment because I know it stands a good chance of lasting a long time, and it'll be easy for other people with less prior knowledge to work with.

When I deliver code based on Web Components, I want to deliver on that same level of quality. I know I'm leaning on newer browser-native technologies, but they're not that new.

I was therefore very surprised to find no adequate way to provide unit tests for Web Components.

My Testing Trophy

I've decided to adopt a standardised set of tools across all of our codebases so that non-specialist developers can drop into any of them with the least amount of friction that include the following:

Static analysis via JSDoc
Unit testing via Vitest
E2E testing via Cypress
Manual testing with Storybook

Most of these tools work really well, but one definitely doesn't...

Abstractions, abstractions everywhere

I think one of the problems is that for a large proportion of people adopting Web Components, they're here after having had experience with another framework, and looking for something equivalent.

That's definitely not true in my case. I was a little annoyed at the way React forces you to think about JavaScript, and I wanted to work more closely with the browser instead of seeming to force JavaScript into a certain pattern.

Fundamentally that meant that I wanted to remove the abstractions of a framework.

There have been well meaning attempts to make a framework out of Web Components. And I see why: If you're used to a framework, which a lot of people will be saying "here's another framework, but for Web Components".

The good thing about those frameworks is that they try to make something look a little more familiar, even when it's not. I looked at Lit Element extensively, and it definitely feels more like Vue. And they've got their ecosystem of tools and testing libraries and tutorials, which certainly helps adoption.

Again, though, that's not what I'm after.

I am a little freaked out going back to classes instead of purely functional components like in React. However there are benefits; understanding inheritance much clearer when you're writing something like

class MyComponent extends HTMLElement {}

There are other pros and cons, but I want to focus on the subject at hand.

Cypress: A Hidden Option

Ok well, not really hidden, but I did not realise I had to manually turn on a setting to enable Cypress to recognise the shadow DOM:

// in cypress.config.js

const config = defineConfig({
  // ...other config
  e2e: {
    includeShadowDom: true,
  },
});

Now I can write tests that include shadow dom selectors:

describe("template spec", () => {
  beforeEach(() => {
    cy.visit("http://localhost:5173");
  });
  it("should play", () => {
    cy.get("media-player#my-player")
      .shadow()
      .find('media-control-bar')
      .find("media-play-button").
      click();
    cy.get("media-player#my-player")
      .should("have.attr", "mediaplaying", "true");
  });
});

Great! Annoying if you don't know about it, but works fine once you do.

Storybook: Works fine

I've found Storybook particularly useful for manually testing that each argument I pass to the component works properly. And since Storybook re-renders the component when you change an input, you can see the player reacting to the arguments you pass it.

It is quite verbose. I have to pass my arguments around several times, but it does a good job once you've got those in:


// basic setup
export default {
  title: 'Media Embeds',
  // these allow controls to render for the arguments
  // here's the first time I'm declaring them
  argTypes: {
    videourl: { 
      control: 'text',
    },
    posterurl: {
      control: 'text',
    },
  }
};

// I found arguments needed to be destructured and
// passed explicitly into my component
// That's 2 and 3
const Template = ({
  videourl,
  posterurl
}) => `
  <media-player 
    videourl="${videourl}"
    posterurl="${posterurl}"
  ></media-player>
`;

export const Player = Template.bind({});


// Now actually declaring defaults for those arguments, 5th time

/**
 * @type {UserOptions}
 */
Player.args = {
  videourl: 'https://stream.mux.com/A3VXy02VoUinw01pwyomEO3bHnG4P32xzV7u1j1FSzjNg/high.mp4',
  posterurl: 'https://image.mux.com/A3VXy02VoUinw01pwyomEO3bHnG4P32xzV7u1j1FSzjNg/thumbnail.jpg',
}

There could be a way to reduce this, hopefully I'll be able to investigate before I hand the project over.

Vitest .. you're on your own

I shouldn't really say this is a Vitest problem. It would be the same if you were using Jest or anything else.

The problem really lies with DOM mocking libraries. None of them allow you to extend the HTML Element.

I have found a library of helpers that got me way further than I had with anything else.

Ficus is billed as a lightweight abstraction of Web Components. There's that word again. But they do have a range of other tools that have been helpful: paticularly a wrapper around JSDom that includes favourable setup for Web Components.

import { describe, it, beforeEach, expect } from 'vitest';
import { init } from '@ficusjs/testing'

import '../src/app.js';

describe("<media-player> ", () => {
  beforeEach(() => {
    init();
  });
  it("should render", async () => {
    document.body.innerHTML = '<media-player></media-player>';
    expect(document.querySelector('media-player')).to.exist;
  });
});

This still meant I have to mock a whole bunch of stuff to even get it to render. And this is as far as I have got. Not even the venerable, cross-framework testing-library covers the Shadow Dom.

I do have a lot of stuff I can unit test with Vitest. But there's so much more I want to do with it.

If you know anything that might help, please let me know.

I've also created a GitHub repo with examples if you're coming across this and are looking for resources for testing Web Components:

https://github.com/endymion1818/example-web-component-test-suite

Conclusion

More than anything, it's been a liberating, fun experience to see how far the web has come in the last few years. I hope I get to work with Web Components for a long time to come.

And I hope more developers can start using them too.

The Future is Headless

Mon, 10 Aug 2015 00:00:00 GMT

Published on Monday, 10 August 2015

If you've been in the industry for any length of time, the chances are that you've been in one of those discussions about CMSes that are never resolved. That is about to change forever ...

Content management systems have revolutionised web development. From the start of their existence, we haven't had web sites any more—we've had web systems.

It's the web's innovation which has contributed to the success of the medium that it is today.

But it does mean that we're always on a bit of a back foot, not understanding all of what is going on around us—because it's in a constant state of flux.

Today I saw a video from Drupalcon that I immediately thought: "That's the future."

What is headless? #

A headless CMS is just that - headless. It doesn't have a web front end. So there is no templating system to hack around. The CMS doesn't affect the web front end in the same way it does today.

It still runs the website, but in a drastically different way, using JSON data and REST.

Think about this: no more PHP mixed in with your HTML. No more overriding plugins that inject icons into your <head> that you're never going to use. Ever.

Sounds like a massive step up to me.

But there are many other advantages to serverside developers and organisations alike. Users can publish their content once and have it displayed on the app, the website and on syndicated sites all at the same time. CMSes can be updated without breaking the frontend ... because there is no frontend.

What it will ask of us #

With any technology change, we have to learn new skillsets. It takes time. But with such clear advantages, and the two biggest CMSes already developing headless systems (Drupal and Wordpress) it might be time to start that road.

Instead of frontend developers having to also know PHP, we will need to concentrate on how to use JSON data via REST. With a frontend framework such as Angular or Backbone, this becomes ... easier.

Actually, I'm having trouble writing this section here with confidence ... I'm still working out how it all comes together. But I don't think that's a bad thing.

Sit through this video, if it blows your mind like it did mine, then we have an interesting road to travel ahead.

Full video"

The Art of Compromise

Thu, 08 Aug 2019 00:00:00 GMT

We developers are inherently precious about things. Our tools. Our technologies. But we can't afford to be precious about those things too much. In fact, working in cross-functional teams and with other teams means that we sometimes need to compromise.

I was seething.

So upset that, I have to admit, I came close to criticising someone whose opinions I really value, and who is a more skilled developer than I. This was because they said they didn't like the way I was leveraging the CSS cascade: "It makes components leaky and not well encapsulated".

I love the cascade, and have been using it to pass down colours to components instead of re-declaring colours on each paragraph, heading and link inside nested components. I thought this was an approach that everyone should take: use the cascade to our advantage.

I nearly voiced some critical opinions to my colleage. Then our designer spoke up. "We need to stop discussing this between just two people. Involve all of the other developers working on this set of components. Then you can reach consensus and all be in alignment."

He was right. Everyone has an opinion, and whether one is correct or another is isn't up for debate. What truly matters is what the consensus is.

Compromise is a strength, not a weakness. And on any project that involves more than one person (which is, like, all of them), it's the only way of moving forwards."

Things I Learned By Working for Free

Sat, 14 Mar 2015 00:00:00 GMT

Developers sometimes get asked to do stuff for free. If that happens, it generally can be a warning sign that would be right to steer well clear of. It's easy to end up feeling cheated. But I've been fortunate to be involved in two projects that, although I earned no money from, I felt that I benefited greatly.

Project 1: Edifice #

Edifice came about from the need that a large voluntary organisation had for tracking teams, projects and skill levels on construction projects throughout London, Hertfordshire and Oxfordshire.

The lead developer, a personal friend, invited me to join the project initially as a front end developer. When I started, my friend was training a largely unskilled workforce in a huge range of different abilities that he needed for his project, including HTML, CSS, Java, JavaScript, and others.

I at once suggested that to save development time he could use a preexisting framework like Foundation or Bootstrap, which would allow the 3 or 4 skilled developers to work on the rest of the Java MVC web app, whilst the less skilled could work on refining the front end views.

Working on this project gave me an opportunity to learn first hand from a former university lecturer about so many subjects. Among other things, I learned how internet protocols work, why we use relational databases instead of spreadsheets and loads of other stuff.

I also got to work closely with another experienced programmer from whom I learned how to map processes and from these write user stories, how to use Git and Github, and even how to format my code properly.

Yes, the project was for no monetary value, but was genuinely instrumental in my career.

Edifice on Github

Project 2: freebabylon5.com #

I volunteered to build this website because - as you'll know if you follow me on Twitter - I love this 90s TV show. The fan campaign to get the show back on television is run by one stalwart fan, and they needed a home website.

I built the site on Bootstrap and Wordpress, very quickly. I think it was in about 3 weeks in my spare time. Whilst it was fun, I didn't do anything spectacularly original during the build.

But after the project went live, I tracked user flows and kept in touch with the fans via social media, and iterated on the site. It was great to have fan feedback and suggestions, and to see when I changed something on the site, what the results could be.

Being motivated to refine the site after the go live taught me so much about how a website is a living document that can change to fit the changing needs of its users.

freebabylon5.com

The Internet - Built on Free #

A huge chunk of the progress in web standards, in the ability to do incredible stuff with websites, in what we know as the internet today came about by dedicated people who were willing to invest their time and effort building the community and constructing the things we benefit from each day.

So-doing stuff for free, generally a bad idea. Unless there's some personal gain or it's for a cause you genuinely care about and are willing to invest in.

You may be surprised what you can learn from getting involved in many incredible projects that really need your support."

The WordPress 'Implosion'

Thu, 17 Oct 2024 00:00:00 GMT

Many years ago I moved away from building things with WordPress. But it's time I tried to help people understand what I think the current situation means for people still in the ecosystem.

If you haven't heard about it yet, try entering "WordPress" and "WPEngine" into a search engine.

Some 8 or so years ago I was fairly invested in the WordPress ecosystem. I liked how empowering WordPress was as a foundation for people without a technical background to be able to create their own content-rich websites.

However when they introduced the "Gutenberg" editor, I read the writing on the wall.

An editor that empowers ... who?

Gutenberg was a new editor environment that focused WordPress heavily on the end user. It allowed users to create more dynamic pages and to edit more content that would have been previously outside of their ability to change (headers and footer content, particularly).

Their decision to merge Gutenberg into WordPress Core brought WordPress much closer to their user base. That's a decision I applaud: its great that they want to be the ones to empower non-technically minded people.

For me, I realised it would be incredibly difficult to support what the editor allowed users to do. I couldn't standardise even a simple thing like a button, because the user could override it using Gutenberg. I couldn't standardise templates, because Gutenberg could inject layout blocks that didn't work on the template I had created.

It was clear from this that WordPress as an organisation didn't care about the intermediaries: digital agencies and other technical organisations. WordPress didn't value us. It was instead built for the far broader, non technical user base. That was all it was built for. I don't think we were even a consideration.

At that point, I decided that I was out. I knew that my technical decisions would continue to be impacted negatively by WordPress as an organisation.

I moved on to have a successful career with JavaScript and left the ecosystem behind.

And hearing all that's been happening, if you're a developer or own a digital agency or other technical company, one that builds sites for end users with WordPress, I encourage you to do the same.

Digital agencies are not WordPress customers

Digital agencies and technical organisations that deliver sites built on WordPress are not who WordPress sees as it's customers.

Therefore the WordPress organisation will continue to make decisions that could easily have a negative impact on your technical decisions for years to come. Impacts that will erode your reputation and your relationships with your users.

Instead, I encourage you to build with some other tool that is built for technical organisations, where we are it's customer. And therefore your customers will continue to see you as those who support them.

What should I use instead?

There are plenty of good alternatives. Ones that are small and ones that support massive content rich sites. I'm not even going to mention any them by name here; they're one deft search string away. If you know me already, then you know which ones I recommend.

If you're invested in the WordPress ecosystem, I encourage you to build commercial themes and plugins for WordPress: Support their ecosystem and let them support their users.

If not, it's time to move on. The heyday is over. Let users be users. And let WordPress be WordPress.

Three Conversations to Serverless

Fri, 09 Nov 2018 00:00:00 GMT

Serverless isn't such a new concept but it is still difficult to understand for many of us. I've been privileged to work with some great people over the past few months, and I've really been helped to understand serverless because of the questions they've raised, many of which I couldn't answer straight away. Often their questions have got me thinking about different ways to explain serverless.

Since many of these people are used to managing, developing or supporting a project that's more monolithic in nature, I'm going to try to write down some of those questions in the form of the paths they often take: realisation, clarification, and comprehension. If you find you don't understand the concept of serverless after these 3 "conversations", remember that the idea of 3 conversations is perhaps a generalisation. One of my colleagues, a much more senior developer than I, was persuaded of the approach after several weeks of discussions!

1. Realisation #

Often, people in my organisation come to me with a requirement for the project, or a solution to an outstanding problem, with an idea of how they would implement it. This is because we are often predisposed to solutions-based thinking. There have been studies of the brain to show we are likely hard wired to do this. It helps us to act quickly based on our previous experiences.

However, with serverless there's usually less common ground with previous experiences we might have had. Often, during the conversation this statement comes up:

"there is no backend"

When I reach this point in a conversation, it's the point that the listener realises the objective they currently have isn't possible in the way they have been thinking.

Whether it's to dynamically render content, process form data, implement redirects, or modify content using a CMS, this statement stops them short. There is no back end to our static frontend that is, in our case, hosted on a dumb S3 bucket and uploaded to a dumb CDN. I mean "dumb" as in there is no processing ability there we can leverage.

Usually, this realisation prompts some critical thinking. "How can I solve my problem with this restriction?" And that's when it starts to get a bit frustrating. Which means you might need some clarification.

2. Clarification #

You, the listener, have probably been mulling over your problem for a while now, and whilst you realise you can't stop a train that's in progress you might be thinking this whole serverless thing is a ludicrous idea.

Why would anyone build apps this way? Why make it so complicated?! When you had a monolithic app, it could do anything you wanted out-of-the-box. Now you might be finding out things. Things that might have been unknown to you before. For instance, form data needs to be processed server-side before it's sent anywhere. That a lot of 3rd party services require auth tokens, which we can't use in JavaScript because it can be read and used by anyone.

This can be frustrating. These paradigms are hard to come to terms with, at least initially. But with each restriction there is a solution that comes with some really interesting benefits. And that's when clarification can turn into comprehension.

3. Comprehension #

Having gone through this process of realising we are dealing with something very different from what you might have imagined, to the clarification that you can't approach your requirement in the same way you have done previously, now we can start to solve the new problem.

When we start doing that, we can find there are some really nice solutions that add a layer of curiosity and, you might say, delight, as listeners come to find out more about the options they have.

Heres a few of the "lightbulb" moments I've enjoyed witnessing over the course of the past few months, credited where I can remember who it was:

"Ah so there's more security and better performance, that's good then."

"So you can get that data from the API at build time, and update it if necessary on the frontend. That's useful." (Tim)

"[to other stakeholders] ... and then anyone else can consume this service if they need to, it's already built for them."

"I can see why you wanted to use Gatsby for this project. GraphQL is ... is really useful actually" (Awais)

"So you can do that even before the site gets to the user? And it would be one request?" (Jonny)

"[After we reported straight As in webpagetest.org] Really impressive lads! Good work!" (Tomas)

This has been a lot of fun for me. I have particularly enjoyed helping others see things in a different light, or with renewed potential. It's great to see people get excited about something so much that they want to go away and do something with it.

Serverless is still only one method to approach development. It's not always the best way, but it is a viable solution to certain situations and use cases. But it really does take time to come to terms with the idea. Teams you interface with also need to be aware that there are things that are going to be different and some problems may require a very different problem solving route.

Static sites in particular shouldn't be made to do everything. Sometimes complexity on the frontend can be taken too far.But we have managed to build a site that is robust, fast, beautifully designed, deceptively simple, and that has some useful microservices that can also be utilised elsewhere.

Working with serverless is definitely my thing. I'm really looking forward seeing more implementation of this approach in the future."

Three UI Component Conundrums

Fri, 30 Oct 2020 00:00:00 GMT

React combined with Styled Components is a flexible and maintainable way of composing reusable user interfaces for the web. However, every set of tools has as well as awesome features, a few tricky problems to solve. Here are three that I've come across recently.

Some conundrums #

1. CSS inheritance: The conundrum #

The first issue concerns encapsulation. Broadly speaking, the main differing opinions on this argue that the results of your functions should be bound to that function only, and not have any effect on anything outside that variable (this is encapsulation), versus the idea that it's OK for things to inherit, or to have an effect that carries on across multiple functions.

I'm weighing into this discussion on a different level to most. In this post, I'm not talking about encapsulation and inheritance as it pertains to object-oriented programming, although the principles are the same. I'm chiefly interested in UI components, and here I'm going to talk about these principles and how they've impacted my recent work.

The basic tenet of CSS assumes inheritance. It's in the name: CascadingStyle Sheets. If you started your web development career from this angle, then you're probably quite used to seeing things like:

div {
  color: lightblue;
  background-color: green;
}
div.banner {
  color: green;
}

The div with the class banner will inherit the background-color from the first div. This is a normal way of writing CSS ... if you've had a certain type of introduction to CSS.

At work, I'm using an internal library of components written in Styled Components. This has some really distinct advantages: it's very simple for developers to keep within the brand guidelines, and even very basic elements are encapsulated, that is, they can be composed together without any unexpected side effects.

But this has also led us into some complications... here's the conundrum and solution for two of them:

Here's a real-world example in React using the CSS-in-JS library Styled Components:

// declared in an external library
const Text = styled.p`
    color: lightblue;
`

// declared in an external library
const Card = ({ children }) => (
    <SCard>
        <Text as="p" color={default}>{children}</Text>
    </SCard>
)

// my override
const SCard = styled(Card)`
    p {
        color: green;
    }
`

In this example, no inheritance will occur, because the style applied directly to <Text/> has higher priority than the color declaration on <SCard/>.

So, how do I make the text in <SCard/> the colour I need it to be? At this point, I have to either set the API of the <Text/> component to recieve a color property (if you have access to that, which I didn't), or override the colour in <SCard/> using an !important declaration.

1: CSS inheritance: The solution #

In this instance, our solution was to change the API of <Text /> so that by default the colour was inherit. This made it possible for both approaches: inheritance where necessary using CSS, or by adding a colour directly to the Text component using it's color property.

2: Large pages: the conundrum #

I'm not sure what the average number of DOM elements on a page is. We have some pages that have a lot of elements, around 1,100. Just about every element is a separate React.Component, and due to design variation, we extend a lot of them in a similar way to this:

import Card from "component-library";

const SCard = styled(Card)`
  background: lightblue;
`;

This seems to wrap the component in a React.Context that serves to keep track of the original component ... after all, if the original component changes, how will we know to update this extended component?

On the pages where there are a lot of these extended components, we are seeing a significant drop in performance. I'm assuming it's because of the extra dependencies involved (1. the original component, 2. the React.Context wrapper, 3. the new component).

2: Large pages: the solution(s) #

We haven't solved this yet, but I found it particularly interesting that there was a significant drop in performance, proving to be a noticeable factor in how many conversions we see from those pages when compared to a previous iteration of the application.

I can think of a few options:

Switch to a css-in-js solution with lower (or zero) runtime, like Linaria
Keep Styled Components but export CSS strings alongside the components so that we can extend them using CSS only (in most, the components' APIs don't change significantly)
Rationalise the design so that it uses styles that already exist in the library

What would you choose?

3: Markdown content: the conundrum #

This one is related to the above, but we are also using these components in Markdown content we pull in from our CMS. So for each <p> tag that's generated from the Markdown, we use our libraries' <Text />component.

This hasn't been an issue yet, but I can see one day we're going to have an extremely long page of content that uses many elements that also impacts page performance.

3: Markdown content: the possible solution(s) #

We're probably going to have to use one of the 3 solutions in the previous conundrum to solve this one too.

Conclusion #

I have tremendous confidence in css-in-js, and believe if you're using React, Styled Components is a great, broadly performant solution that gets most people where they need to be.

However, it has to be noted that there is a runtime overhead that increases with every component you extend.

I am interested to see if there's a way of compiling all components at build time and avoiding that React.Context link, since in our case we don't expect the original components to change after that.

And I'm interested to see whether other solutions, like Linaria, can provide a solution we can utilise in the future."

To Rome, a Poem

Mon, 23 Nov 2015 00:00:00 GMT

Published on Monday, 23 November 2015

I wrote this poem more than 10 years ago, but I've never published it anywhere. I've always been afraid it's not any good or that it's of not enough interest to warrant any merit ... but that seems to matter less now. Anyway, here it is!

I wrote it after I read Andrew Motion's biography of John Keats. I was so moved by the tragic life Keats had and how he'd created some of the world's best poetry despite that.

To Rome #

<div class="[&>pre]:ml-0! [&>pre]:mr-0! [&>pre]:bg-transparent!">

It's a rose you've given, and a rose I'll take
  As I leave you awash in my wake.
Though I travel to distant shores
  I'll remember that rose that it was yours.

Now it's the third day out to sea
  And closer the shores of Napoli
Today I feel a lost cause
  Remembering that rose, that it was yours.

Never can an absolution be found
  This ship of ours has run aground
And climbing the steps to the Piazza's palace
  I lift to you a golden chalice:

Not Apollo's goblet, nor Endymion's cup
  Only for you would I offer up
The dregs become of this earthly root
  That my soul might sprout forth a shoot

And I might live again to see
  That rose you gave, you gave to me.

</div>

Notes

I imagined that when he left his lover and former fianceé, that she gave him a rose. But the fact is, Keats left the country without even saying goodbye to her, despite the deep emotional pain that caused him.

I imagined him drinking a cup to his lover, but wanted to mention that it wasn't a glorious cup of a god like Apollo (Keats never saw himself that way), nor was it Endymion's (who lived forever).

The dregs become of this earthly rootThat my soul might sprout forth a shoot

These 2 lines are from the poem Endymion, by Keats, and allude to the way his romance with Fanny Brawne made him feel."

Tools for Production Ready Code

Wed, 14 Oct 2020 00:00:00 GMT

Published on Wednesday, 14 October 2020

A note about this article: This was a feature I wrote for Net magazine which was sadly closed down before it was published, so I published it on dev.to initially. It's aimed at people who want to get into devOps, or product owners who want to understand what it is they're getting themselves into. I hope it proves to be useful.

Websites are great tools for communication, but it's remarkably easy to take them offline. One of the most panic-inducing moments for any operation can be that first call from a customer informing you that the site is down ... and then the next call, and then fifty more ... suddenly the company grinds to a halt until the problem can be fixed.

Downtime might not be much of a problem for every site out there. If you take down your own personal blog for a few hours, it probably doesn't matter a huge amount. However, most companies and other organisations now depend upon their web sites being accessible 24 hours a day, 7 days a week. Any downtime for these organisations results in loss: the company could be losing money, critical information might not be reaching the right people in a disaster zone, or at the very least, the organisation's reputation is being eroded, minute by minute.

For those organisations in which there's a huge financial investment and thousands — perhaps millions - of customers, reliability is a huge concern. There are now a large range of tools, products and - above all - strategies - for ensuring that if (or rather, when) something untoward happens, the right people can resolve the situation as quickly as possible.

In this article we're going to provide some answers to these questions:

How can we as development teams continue to offer high levels of confidence?
How can we trust that new features are going to work as expected?
What can we do to minimise issues that do occur?
What steps can we take to make sure any unwanted surprises can be fixed quickly?
How can we make sure that the duration of any downtime is as short as possible?

We have at our fingertips a wealth of tools that can be added to a code base, and a process flow, to catch issues before they reach production ... and if they do reach production, to minimise the impact.

The strategies and tools we're looking at here cover both prevention and resolution of problems that could occur to any web site, but with a focus on full-stack JavaScript.

Situation: When JavaScript fails #

JavaScript is a dynamic language. It's great for coercing strings into numbers and back again. But sometimes this causes issues. We have to make sure that the values we're working with are of the same type, otherwise one error in the JavaScript code could break the site entirely.

There have been a number of notable projects that make an effort to deal with this kind of problem by turning JavaScript from a dynamic to a statically typed language:

TypeScript, Flow or Reason #

TypeScript and Flow are the two most popular ways of turning JavaScript into a statically-typed language. Both approaches are a superscript of JavaScript, which means your code needs to be transpiled back into normal JavaScript in order to work on the web.

TypeScript is run by Microsoft and is developed for building applications at scale. Flow is a great alternative with a similar api.

Reason is a more interpretive system that allows you to write very succinct type-safe code. There are a lot of features to this language that mean it can appear quite different to the JavaScript developers are used to writing.

Neither tool is perfect: each of these languages still needs to be compiled to the kind of JavaScript that runs in browsers. Developers need to also adopt strategies to ensure that compiled code still coerces values correctly by writing functions that check types where it could be of an unexpected type, for example data being requested from a 3rd party via an API.

Situation: When things aren't working as expected #

The development team stand back proudly, showing you the work they've done over the last three months. Their pride quickly melts into dismay as the stakeholders say, "It wasn't supposed to work like that...!" Suddenly, the team has some costly refactoring work to do.

Assumptions are a development project's worst enemy. The more assumptions we make about how something is meant to work, the more risk we are creating. The best way to avoid assumptions is by investing time in gathering requirements and writing up clear documentation before any development work starts. If the documentation is loose, unclear or even non-existent, there will be a higher chance of something not working as expected.

"“Hell is other people’s undocumented assumptions.” — Nicole Fenton

Project Management Tools #

Project management tools such as Trello, JIRA, Monday and others can be set up so that requirements can be clearly documented before project work can begin, and work can be broken down into manageable chunks. Some things that should be included in the documentation should include:

Feature requirements
Scope (what it should not include)
UI Mockups
Testing approach

Documenting the features as a team can be instrumental in reducing undocumented assumptions, allowing the developer to write code relevant to solving the task in scope.

For more information on this strategy, see the box "Documenting Features" below.

Supplementary info: Documenting Features #

If you are able to find an issue early in the process then it costs far less time to fix than if you found it later on. For this reason, good documentation is key to having production ready code.

Documenting a feature should include acceptance criteria, which forms the basis for integration or E2E tests, and can be written in the form of given - when - then statements, for example:

Given I am using the websiteWhen I visit the signup pageThen I can create an account

Here's another:

Given I am using the websiteWhen I visit the signup page and have forgotten my passwordThen I can reset my password

The first part of the statement, Given, may sound a bit redundant at first, and it could be tempting to miss it out altogether. But it really helps to frame the feature and can become very useful as a reference point further on in the process.

If the E2E tests fail against these acceptance criteria, the feature can be fixed or rethought.

Unit tests #

Unit tests using a testing library (such as Jest, Mocha, Chai, Jasmine or perhaps something else) are the foundation of developers being able to ensure things are working as they should. How you adopt this approach can vary, but production-ready code means there are at least tests for each possible outcome of the component.

TDD, or test-driven development, means that a developer will write tests before they write the component they are building. This helps with conceptual thinking because the developer can have each outcome clearly in mind before they start writing code. This often results in less code refactors, and more optimal code that doesn't have as much legacy or unnecessary lines.

For TDD to be effective, the component to be built needs to be planned carefully first.

Situation: When your users do something you didn't expect #

The phrase "expect the unexpected" is most apt when used in connection with software and those who use it! Quite often, people using your software will be able to use things in a way you didn't anticipate. The best way to avoid this is to invite people to use the software before it's released to everyone.

Quality Assurance testing #

QA Engineers can be viewed as the secret superpower of a development team. It's surprising how many times a good QA Engineer has saved a developer from themselves, or saved the development team from its own assumptions ... or even saved an entire business from its own good intentions. An on-hand QA Engineer is invaluable to a development team, but there are organisations you can contract who will run through your app and uncover issues you would never have been able to think up.

Two of those companies are Global App Testing and User Testing. Both only have subscription plans. There might be others who can offer a one-off test of your app on a single payment basis.

User acceptance testing #

Some companies have a "UAT" environment that serves as a staging ground for code that is a candidate for release to the production site. They can then use that environment to organise a group of people to come in from the public to give their opinions on a feature to be released. This gives some qualitative assurance that the feature works and is going to benefit the intended audience. There are a number of companies who organise the individuals to come and test the software, which would involve a fee for their time.

To be safest, it's best that everything in the stack - the database, the external functions, the frontend, the authentication server, is duplicated for UAT. That way, the complete user journey can be tested in isolation from production. The result is that any errors in the code won't accidentally knock out a production service. UAT should be a safer environment for developers to take necessary risks.

Multivariate, or A/B, Testing #

As well as qualitative testing (validating by quality), quantitative testing (validating by quantity) is also a good way of decreasing risk that a new feature could be causing a problem. Once Google Optimize, Optimizelyor a similar tool are integrated with your site, you can release a new feature to a segment of your potential customers for a set period of time.

Once there has been a significant enough amount of traffic, you can analyse the results and check if conversions are increasing, decreasing, or have stopped altogether, and if the results aren't positive, you can turn it off so that no further losses occur.

User behaviour monitoring #

Getting an insight into user behaviour is invaluable to any experience. Tools like Hotjar, sessioncam, CrazyEgg and others record where the user clicks, and what journey they took through your site, which is particularly useful when introducing new user flows or features. However, these tools have a performance cost, and in my view shouldn't be left on indefinitely.

Analytics tools such as Fathom, Google Analytics and others can also be a good source of insight into what could be breaking user flow. Some can even be set up to report activity you want to be monitored, for example if the number of 404 or 500 errors increases above a certain amount in a day or hour.

Situation: When components clash #

The components we build for our websites might be perfectly built in themselves ... but what happens when we mix them with other components? This can often cause unexpected side effects that can crash your site or result in other issues or errors.

For this reason, it's best to check beforehand how they integrate with other components, and whether your visitors can actually complete the tasks you want them to.

Integration testing #

Often written in the same way as unit tests (see above), integration tests can be written for several components that are known to be used together, for example: input fields and a submit button.

End-to-end testing #

Tools like Selenium (which is cross-platform) and Cypress (JavaScript) can test a user flow from beginning to end (hence the name). This type of tool is critical to ensuring users can complete their journey from first visiting a site, to checking out and paying for a product, or any other journey you might want someone to take.

End-to-end tests can often be written by the developers but the journey, and each possible outcome, need to be planned by the development team.

Situation: When you need to roll back quickly #

Exceptions in an application that are not caused by malicious intent or load are often caused by new releases. Debugging in production isn't something you ever want to do, so once the cause has been identified it's wise to roll back to the previous release. Then the development team can continue to work on a fix without impeding the business.

Tools that store each built version of an application, such as JFrog's Artifactory, can ensure that the time to rollback is kept to the absolute minimum. When this has been configured, you don't need to rebuild the version you're rolling back to, which could take from several minutes to potentially hours.

Situation: When you need to understand what the problem is #

The development team won't be able to fix the issue unless they clearly understand the problem ... that's why it's important to implement different types of monitoring, both on the app, and separately too.

Application Reporting #

Logging errors to the browser console is the most basic kind of application reporting, but that limits debugging to only the issue that occur on one person's machine ... what if you want to see what happened on someone else's?

Tools such as Sentry, LogRocket, BugSnag and a plethora of others, allow you to send errors to their platform, and from there to Slack, Email or to a telephone system.

This way, whenever there is a serious error in production, the appropriate people can be notified immediately.

However, some issues are not immediately obvious and require more monitoring over longer time periods.

Application Monitoring #

Tools like Splunk, DataDog, Honeycomb or AWS X-Ray allow you to log other useful data so that you can closely monitor the activity of your application or distributed systems.

A word of caution: it's important to be careful that no personal data is being logged without users being informed and consent obtained.

Knowing what needs to be logged and what doesn't takes time to discover: be prepared to keep adapting your strategy until you log useful data ... and only the useful data!

Health reporting #

Of course, if your site is throwing a server error (HTTP 500 code), no amount of in-app logging is going to inform you of that: your web app is down! For that reason, external services can help monitor the health of your app, and inform you when something happens.

One of the most basic of these is Uptime Monitor, which pings a site at regular intervals. There are more sophisticated tools like AWS Cloudwatch, Pingdom, Retrace and others which allow you to analyse your site in real time and send out alerts to your team.

Some also take the view that automated QA tests should run in production too, so that you can fully grasp the state of the production app. This is really helpful especially if you can't replicate your testing environment identically like your production environment. But we need to be careful: if we have a monolithic app, the extra load created by the automated tests could cause crashes at peak times. If it's a serverless app, you could find you've incurred extra costs for runtime.

Situation: When you're under attack #

This kind of issue is out of scope for this article, but I feel it's worth mentioning a valuable resource that has become the go-to handbook for modern DevOps teams, and is highly valued for its insights into the way Google manages applications at vast scale: the Site Reliability Handbook, published by O'Reilly Media. It's available to read online here: https://landing.google.com/sre/sre-book/toc/index.html

Supplementary info: How to Manage Incidents #

Despite everyone's best efforts, failures in production will occur. There are strategies and tools to help reduce the downtime and get your services back up and running more quickly:

Step 1. Define a severity level #

When an incident occurs, it's everyone's first priority to get things working again. However, defining different levels of severity helps everyone understand the impact it's having on customers and the organisation: some incidents will naturally have higher impact than others; some will have no immediate impact but could still require an incident level response.

Step 2. Delegate and manage communication #

Having a dedicated incident management team is invaluable when there are issues in production. Otherwise, someone should be assigned to keep stakeholders and other interested parties up to date with what's happening. Splitting this responsibility away frees the developers to focus on fixing the problem and informs other developers who are not involved to stop them from jumping in with well-meaning patches — patches that could make the situation worse.

Step 3. Arrange post mortem meetings #

Once a situation has been resolved, post mortem meetings can be very helpful to identify what the problem was and put things in place to avoid a similar issue from arising again. Such meetings aren't focused on blame; rather they should be about understanding the problem and how to avoid it happening in future.

Supplementary info: A word about code reviews #

One of the best ways of catching a range of potential issues early on in the development process is by the use of effective code reviews. Having two or more developers that are familiar with a codebase and that use it regularly review each other's code means it's going to reflect a more well-thought out and well-structured project. Good code reviews can also point out flaws in logic and gaps in tests that might easily slip through without notice if there was just one developer looking after a project.

Conclusion #

The web is still a new frontier, the boundaries of which we may never fully define. Its great strength is in diversity and potential to be used in a variety of ways: from just one individual to a small family unit; for a community or enterprise, to a tool for the betterment of humankind.

But with great power comes great responsibility. I hope some of the strategies we've looked at, and the tools to mitigate them, help your organisation to provide stable, predictable production-ready code in the face of ever-changing challenges of the expanding, mutating and growing world wide web."

Towards better hydration

Sat, 08 May 2021 00:00:00 GMT

Published on Saturday, 8 May 2021

I came across a tool recently that I'm really excited about ... it's something that I think can solve one of the most pressing concerns in JavaScript development: rehydration.

Imagine how orange juice is transported: the water is removed to reduce the quantity, and water is added back - "rehydrated" - at the destination.

A similar thing happens with a subset of modern JavaScript frameworks: they build static HTML pages (either when the application is built or on the server) send that down to the browser. Then they send and parse a JavaScript bundle that also contains the layout of the page, except this one is interactive (hence "rehydrated"). It's that JavaScript generated representation of the HTML you see when you visit the site.

These frameworks have really caught on because they allow for the greatest expressiveness possible without splitting concerns: they allow developers to write components instead of separate HTML, CSS and JavaScript.

But I think there is a fundamental problem with this idea that I think needs to be solved before we can scale our applications for the next generation of frontend architecture.

I have just come across a tool that makes me think we're much closer to finding a suitable answer to this significant issue.

The issue #

JavaScript is inextricably tied to the other building blocks of the web: html and css. There's one single line of code that has allowed more and more control to be given to JavaScript:

const root = document.getElementById("root");

If you've spent any time with JavaScript, you'll recognise that this is where it takes over the DOM, rendering pages in JavaScript instead of native HTML, facilitating our single page applications.

But this creates a few issues:

Nobody can see the page until the JavaScript is parsed and that version of the page renders (even if we're using static or server rendering),
Bundle sizes grow larger and larger dependent on the requirements of the application, meaning that the JS pages take longer dependent on the bundle size (and therefore the complexity of the project),
We also create performance issues such as time to interactive delays, or poor interactivity for people on lower-powered devices, because JavaScript runtime takes more processing power (not to mention battery energy) than plain old HTML.

There have been three main ideas around how to solve this up until now:

Solution 1: Static all the way ... #

The first answer to this problem is static-site generation: use JS to compile in the pipeline, and serve only static HTML to the browser. Certain static-site generators (like Hugo, Jekyll, Eleventy etc), excel at this.

The advantage of this approach is that HTML is still the fastest and best way to serve up content.

However, when you want to add any dynamic functionality, you need to load on a separate JavaScript framework, like Vue, Alpine or React. Some static site generators bundle this functionality on, some just focus on statically generating pages.

I think the problem still remains: the static site and the javascript are separate entities, and it could sometimes be challenging to keep them in sync.

Solution 2: Rehydrate all the things #

In sharp contrast to the static generators above, Next, Nuxt, Gatsby, SvelteKit and others send down HTML and then rehydrate the entire page in JavaScript.

This has some cool advantages, such as literally no overhead when making components dynamic or interactive, because, at the end of the day, the whole page is already rendered in JavaScript.

But ... and this is a big but! ... it's expensive.

Every page has both HTML and JavaScript counterparts. This means bundle sizes are larger and browsers are working harder to process JavaScript for scroll position, routing and even CSS styles.

Solution 3: Splitting traffic #

I've been trying to figure out how we negate these problems, because until we do, we face an impasse which prevents us from scaling our applications as we want — and from reaching broader audiences.

(Remember, the next billion web users come from countries where conditions are a lot less ... guaranteed).

My attempt at this idea is what I currently do on my website: I allow people the option to visit a statically-rendered HTML and CSS site, or to visit a JavaScript SPA rendered with Preact.

There's a variation on this one which I've been thinking about on and off for a while: using an edge handler (such as those Cloudflare or Cloudfront provide, short running standalone functions) to split traffic (or transform websites even before they get to the viewer) based on some sort of browser preference setting. With this idea, we could route people based on one of these three settings:

device has low battery
device has a poor network connection
user has enabled a setting preferring less JavaScript

The trouble with each of these in turn is that (1) battery status can be used to track people against their wishes, (2) the state of the network isn't typically known until the site has finished downloading, and (3) this one gets awfully muddy awfully quickly: all JavaScript? That would be like saying turn off the browsers' ability to drop cookies: loads of services normal people use every day would be useless.

So no, splitting traffic isn't a good option. It may be fine to do that on small blogs like this one, but it's not a viable option for sites when some complex functionality is required .. or even some simple functionality: my HTML-only site doesn't have animations or a search tool.

A solution #

What we need is something in-between these options. Something that can build HTML statically (or on the server), and partially rehydrate elements on the client where that is needed.

This isn't a new idea. The core React team were hoping that concurrent mode would allow you to do this. However, building up a DOM tree is pretty difficult to achieve asynchronously.

Massive steps towards this idea have been made by NextJS with server rendering (building pages on demand), and with SvelteKit's optional rendering (which allows individual pages to be statically rendered as HTML or JS) ... but it's still happening on a per-page basis, meaning that for those pages: there's either no possibility of interactivity, or there's a JavaScript overhead. Sometimes both.

This idea of partial rehydration, or "Islands architecture" by Jason Miller (creator of, among other things, Preact), is something I had really hoped we'd get to at some point.

And now it seems we are here. This talk at Svelte Summit 2021 blew my mind.

Astro is the first tool I've come across that has this ambition: to facilitate importing and selectively hydrating components in your page as you require them.

And Astro goes further: we can share state across components, or potentially use different modern frameworks (react, svelte, Angular, you name it!) as we want, and we can request JavaScript to only load when the component is in the viewport.

This has the potential to solve all of these problems:

No more "uncanny valley" of pages being visible but not being interactive
No large bundles being loaded upfront
We could even give the user control over whether they want to load certain components

Astro is in early alpha, and seemingly in stealth mode too! I'm so eager to get building with this tool now, but it seems I'm going to have to wait a while before the team are ready to open it to a public beta offering.

This could be a really powerful tool, and fundamentally improve the way we build for the web. How exciting would that be!

PS. slightly more detail here on Astro's website

PPS. Of course Chris Coiyer has already written a post on CSS Tricks about Astro"

Tracking video plays

Thu, 16 Nov 2023 00:00:00 GMT

The scope of this project was to track how much of a video has been watched periodically, send the data to an API at regular intervals, to be able to track different video providers (YouTube, Vimeo and others), and to track multiple videos on a page, all of which could be playing simultaneously.

It was a little tricky because each player sends different events in the course of a video play, for example when the video has been paused, that we would need to respond to.

Also to provide continuous tracking we sometimes need to call the player to return data, such as the video title (which can sometimes change mid-play), and the duration of play that has elapsed.

This was a rebuild project became necessary because the original codebase was very difficult to modify. It had the following issues:

The code had been written very poorly
There were no tests or types

In addition to the above, I wanted to make the new tracker with such good developer experience that it would be as easy as possible to understand the code. The aim was to leave it open to modification by other developers. This would increase the longevity of the tracker and reduce the time investment over the lifecycle of the project.

First Principles

We had already decided to adopt JSDoc instead of TypeScript for our codebase. I wrote a separate article about this if you're interested.

In addition, I have been standardising all of our codebases to use ESModules and Vite, so it made sense to opt for Vitest for our testing suite across all projects.

The other thing I wished to change was that this script was hosted on a domain as a text file, and from there pulled into applications that needed it. I didn't think this was robust enough for such a key part of our business, so after discussion we decided to bundle it as a package.

This meant it could be versioned, and any rollbacks that were necessary could be done on a per-application basis and fixed forward instead of rolling back the tracker for every application simultaneously.

The other advantage to this is that changes can be rolled out gradually instead of launching on all of our codebases at the same time.

The application would also not be bound to the global window scope. I didn't see any advantage to doing that. Instead, it would be called in the consuming application and maintain it's own context.

Setup

With those rules established, I implemented the initial API for calling the tracker, which would look something like this for a HTML5 video:

<video id="my-video">
  <source src="video.mp4">
</video>

You would initiate tracking by passing the element's id and the video type, in this case:

tracker('my-video').setup('html5')

I'm not giving away too much here deliberately; there's a bunch of other stuff happening in the tracker function like plugins, event listeners and such, which are out of scope for this article.

The main object of this function is to match the type of player (we call them "providers" for disambiguation) with specific code needed to listen to events from that player.

Our setupProvider contains an object that is keyed to all of our providers, and dynamically imports the configuration for that provider, like so:

const providers = {
	youtube: async(elementId) => {
		const addYouTubeTracking = await import("./providers/add-youtube-tracking")
		addYouTubeTracking.default(elementId)
	}
	vimeo: async(elementId) => {
		...
	}
}

This makes it very easy to add a new provider. You need only instantiate the code for that player and add the appropriate designation to this object.

In the parent function we can parse this object and if there's no match inform the user:

function setupProvider(provider, id) {
	const supportedProviders = Object.keys(providers)
	if(!supportedProviders.includes(provider)) {
		// handle the error
	}
	providers[provider](elementId)
}

I find object literals with this catch pattern very practical; as well as being more performant, it's easier to catch errors and results in less duplication of code.

Next, we set up the addYouTubeTracking() function to listen to events from that player. I leveraged the YouTube Player API abstraction for querying the player:


function addYouTubeTracking(elementId){
	// Check if the element exists in the DOM
	const element = document.querySelector(`${elementId}`)
	if(!element) {
		// handle error
	}
	// set up YouTube API
	player.loadVideoByUrl(element.src)
	player.on("ready", function() {
		// See below for details
		playerEvents("ready", elementId)
	})
}

Instantiating providers separately like this not only avoids having to load code that isn't used in our application but it also allows us to standardise an API for talking to the rest of the application.

After this function, we have two significant pieces: playerEvents(), which can respond to events from every player in a uniform fashion, and dataEvents() which collates the data and passes it to an API.

Responding to player events

To enable playerEvents() to do this, each add[provider]Tracking() passes down events and also a group of callback functions that provide standardised APIs for getting the title of the video, the elapsed duration and other useful information. It means we can call the provider and get accurate data as the play is progressing without having to wire them all up individually: the playerEvents() call looks more like this:

const playerFunctions = {
	getElapsedDuration: () => player.getCurrentTime()
}

playerEvents("ready", elementId, playerFunctions)

Since the nomenclature of the elapsed duration varies from player to player, I'm handling that variation here in the same context. This saves developers from the mental overhead later, when they should only be concerned with the next stage of the application.

In the playerEvents() function I follow the same pattern as in setupProvider(), with an object enumerating the events and errors in case of unhandled events.

Here's an example of how we handle an event:

play: () => {
  dataEvents("update", elementId, {
    action: "play",
    updated: Math.floor(Date.now() / 1000),
  });
},

Again, there's a lot more in this function that I'm not telling you about. Not shown is how use a setInterval() on a play event so we can call back to the provider and update the elapsed duration from the provider.

Other events that could be sent from the player are also handled here include seek (our user has skipped forward or backwards), end, pause etc.

Parsing data events

dataEvents() follows the same pattern as the two previous functions. Yeah, I really like this method of indexing. But it's also practical: if a new developer comes into this code they should more easily be able to understand all of the logic in the application flow.

If it's a play event, we actually send an update event, since we should have already been sent a new event when the provider declared it was ready.

Just in case though, we check whether we have already collected data from this tracked video, and if not we call the new function to instantiate a new set of data.

How do we check whether we've heard from this tracked video before? Aha, this is one of my favourite pieces.

One Direction for Data

No that's not the name of some electronic boy band.

To be able to maintain a unidirectional data flow, I use the browser cache API to store events that come in.

    update: async () => {
      // do we already have a cache object for this?
      const cache = await caches.open(url);

      const cachedResponse = await cache.match(url);
		
		// looks like a no, let's create an initial event object
      if (!cachedResponse || !cachedResponse.ok) {
        dataEvents("new", elementId, null, incomingIndexes);
        return;
      }
      // if there's no old data or no valid data, we need to start over too
      const oldData = await cachedResponse?.json();
      const hasValidData = Object.keys(oldData[0]).length;
      
      if (!oldData || !hasValidData) {
        dataEvents("new", elementId, null, incomingIndexes);
        return;
      }

      // if the data is more than 4 hours old, we also want to start over
      const fourHoursAgo = Date.now() - 1000 * 60 * 60 * 4
      const isRecentData = fourHoursAgo > oldData[0].created;

      if(!isRecentData) {
      		// delete existing records
        cache.delete(url);
        // set it up again
        dataEvents("new", elementId, null, incomingIndexes);
        return;
      }

		// ok, we're good to go
		// This function overwrites the object retrieved from the cache with new values we obtained from the tracked event
      const outgoingIndexes = modifyTrackingData(
        oldData,
        updatedData,
      );
      
      // drop it into the cache
      cache.put(url, new Response(JSON.stringify(outgoingIndexes)));
		// Allamaraine!
      sendData(outgoingIndexes);
    },

Maintaining data in the cache comes with a warning. I provide a unique url string for the data object, and delete if it's more than 4 hours old. But even then we will have to be a little careful about when we release API changes.

However the advantages are clear: we can provide all of the data required by the API, updating it as the video progresses, and we're not jumping through the hoops of creating a new tracked event index each time.

Conclusion

It's been really fun to use some new browser APIs on this project and to stretch the limits of my knowledge, particularly on how to handle events and different video platforms.

But what I really enjoyed was making this code with future me, and others, in mind. I hope that it will make this code robust enough (along with the tests and types I've created) so that it will last a long time and be malleable enough to fulfil other business needs in the future.

How to scope Typecheck to your project folder

Tue, 31 Mar 2020 00:00:00 GMT

Published on Tuesday, 31 March 2020

I configured a command so that I can run a type check on my project, but the results always include stuff in the node_modules folder. It took me a lot of investigation and asking around before I figured out how to scope tsc to only the ./src/ folder?

in package.json:

"scripts: {
  "typecheck": "tsc --noEmit"
}

With this configuration, he compiler should pick up config from tsconfig.json, which looks like this:

{
  "compilerOptions": {
  ...
  },
  "include": ["src/**/*"],
  "exclude": ["node_modules", "**/*.test.*"]

But I still get results from node_modules when I run the command. It took me ages to figure it out, but for anyone else coming across this one, I managed to find that submitting an empty array to types did it:

"compilerOptions": {
    "types": []
}
```"

Is JSDoc Better Than TypeScript? A Real World Example

Thu, 25 May 2023 00:00:00 GMT

I've become an avid fan of TypeScript in recent years as it's improved my output, tightened up my appreciation for JavaScript's type coercion, and generally reduced my anxiety levels at work. However recently I've come into a place of work where JavaScript is written and often maintained by software engineers for whom JavaScript isn't their specialty. Not wanting to go back to the wild west of plain JavaScript, I adopted JSDoc. Here's how it went.

Update: I've written a follow up article about how JSDoc can result in better type safety than TypeScript.

I remember going through projects a few years ago and taking out all of the JSDoc notations. We were switching to TypeScript and JSDoc was seen as old and no longer needed. However it's gained a resurgence in recent months, most notably because Svelte is switching back to JSDoc from TS. But it has other benefits, and disadvantages, compared to TypeScript. Here's a few of the ones I've noticed.

Heads up! If you're looking for more info about this subject, there's an excellent syntax.fm podcast episode about it, which covers more ground than I have here.

Benefits of JSDoc compared to TypeScript

TypeScript has always been my favourite approach to typing JavaScript, but it's not all positive. Here's some gotchas that I encounter and which still are sticking points with the language.

1. No compile step

This is one of the most cited reasons for not using TS. TypeScript doesn't run in any browser, so each time you're writing TS, you're having to take that out of your built JS again at run time. Not only does this mean that fancy typed code could still be wrong, you've also got the overhead of compilation, and the extra tooling that goes with that.

For my team most of our JS is collocated in large applications built predominantly with PHP. Having an extra compile step for JS as well as building the PHP application might be frustrating and lead to overhead in pipelines.

2. Easier to follow

This is the main reason my team were not up for switching at the moment. Although they're pretty good at JavaScript overall, they're not used to newer ECMAScript syntax, so when they are working on a project without my involvement, they will struggle to understand what's going on and it could slow them down significantly. Training is the solution here, but the business (and the engineers themselves) have to see the real benefit to it first. At the moment, that benefit isn't clear to them, and it's likely that other teams with engineers who have backgrounds in different languages and tools won't see it either.

JSDoc is a totally different syntax but it is feature-rich, especially when compared to TypeScript, for instance you can even specify what error is thrown by a function, something TypeScript cannot yet do.

Disadvantages of JSDoc compared to TypeScript

To be honest, since using it seriously, I've been impressed with how far I've been able to get with JSDoc. I have been able to import type dictionaries and found out how to use VSCode Intellisense to get type hinting. But there have still been some caveats.

1. It's not as flexible

UPDATE: It is as flexible, I just didn't know enough when I wrote this ... please see the "addendums" section on how I solve these issues

If I need to negate the "object is possible undefined" error that you've probably seen a hundred times if you're doing DOM manipulation or data fetching:

const thing = document.querySelector('#my-element')
addStuff(thing); // Error: type undefined is not assignable to type Thing 
// or
const data = await fetch('https://my-url.dev');
return data.results // error: data is possibly undefined

In TS, you can either force or cast it:

const thing = document.querySelector('#my-element')
addStuff(thing!); // although probably don't do this 
addStuff(thing as Thing) // still not great but maybe you don't want to cascade down to catch undefined and oh yes this is so contrived

But in JSDoc I'm forced to add a lengthier explanation:

const thing = document.querySelector('#my-element')
addStuff(
// thing is definitely defined by this point, see line 120
// @ts-ignore-next-line
  thing
);

I don't know maybe this is better than a sneaky hashbang. But it sometimes feels quite frustrating especially with a lot of them on multiple lines.

2. More shenanigans which would be easier with casting

OK, so this is quite similar but I found it super frustrating.

const thing = document.querySelector('#my-element');

thing.style.display = 'block'; // Error: property style is not available on type Element

The querySelector method annoyingly returns an Element type, not HTMLElement, so every time I need to set styles via JavaScript (although it doesn't happen often), I have to do a ts-ignore-next-line. Again another situation where casting would have come in really handy.

How?

If you're looking for an explanation of how to implement this in your project, head over to Jared White's That HTML Blog website, his article called "The Nuances of JavaScript Typing using JSDoc" explains his method for how to use TypeScript with JSDoc.

Conclusion

All in all, I'm super happy JSDoc exists. Without it, I would be getting super frustrated because there would be no types - and therefore no documentation - for our codebases. However, I'll be even happier if we can make the switch to TypeScript syntax one day.

Addendum

Since writing this article, I have found a few tools that have improved the feedback I'm getting from the TypeScript compiler: instead of relying on my IDE (which caused issues when other team members were using a different IDE), I've since implemented eslint-plugin-jsdoc, which as you might imagine, has much better standardised linting and ensures that any errors or warnings aren't left up to the IDE, because both WebStorm and VS Code report different errors and sometimes miss things natively."

Addendum 2

As things have progressed, I have learned how you can override types for document.querySelector():

/** @type {HTMLElement | null} */
const thing = document.querySelector('#my-element');

thing.style.display = 'block'; // no error :tada:

The only thing I'm having real trouble with is React's setState hook. I'm hoping to have a write up for that ... once I solve it.

Follow up article

This article is overwhelmingly the most popular article on my blog, so I'll keep writing more articles as I solve different issues.

I've written a follow up article which includes a section about typing events with JSDoc.

Better Performance with the React Context API

Mon, 08 Apr 2019 00:00:00 GMT

Published on Monday, 8 April 2019 , originally at https://www.creativebloq.com

People are demanding more and more out of their online services, and businesses strive to excel these demands, leading to more and more complex sites ... and larger JavaScript bundles too!

How can we continue to deliver a first-class experience for users whilst still building for these levels of complexity? One good way of doing this - whilst at the same time as delivering more performant apps - is by using React's Context API.

This article was first published in Net Magazine. However, the Context API has been updated since. Please visit the React docs for a more up to date version of how to use Context.

What is Context and Why Use It? #

Sharing data and other properties across multiple React components previously required either an additional library (popular ones include Redux or Apollo GraphQL) or a lot of "prop drilling". "Prop drilling" means passing data up and down multiple layers of components which could be labourious, difficult to change when new requirements are built, and lead to confusing code. The Context API, released in React 16.3, aims to reduce our dependency on prop drilling and external libraries.

Apps can perform faster because changes made to properties no longer require a huge amount of re-rendering on a parent component, since the changes can be made in a separate component and handed down to child components from there.

How to Use Context #

I have set up a demo of Context in use on Code Sandbox: https://codesandbox.io/s/0pl62xq030. Let's walk through this code.

Context consists of 2 components:

A "Provider"
A "Consumer"

The easiest way I can think of explaining this relationship is that the "Consumer" is what will be using features "Provided" by our Context.

Initial Setup #

Create a new module called "Context.js" adding the following code:

import React from "react";

export const MyContext = React.createContext();
export const MyContextConsumer = MyContext.Consumer;

export default class myContextProvider extends React.Component {
  state = {
    aRangeofOptions: ["one", "two", "three"],
    updatesOnClick: null,
    onItemSelect: item => {
      this.setState({
        updatesOnClick: item
      });
    }
  };
  render() {
    const { children } = this.props;
    return (
      <MyContext.Provider
        value={{
          state: this.state
        }}
      >
        {children}
      </MyContext.Provider>
    );
  }
}

You might be able to see that on lines 2 & 3, we're calling on our core library, React, not some external dependency. That's the first benefit already!

I like to set up the Consumer in this file, even though we don't use it here. It helps me keep track of the logic when it's all in one place.

Inside our class I've set up a few items in state to demonstrate what's going on here.

Next, create a module containing a few items the user can interact with:

import React from "react";
import { MyContextConsumer } from "./Context";

export default class Module1 extends React.Component {
  render() {
    return (
      <div>
        <h2>Module One</h2>
        <MyContextConsumer>
          {context => (
            <div>
              {context.state.aRangeofOptions.map((item, index) => (
                <>
                  {item}
                  <input
                    type="radio"
                    name="selection"
                    key={item.Optionvalue}
                    onClick={() => context.state.onItemSelect(item)}
                  />
                </>
              ))}
              <div style={{ margin: "20px 0" }}>
                selected: {context.state.updatesOnClick || "none"}{" "}
              </div>
            </div>
          )}
        </MyContextConsumer>
      </div>
    );
  }
}

In this component, we can loop over an array in our context.state to create a few checkboxes, and a text description of the output.

In the next module we're basically going to duplicate this code but leave out the loop. This module is so we can demonstrate how context can be passed down.

Next we can create our main app.js file, or index.js if you like:

import React from "react";
import ReactDOM from "react-dom";
import MyContextProvider from "./Context";
import Module1 from "./Module1";
import Module2 from "./Module2";
import "./styles.css";

function App() {
  return (
    <div className="App">
      <MyContextProvider>
        <h1>Hello Context</h1>
        <Module1 />
        <hr />
        <p>It's magic!</p>
        <hr />
        <Module2 />
      </MyContextProvider>
    </div>
  );
}

const rootElement = document.getElementById("root");
ReactDOM.render(<App />, rootElement);

As you can see, the two modules we've created don't have any props passed down from the parent. Instead you get this functionality with the Context provider and children inside that.

But wait! There's more we can do to boost performance here.

Boxout: Analyse Performance with React Developer Tools #

React Developer Tools, an extension available on both Firefox and Chrome browsers, is a tool provided by Facebook to help debug your React apps.

It can also be pretty helpful at identifying where re-renders are happening in your app, allowing you the opportunity to refactor the code if you think it's necessary.

To find out where re-renders are occuring in your app, open the developer tools. Along the top of the pane you should see a number of options, including "Elements", "console" and other tools you've probably used before. Once you've installed the React Developer Tools you should see a new item labeled "React" Near the far end of this list.

If you click on the cog icon, you'll see some additional settings, including "Hilight Updates".

Try interacting with your site when "Hilight Updates" is ticked. You might be surprised at how much re-rendering is happening, slowing down the experience for users and having a negative impact on your conversions.

Now you know where to start fixing the problem!

Boxout: Reducing Re-rendering #

First of all, let's explain that term. Re-rensweing is what happens when the component you're using is updated. This has performance implications especially for interactions like dragging (think sliders), and inputting text. Without proper care users could notice lagginess and be put off from using your site. And the potential for laggy or unresponsiveness increases with complexity (because more re-rendering will happen).

I have found this to be especially true on devices that aren't the most expensive on the market. JavaScript parsing times can increase horiffically on low- and mid-range devices that don't run recent versions of Chrome or Firefox. A huge group of people could be impacted, severely affecting your conversions.

One way I've been able to reduce re-renders if by using my context's state to hold functions as well.

By moving onItemSelect out of the class and into the state object, we can call it in a similar way (context.onItemSelect becomes context.state.onItemSelect in our modules), and avoid a component update.

Try Using Context Today #

Have you found yourself writing props={props} all over the place, or using the spread operator to pass props down anonymously (like this: <MyModule {...props}/>)? Then perhaps Context is a way out of the hole you could have drilled yourself into.

If you're looking for a way to reduce the potential for complexity in a project, or reduce performance bottlenecks and you don't need the extra features that a dedicated state management tool or client-side GraphQL server would provide, chances are Context could equally be your tool for the job.

Context isn't going to fit everyone's use case, nor was it meant to. But it is a very useful API with some practical features that you might find give your code a little more ... well, context.

Check out the working code on CodeSandbox: https://codesandbox.io/s/0pl62xq030

Resources #

You can learn more about how to use the Context API by using the following resources:

Official Documentation #

The official React documentation is a great place to start especially if you want to understand new APIs like Context, because they're written so well and shows real world examples that are easy to follow.

Courses & Blog posts #

I used these docs to get started using Context, as well as Flavio Copes' guide to the Context API (https://flaviocopes.com/react-context-api/) and Kent C Dodds' course "Advanced React Component Patterns" on egghead.io (https://egghead.io/courses/advanced-react-component-patterns).

Those resources are great. But there's nothing like diving in and trying it for yourself. It'll be challenging to begin with but that's the best way to learn! I freely admit to getting frustrated, writing poor implementations, and having to start over many times before I understood Context well enough to use it in a project. But if you give up, you only have to come back to it one more time ... until you finally crack it!

To do that, you could use two fantastic resources that have recently become available:

Codesandbox #

Codesandbox (codesandbox.io) is an online editor for creating apps in many languages, with built-in support for React. It's like codesandbox but with support for javascript modules

Spectrum.chat #

Spectrum.chat, built by the creator of styled-components and recently acquired by GitHub, is rapidly becoming a lively community. I've found it a great place to share ideas, ask for help and chat about the latest developments in JavaScript, React and other tools I use every day. Hope to see you there!"

Using CSS Grid In Production Today

Sun, 09 Jul 2017 00:00:00 GMT

For me, as well as a lot of others, CSS Grid is the most exciting thing to happen since CSS3 ... possibly even CSS2. But many fear using it in production. This post tackles one common use case where a Flexbox fallback provides support for Internet Explorer and Edge.

I'm really, really excited about CSS Grid. This new technology, which allows developers to finally build a structured document layout natively without using a hack, was released in most major browsers in about March of this year.

I've been closely following the work of Jen Simmons and Rachel Andrew on this subject, and have attended other talks about the subject. But now I want to join the chorus and say with a definite certainty: yes, you can use CSS Grid in production today.

I already have.

Why CSS Grid? #

I imagine that many of us are quite happy with the tools we currently use to lay out our web pages. I too was pretty confident using either Bootstrap's grid, Susy, or something I custom-built, to lay out my document using floats.

I don't need to say here what's already been said, and which you've probably heard, about floats being a hack. Suffice to say, that this approach leads to increased maintenance cost over the lifetime of a project and contributes to poor performance of a website.

But floats are also a hack, just like tables were before that. CSS Grid is the hero the web needs.

But ... It's Not Supported! #

Like Rachel Andrew said in her recent blog post (link here) around 70% of browsers have support for Grid. That's most of the web. You might want to check your own analytics to see what you need to support for your project, however I believe I have come up with a workable solution for when you need to support IE11.

The Scenario #

I often find myself with this UI layout in my projects: 2 repeating columns of content, inside which are elements, usually designed with dummy content of the same height. This never works in production with floats. Sooner or later I get gaps like great big ugly missing teeth everywhere.

The way I've traditionally coped with this is very convoluted:

<div class="row">
<?php $count=0; foreach ($items as $item) : ?>
<?php if ($count % 2 == 0 { ?>
</div><div class="row">
<?php } ?>
<?php $count++; endforeach; ?>
</row>

Phew!

This assumes that 1) Your frontend developer is authorised to, and competent at, writing server-side code, 2) you don't mond doing a lot of counting, 3) you don't mind spurious markup everywhere.

This approach is messy, but it works.

Supporting 97% of the Web #

Here's the alternative. I can support IE11, Edge, and older versions of some browsers by using flexbox instead of grid.

First, I declare the flexbox layout:

@media(min-width: 768px) {
  .container {
    display: flex;
    flex-wrap: wrap;
    margin-left: -15px;
    margin-right: -15px;

    > * {
      flex: 1;
      flex-basis: 46%;
      margin: 15px;
    }
  }
}
@media(min-width: 1200px) {
  .container {

    > * {
      flex-basis: 47%;
    }
  }
}

Please note: this is 22 lines of code that I hope I will be able to remove from the project at some point. If not, it isn't doing any harm where it's not used.

Now, below that I use an @supports rule to scope the grid layouts:

@supports (display: grid) {

  @media (min-width: 768px) {
    .container {
      display: grid;
      grid-template-columns: repeat(2, 50fr);
      grid-gap: 15px;

      > * {
        margin: 0;
      }
    }
  }
}

This is only 11 lines of code (not counting the lack of a <div class="row"> etc, and doesn't use hacks like negative margins. Much better.

This works in 97% of browsers today, even Internet Explorer.

Now, in both layouts, you get 2 columns of content, each item on a row has the same height, and they can expand to the content that you put into them.

One thing to note: I'm not replacing my floated grid in this project. I'm complementing them at this stage for when floats are inadequate.

But in the near future, I'm really looking forward to swapping out this approach for more semantic, web-native code.

I hope this little demonstratons helps you to see how CSS grid works, is faster, less costly to maintain, and great for the web."

Using a moka pot on an induction hob

Wed, 29 May 2024 00:00:00 GMT

This is a little trick that wasn't on any of the articles I found using the old search engines. Although those articles did contain a lot of ads .. if you, y'know ... like ads. So I'm posting it here. It's about how you can use an aluminium coffee mocha on an induction hob.

I went to Southern Italy a few weeks ago. It was great, thank you for asking. We stayed with a local family so enjoyed the best of the area and the cuisine.

Our friends have one of those induction hobs, which I love by the way. They're very easy to keep clean. They also have an moka pot, one of those little silver things that you see on gas stoves often.

Trouble is the pots are usually aluminium, which won't work on induction hobs. Don't get technical on me, I have no idea why.

It turns out what they'd done is put the moka pot inside a frying pan on the oven. Hah. Simple as that. The frying pan can transfer the heat, and the moka pot heats up beautifully.

So now you know. By the way, this could be a bad idea for some reason I haven't thought of, so just to be clear: try this at your own risk!

How to make great coffee

Let me just share this with you because if you care enough about coffee to get a moka pot, you probably would like to know how my Italian friends made us cuppacinos every morning.

First thing to do is put the moka pot on. You can cheat a bit and add boiling water to the pot. It's much faster but you have to mind you don't burn your fingers when you screw the pot back together.

After putting the moka pot on, they heated up half a cup of milk. Then poured that into a french press and plunged the milk inside the press for around 30 seconds. This turned the milk into a lovely frothy texture.

Once the moka was done, they poured the coffee into a cup with 2 teaspoons of sugar and mixed it together vigorously until all the sugar was absorbed.

The final step was to pour the frothy milk into a cup then add the sugary coffee.

I'm sorry. You're likely going to start resenting the £4 or more those places charge for coffee once you've tried this!

Using Pop!_OS for Work

Sat, 10 Feb 2018 00:00:00 GMT

In November last year I switched from using Windows at my place of work to using Pop!_OS by System76, an Ubuntu derivative designed for developers, researchers and scientists. What motivated me to make the switch? What benefits, and disadvantages, has it brought me?

When I came to work at my current employer, I got given a PC to develop on. I was eager to fit in and please everyone, and accepted the situation with the single caveat that it had to run WSL (Windows Subsystem for Linux, commonly known as Linux on Windows).

Using Windows Subsystem for Linux #

I've enjoyed using WSL, and it certainly has made Windows much more painless to use as a developer. I spend a lot of my time in the terminal day-to-day, using various web development tools including git, node, yarn, and other tools such as WPScan to analyse legacy sites. But it does have certain limitations. For example, I could never get Mysql to work on the command line. Only Atom allowed me to open files straight into my text editor. WSL is a brilliant innovation and I hope the team continues to push the boundaries there.

But in the end, it wasn't quite enough for me. I also never got embedded into the Windows workflow. You know, little things like how windows layer on top of one another instead of occupying their own spaces. I never got over little things like that which interrupted my mental model of my workspace.

Why Not MacOS? #

For a loooong time I've been a Macintosh afficionado, ever since my friend Martin Harries sold me a Performa in the early 90s. Although the OS is a great solution for a lot of individuals, and I can't argue that it's visually appealing.

But when you start getting under the hood you come up against its inadequacies.

I didn't like the absence of a package manager to maintain the OS from the command line. Also, a lot of the apps are a layer on top of command line programs which interfaces I now prefer for a lot of tasks.

Enter Pop!_OS #

Pop!_OS is billed as a system built for creators. It's interface is clean and out-of-the way, although it reflects well the 70s design ethos of its creators, System 76.

Where Pop! came into its own for me was it's inbuilt terminal. I started off downloading and configuring Hyper, my go-to choice on the Mac. But after a while this broke, leading me to use the native terminal. It didn't take me long to figure out this had already been configured really well with a lot of the extensions I would've added anyway.

The other great thing is the Super key. On Windows, the Super key will allow you to search your programs. On Mac, the super key with a modifier gives you either a search / app launcher, or it tabs through your open apps.

But on Ubuntu (of which Pop is a derivative), it serves both as an app launcher and zooms out all of your open apps, allowing you to visually switch between everything, or start a new workflow. It also gives you access to your desktop spaces so you can see other workflows in progress.

For my mental model this is absolute genius. I've got a pretty visual mind so seeing all my work spaces tiled out like this is really helpful in giving me some context.

Caveats, or How Not to Break Your Computer #

For many people, using Windows or MacOS is insurance ... you know you're probably not going to be able to break the OS irrevocably, and if you do you can just restore it from the cloud. Ubuntu can have this kind of setup, however I ... haven't configured it. I have broken things a few times, mostly by revoking my own permissions to edit files or folders.

Hah, yeah, I'm such a newbie at this.

But on the other hand, I have always found a solution from the excellent community support forums, or helpful FOSS software maintainers.

Whilst there's cause for concern, I'm not worried to the point of paralysis. It's a bit like doing DIY. Yes, you can break stuff, but you have to try your best, and call in the experts when you get stuck.

One day, in the far distant future, I hope I can help others who might be in a similar position.

App Availability #

One thing that grates against the argument that you do need a bit of familiarity to aid your workflow is that a lot of the apps I'm used to using aren't available for Linux.

A good example is Sketch, which was my go-to design tool.

I've found a great replacement in Figma, which is superbly responsive on a browser if it doesn't currently have a native app. I'm also really looking forward to Alecaddd's Akira, which is in beta but looks really really promising.

For raster graphics editing, I have used GIMP. As much as people rave about it, I have to say ... I hate it. It's just so counterintuitive. Instead, I've been using Photopea, which opens just about any graphic format, and is a pretty decent editor in the style of Photoshop.

These tools have meant I hardly skip a beat when I am handed a set of design files.

Conclusion #

Pop!_OS is a smart, well designed release of Ubuntu which has sped up my workflow massively. I thoroughly enjoy using it. I'm seriously considering replacing my home laptop MacOS system in favour of it. The only thing that's holding me back is the thought that I might lose the ability to play the platform game Braid. That would be very sad, but not essential!"

Using the Srcset attribute on images

Fri, 15 Jan 2016 00:00:00 GMT

Using the Srcset attribute on images

Published on Friday, 15 January 2016

So, I've been using the relatively new (and somewhat confusing) srcset attribute to serve a couple of interesting use cases lately. This spec is really, really useful in certain circumstances, two of which I'd like to share with you here.

Situation 1: You want to use SVGs, but need to support browsers that don't #

SVG support is fairly advanced across the browser spectrum (above), but srcset is supported less, especially by legacy IE browsers (below). This allows us to use the srcset attribute to specify an SVG image (or WebP for that matter), allowing newer browsers to download the Srcset image in preference, and older browsers to use the fallback .png file (or whatever else).

I can't claim to have thought of this myself, Sara Soueidan covered this concept on her blog in February 2015.

The markup for this could actually be quite minimal:

<img src="logo.png"	srcset="logo.svg"	alt="alt text goes here" />

The browsers that recognise srcset will download that asset, so you'll get a nice crisp SVG, otherwise older browsers (which don't have support for svg either) will get the image.

I routinely add dimensions to my srcsets to avoid rendering problems in certain browsers where they will always render the SVG at the natural height & width of the SVG, so if you have a different sized png, things can get a little crazy.

Situation 2: You want to use 2 different images at different breakpoints

Just recently I had a client requirement for an image that would be wide & narrow at large breakpoints, and thinner and taller at smaller, like this image demonstrates:

Yes I could do this with background images in CSS, but then the browser would have to download both images, increasing the page weight. Srcset solves this problem by fetching the appropriate image based on what sizes you stipulate in your media queries. This took me a few tries to get right, but I ended up with a robust solution that would conserve bandwidth:

<img
	src="fallback.png"
	srcset="
			small.png 649w,
			large.png 1140w
		"
	sizes="
		(max-width:767px) 649w,
		(min-width:768px) 1140w
	"
       alt="descriptive text"
/>

What I'm doing is specifying that the browser should behave this way:

If the viewport width is below this, show the smaller image
If the viewport is above that, use the other image

I started off without the max-width query, but was always getting the larger image, which seemed to render correctly after I introduced this query.

Eric Portis' comprehensive explanation was great at getting me to think about how to do this, what I'm trying to do here is break it down to be something I can use in production without having to fry my little brain at the staggering genius that are the foundation tools of our trade."

Using WordPress as a Headless CMS

Wed, 25 Jul 2018 00:00:00 GMT

In case you missed it, here's the article and links to the example code I wrote to help developers get started with Gatsbyjs.

Code for this example: https://github.com/endymion1818/netmag-wpheadless

You Will Need #

A computer with an internet connection (obvs)
A code editor (Atom or VS Code)
A command line shell / terminal
A recent version of NodeJS installed (you can download & install from here: https://nodejs.org/en/)
A WordPress site to get content from. If you don't have one, you can use the WordPress.com platform (wordpress.com) or this Heroku build pack: https://www.technomile.com/capabilities/application-development/heroku/wordpress

Getting Started #

Using WordPress

WordPress comes out-of-the box with the REST API, and that is what we're going to use to query your data. So we don't really need anything else! Our display site is totally separate from our content site, so we won't need a theme or any other customisation outside a few (optional) plugins.

The exception is if you need custom meta fields for extra content areas you're probably using Advanced Custom Fields to do so. You can add that data to the WordPress API buy installing this plugin: https://en-gb.wordpress.org/plugins/acf-to-rest-api

Using a Static Site Generator

Now that we have our content source, let's fetch the data and display it using a static site generator (see the box "why use a static site generator"). My weapon of choice in this realm is Gatsby (https://gatsbyjs.org), an excellent static site generator built with JavaScript.

If you're looking for a good way to build on your JavaScript skills, and learn React by getting stuck into some code, I highly recommend trying Gatsby to do so. I've learned a lot myself by playing with it.

First, let's install a command-line tool that allows us to create Gatsby sites:

npm install -global gatsby-cli

Now, navigate to the folder where you want to keep your site, and run this command:

gatsby new blog

This will create a new folder called 'blog' and install Gatsby and its dependencies to this folder. Open this folder in your favourite text editor. There might seem to be a lot of files there. Don't worry, we'll only be directly editing the gatsby-config.js, gatsby-node.js files, and the src folder, which is where our templates live.

Why Headless? #

I first heard about this approach from a talk I watched from Twin Cities Drupal (https://www.youtube.com/watch?v=GX9z5M9mz30). I liked the idea of a seperation of concerns between authoring content and viewing that content.

I had also already experienced how easily one server can go down, taking out all of the sites on it, leading to minutes or hours of heart-stopping panic (there were 24 hours on mine). I had also seen how a monolithic CMS-based site could suffer a security compromise and take a lot of effort to repair (that took me more than 2 days of unpaid work).

I had also seen the benefits of CDNs (content delivery networks) that can store your image, audio and video files on servers optimized for fast delivery, and can duplicate those files across the world for speedy delivery to those regions if needed. What if your entire website could benefit from this approach? I gave a talk at WordCamp London 2018 about this subject

Getting our Content #

The first step we want to take is to fetch our content from the WordPress site's API.

To do that, we're going to install gatsby-source-wordpress, a prewritten plugin for WordPress. This is one of the main reasons I love Gatsby - you can get your data from many different sources. A lot of static site generators are restricted to using Markdown files, but Gatsby is very flexible.

Gatsby's plugin ecosystem is very mature with loads of prewritten ways of getting your data, and lots of other functionality that comes in useful too.

To install the plugin, first change directory into your new Gatsby site by using this command: cd blog.

Now run this command: npm install --save gatsby-source-wordpress

Now open the gatsby-config.js file. There will already be some basic setup Gatsby gives us by default. We're going to add to that to configure our plugin here.

Did it work?

You can check by opening your terminal, typing gatsby develop and watch what happens. Be warned! Even if you got your settings correct, You will get some warnings anyway, this may be Gatsby looking for content which you haven't written yet.

If the last few lines looks something like this, you're probably OK:

You can now view gatsby-starter-default in the browser.

  https://localhost:8000/

View GraphiQL, an in-browser IDE, to explore your site's data and schema

  https://localhost:8000/___graphql

Note that the development build is not optimized.
To create a production build, use gatsby build

If that isn't what you're getting, check your WordPress site isn't on a subdomain, and that it's definitely using HTTPS or HTTP, and you have the same in your settings.

Now we can go to https://localhost:8000/ and see our Gatsby site!

Can We Query our Data?

You may have noticed that there is no WordPress content here. This is because we haven't told Gatsby what to do with it yet. Before we do that, let's just check we have actually got our content available to GatsbyTo do that, visit this url:

https://localhost:8000/___graphql

This built-in tool is called GraphiQL, and is another secret power of Gatsby.

GraphQL is similar to REST : it's a way to query data. But with GraphQL, you can interact with your data much more easily. GraphiQL (a visual IDE for GraphQL) can show us some of these tricks. On the left panel, try typing the following:

{  allWordpressPost {    edges {      node {        id        slug        status        template        format      }    }  }}

This might look a bit like JSON, but it's not. It's a new query language that I think one day will largely replace REST as a way of communicating with APIs.

What did you get when you pressed CTRL + Enter in GraphiQL? You hopefully will have seen your WordPress posts on the right site of the screen, something like this:

We are actually going to use this query in our next step, so keep it handy! You might want to see what other data you can get with GraphiQL whilst you're here. If you want to do that, try moving the cursor around and typing either CTRL + Space and / or CTRL + Enter. That will reveal other groups of content.

So, we now have content in Gatsby. Next, we need to display it...

Displaying our Posts #

For this next step we're going to use the gatsby-node.js file.

gatsby-node.js is a file you can use to interact with Gatsby's "Node API". Here you can control how your site is generated and create pages, posts and more.

We're going to write some instructions here to tell Gatsby what to do with our data:

This code creates pages from our GraphQL query, and for each page it'll use a template we've defined (/src/templates/post.js). So next, we need to create that file!

Create post template

Inside the /src/ folder, create a folder called templates, and a file inside that called post.js. Add to it this code:

This uses a different GraphQL query to get data about the specific post it's been fed by the gatsby-node.js file, then uses React to render that out into the browser.

If you want to quickly see a list of all your posts, you can type https://localhost:8000/a into your browser's address bar. This will take you to a development 404 page, which lists all of your posts. Click on one to visit it!

Next Steps #

We've scratched the surface of how to use WordPress as a headless CMS, and I hope I've introduced you to some interesting concepts and tools that you might be able to use and experiment with in the future.

There's a lot more to this story, and my colleagues and I have blogged about it extensively, and I've also written more on my personal blog, Delicious Reverie (https://deliciousreverie.co.uk/).

Please, keep in touch with me via those channels and on Twitter to hear more exciting developments in the world of headless CMS!

Why use a Static Site Generator? #

Traditionally, when a user visits a site, some code where the website lives kicks into action, getting data from a database, inserting it into templates, and stitches page pieces together before sending that code down to the user. This can take quite a bit of time.

A static site generator does all of this before the user gets to the website. Because of this they can be a lot faster, and a lot less prone to security risks (because there's no code running behind the scenes on the server). They allow us to have a website with dynamic content (blog posts, pages, products, or any other class of content) which might need extra pages for archives, categories, and a way of making & saving drafts.

Static Sites can be hosted entirely on CDN services like Netlify (https://www.netlify.com/) that have generous free plans and take away the worry of managing servers."

Using WSL (BASH for Windows)

Tue, 01 Nov 2016 00:00:00 GMT

Published on Tuesday, 1 November 2016

At a new job I've been tasked with moving to Windows as my main production environment. At first approaching this idea with some trepidation, I have since discovered and been able to install the new Windows Subsystem for Linux on my PC, and it's proven to be a really useful tool.

"Bash on Windows" isn't really what it says it is. It's not simply the terminal, but linux commands for the whole of the windows subsystem. There's a video here that demonstrates what's possible, but as a "normal-ish" developer with the sole aim of developing websites, and no special knowledge of Linux or Windows, I thought it would be interesting to share what I found to be most helpful, and most challenging, about using it.

The Problem #

I was pleased that I was able to install WSL without a hitch using this guide on how-to geek, but that's where it stopped being easy for me.

Problems arose because the build process my new employer had established required using a command-line php script which was a wrapper for WP-CLI, the command-line interface for Wordpress. This script ran through a few steps that would be essential to me getting up and running and developing new sites:

Download Wordpress Core
Add build plugins and remove existing dummy content
Create a wp-config file
Connect to MySQL, add and configure a database
Download plugins and themes from a private Git repository

As you can see, this required a number of connecting services, which were not already installed on either Windows, or WSL. I made the decision that I would install these first inside WSL, and see if that would be more useable. I figured that it would be more difficult to configure everything I needed using the more unfamiliar Windows Command line.

Command-line PHP and WP-CLI #

I found installing these two tools was fairly easy using wget, and adding to my BASH profile. The only hitch (that I discovered later) was that the php user didn't have the necessary permissions to create the wp-config.phpfile. I discovered that by changing ownership of the folder I was using for my web projects to www-data was the key.

$ chown -R www-data /var/www/html/

AMP (Apache, MySQL and PHP) #

This was the most difficult step out of all of them. I discovered quickly that my regular tools of choice (MAMP or WAMP) meant that MYSQL was a Windows executable, and so not useable in bash. My heart sorta sank at this point. However, configuring my own AMP stack is something that I had wanted to do so ...

I quickly found this excellent tutorial on Linux Mint community forum which takes you step-by-step through nearly everything I needed to download and install the AMP stack.

One major hitch that occurred was when I did get my website up and running, but all links on the site were 404ing. Turns out that I needed to configure MOD_REWRITE on Apache. This tutorial on Digital Ocean and this question on Stack Overflow helped immensely.

Once I had gone through all this and changed a few things, I found that links still weren't working. Then, I noticed that Wordpress was telling me that it couldn't write to .htaccess, and gave me a snippet to paste there. Voilá, working websites.

GIT Access #

The final piece in this particular puzzle was GIT. I needed to set up global credentials and store them in the system memory so that the build script could use them when it required. I eventually discovered that you can use a credential helper to store your user and pass in memory:

$ git config --global credential.helper cache

Once these were configured, I was able to run my company's build script, and all was well with the world.

Why I'm Not Using It Now #

So, after going through all this pain, I eventually did manage to get over my particular hurdle and configure my Windows environment to use our in-house script and WP-CLI, although it was a challenge.

Why?

The principle reason was that there is a barrier between WSL and the Windows environment. It's not always perceptible, but it's there.

The main thing is that the file system is still in flux. I found that the Linux filesystem aforementioned tutorial on How-To Geek has moved location in latest builds of WSL, and I couldn't locate it. This means that opening files in my text editor (Sublime Text or Atom) was an impossibility.

Even if I had pursued it, I found out that there are incompatibilities in the filesystem which could mean that files might not be compatible after opening. Some have reported that files become invisible in WSL after being opened using a Windows application.

I did start using the venerable Vim to edit my files, and actually, it's something I want to go back to one day. But I'm pretty sure in the short term at least, with looming project deadlines, it would frustrate me very much.

There are also currently some issues with MySQL which the WSL team are looking into, and I did find it to be buggy. Hey, this is an early release, there's bound to be glitches that need ironing out.

What I Learned #

I still use WSL every day to navigate around my file system and to run build scripts, and that isn't going to change. For me, it's far more intuitive and I'm comfortable with it.

Although it's really nice to have it, but actually I don't think it's an essential. Being able to use BASH to open folders, or open folders in .exe apps, now that would be incredible."

Validate your commits with Git Hooks

Fri, 21 Feb 2025 00:00:00 GMT

Many of us run linters (probably eslint) and formatters (like prettier) to make sure code is in a suitable state before merging. We do this so that it's easier for other people to modify the code we've written. But have you, like me, gotten frustrated by failures in pipelines due to this seemingly small problems? It's comparatively costly for failures to happen there too, since the compute time in pipelines needs to paid for.

With everything else we need to remember, checking formatting and linting is easy to forget. If this sounds like you, there are some great solutions out there.

Husky

Probably the most popular solution, Husky, can be installed on a per-project basis. It runs before you commit code to your project, modifying your changed files so that they are linted and formatted. Anything that's fixable will be modified and added to the commit.

This is a simple, small, easy to use library that can be modified to suit your needs.

But what if you can't use Husky, or don't want to?

Git Hooks

Another way to make sure you run linters is by using Git Hooks. I typically configure my global git so that each time I run git commit my hook runs.

Here are the steps I took to make it work.

1. Set up files and folders

Git recommends you use a folder called .git-templates, and inside that a folder called hooks.

# from your Home folder
mkdir .git-templates
cd .git-templates
mkdir hooks
cd hooks

Now create a file called pre-commit, no extension and make it executable.

touch pre-commit
chmod +x ./pre-commit

2. Tell git about it

Next we need to tell git where to look for this hook. We can let it know by running the following

git config --global init.templateDir '~/.git-templates'

Now Git will look for the specific path (/hooks/pre-commit) that we set up and run the executable.

3. Set up the executable

Open the file pre-commit in your favourite editor, and add the following

# in the ~/.git-templates/hooks/pre-commit file
#!/bin/sh

This line tells the executable which shell to use. I'm using sh, but you can use bash or something else.

Below that we can add the commands we want to run.

Now this is where, because this is a global, it helps if you've standardised your build / test/ format / lint commands. I've done across many of my projects so I can add them to the file:

npm run lint:check
npm run format:check
npm run test

Now when I run git commit in my project, I get the following message:

gc -m 'test'

> myproject@1.0.0 lint:check
> eslint '**/*.{js, ts}

> myproject@1.0.0 format:check
> prettier --check '**/*.{js, ts}

checking formatting ...
[warn] src/my-file.js
[warn] Code style issues found in the above file. Run Prettier with --write to fix

Git Hooks are great

What's even better is that if the project doesn't have any of these scripts, it'll fail silently and still allow you to commit as normal.

I really love Husky, but I also am really happy to use a native solution that's built into Git, along with bash scripts, to make my coding workflow both cheaper and faster.

Waiting for an Element to appear in the DOM or shadow DOM with an Intersection Observer

Mon, 16 Dec 2024 00:00:00 GMT

An asynchronous solution which waits for elements to appear in the document. Covers both a normal Document and the Shadow DOM.

I've been working extensively with asynchronous HTML loading lately. Normally there are built-in APIs you can use to hook into when DOM elements are available. But a few times lately, my code has run outside of those APIs.

So I came up with this implementation that's proved really useful for resolving elements that can appear asynchronously and even if they're inside the shadow DOM.

Using on a Drupal Project

If you're writing JavaScript for a Drupal project, youll notice that Drupal loads the DOM asynchronously. This is because it can have normal taxonomies and content, but also featured queues, which as I understand it, can get quite complex and need to be retrieved from the database. This can result in DOM elements being loaded out of order.

If you're wanting to animate those elements, say for instance in a carousel, that's going to prove challenging.

Drupal ships with it's own solution for this, once(), which I understand it is originally a jQuery thing.

So if you're in Drupal-land, you might want to use that.

Any project

Although my project was Drupal, my code existed outside of that ecosystem as a standalone project. Therefore I rolled this code.

Let's walk through it.

Getting up and running

async function waitForElement(root, selector, timeout = 1000) {}

The arguments in the function are root, the root element. This is usually the document, however as you'll see in the implementation there could be other situations where that's not true.

For example, I can pass in a shadow Root, which is particularly handy for hooking into Web Components. I've built a complex one as part of this application which I hope to blog about sometime.

Then we have the selector. This is the selector for the element you're waiting for as a string.

Lastly, we have a timeout. This sounds awful to start with but we don't want this thing running in the background forever, so the timeout of course means we can resolve after a reasonable amount of time has passed.

This is hazardous I know. If we're on slower networks, it's going to take longer to load, meaning it could very easily timeout before the element appears.. what can I say, sometimes stuff fails.

Promises made to be broken

return new Promise((resolve, reject) => {
  let element = root.querySelector(selector);
  if (element) {
    resolve(element);
    return;
  }

We're setting up a new Promise so that we can resolve it with the element if we need to, or reject it if if fails. You could also instead return null in the case of failures and handle it in the application.

Next I check to see if the element is already there. It might have loaded already after all!

Observing mutations

Hmm now I sound like some DNA mad scientist. Promise this isn't going to get icky. Much.

const observer = new MutationObserver((mutations) => {
      mutations.forEach((mutation) => {

So we're newing up a MutationObserver. A mutation is triggered when anything in the DOM changes, and allows you to write a callback for when that happens.

if (mutation.type !== "childList") {
  return;
}

Because it fires on every mutation we need to filter for the ones we're looking for. They can be different types of mutation, but we're looking for a childList, as opposed to changes in text or attributes of an element.

Next we need to cycle through each of the addedNodes to filter through them and discover if they might include our element. However, we also need to filter out a few more things:

mutation.addedNodes.forEach((node) => {
  if (node.nodeType !== Node.ELEMENT_NODE) {
    return;
  }

Firstly, if it's an ELEMENT_NODE, so a DOM element instead of a comment block () or the document itself, then we break out of the forEach early, since we're only looking for those.

However if it is we might have a warranted early success.

Early success

Let's be optimistic, if the first valid mutation is the selector we're waiting for, then we have a win. However of course this might not be the first time our oberver has been called, in which case we might need to cancel the timeout. It would also be sensible to disconnect the observer at that point.

if (node.matches(selector)) {
  observer.disconnect();
  clearTimeout(timeoutId);
  resolve(node);
  return;
}

Check if shadow root

If we haven't yet found the element we should first check if the node is in the shadow DOM, and whether we have it in this mutation:

if (node.shadowRoot) {
  const shadowElement = node.shadowRoot.querySelector(selector);
  if (shadowElement) {
    observer.disconnect();
    clearTimeout(timeoutId);
    resolve(shadowElement);
    return;
  }
}

Otherwise we should also set up an observer for the shadow root to see if it is going to appear there:

observer.observe(node.shadowRoot, {
  childList: true,
  subtree: true,
});

Next, let's check if the element we're looking for is within the root now that it's been updated. If so we can resolve that element and disconnect the observer.

element = root.querySelector(selector);
if (element) {
  observer.disconnect();
  clearTimeout(timeoutId);
  resolve(element);
}

Listening for further changes

Now we have the substance of our observer we need to set it off by calling observe which will keep watching the root element we supplied. That way any other changes will reveal whether the element we're looking for has arrived in the DOM:

observer.observe(root, { childList: true, subtree: true });

In my example we watch the childList and the subTree in case the element appears as a descendent of the parent and not a direct child. If you know that it's going to be a direct child of the element you don't need to supply these options.

Timeout

As we mentioned before, we don't want this to keep running indefinitely. At some point the DOM is going to either finish loading or we're going to have to stop waiting.


timeoutId = setTimeout(() => {
  observer.disconnect();
  reject(`Element not found: ${selector}`)
}, timeout);

This way we disconnect the observer and we can either return and handle the error here or as I've done pass it up the call stack with a Promise.reject().

In this case you'll need to wrap a call to waitForElement() with a try {} catch {} block.

So the flow for this is that if the element isn't found immediately we'll call the observer, then set a timeout to disconnect if it isn't ultimately found.

Happy Copypasta

Here's the full code which you can exploit to your heart's content.

/**
 * Wait for an element to be added to the DOM or shadow DOM.
 * @param {ShadowRoot | Document} root - The root to observe.
 * @param {string} selector - The selector of the element to wait for.
 * @param {number} [timeout] - The time in milliseconds to wait before rejecting the promise.
 * @return {Promise<HTMLElement | Node>} - A promise that resolves with the element when it is added to the DOM or shadow DOM.
 */
function waitForElement(root, selector, timeout = 1000) {
  let timeoutId;
  return new Promise((resolve, reject) => {
    let element = root.querySelector(selector);
    if (element) {
      resolve(element);
      return;
    }
    const observer = new MutationObserver((mutations) => {
      mutations.forEach((mutation) => {
        if (mutation.type !== "childList") {
          return;
        }
        mutation.addedNodes.forEach((node) => {
          if (node.nodeType !== Node.ELEMENT_NODE) {
            return;
          }
          if (node.matches(selector)) {
            observer.disconnect();
            clearTimeout(timeoutId);
            resolve(node);
            return;
          }
          const shadowElement = node?.shadowRoot?.querySelector(selector);

          if (shadowElement) {
            observer.disconnect();
            clearTimeout(timeoutId);
            resolve(shadowElement);
            return;
          }
          if (node.shadowRoot) {
            observer.observe(node.shadowRoot, {
              childList: true,
              subtree: true,
            });
          }
          element = root.querySelector(selector);
          if (element) {
            observer.disconnect();
            clearTimeout(timeoutId);
            resolve(element);
          }
        });
      });
    });

    observer.observe(root, { childList: true, subtree: true });

    timeoutId = setTimeout(() => {
      observer.disconnect();
      reject(new Error(`Element not found: ${selector}`));
    }, timeout);
  });
}
export default waitForElement;

On website longevity

Tue, 31 Mar 2020 00:00:00 GMT

When you build something, how long do you expect it to last? Whilst it might be easy to think about the things we create as our legacy to the world, that might not be the case when it comes to our online creations ...

When I was first experimenting with Gatsby, I built a site that I was really proud of. It was my first JAMStack project, and allowed me to publish content for a cause I wanted to support. However, only a few years later I found I could not continue to use it.

What had happened was that things had moved on. I had built it with an early alpha of Gatsby v1, and since then the project has continued to iterate, being supported by it's vast online community.

All of the dependencies that Gatsby uses under-the-hood had been updated too. React router had made way for reach router, there were new ways of handling many different things ... no doubt better ways, but ways that still left my little old website flaky after only about three years of uptime.

This isn't necessarily a problem. People can still view the content I posted there in all those years that I was able to use it. And I could still have run it locally ... had I not wiped my computer and started over with a new install of Nodejs, Gatsby cli, and everything else.

That last step seems to have rendered it challenging for me to even add a new locally-hosted Markdown blog page, and so I find I haven't updated the blog in quite a while now.

I have three young kids ... if you have kids, you know that means practically zero time out. So there's no time for me to even plan - let alone carry out - a rebuild of the site.

I guess what I'm saying is that these new tools are awesome, and I don't want to stop using them. I just wish they didn't result in things being quite so ... ephermeral."

What is Typescript?

Thu, 14 May 2020 00:00:00 GMT

What is TypeScript? How can it be useful to a person learning web development? I was asked this question on a Slack dicussion recently, and came up with the following example which seemed to be enough for my friends to grasp it.

JavaScript is a dynamic language.Here’s an example using the browser’s console:

I assigned the var a to a number , the number 1, then I assigned the next var, b, to a string, 'two' ... and then I added them together.

The result was a string (1two) because JavaScript changed my number into a string. In a lot of other languages, you cannot do this. The compiler, what interprets your code, would throw an error.

This example shows that JavaScript is a dynamic language, it changes types on the go (not just from strings to numbers but lots of others too) depending on a set of it’s own rules which sometimes don’t make sense. It’s a bit like the English language!

TypeScript is a compiler which tries to stop JavaScript from doing this crazy stuff so it becomes more predictable.

When writing code #

TypeScript can also be useful when using certain code editors.

For example, I've typed something that doesn't exist in the data structure here, and my compiler can warn me about it, meaning I don't have to see an error in the browser, then switch back to my code editor:

I can also get useful tools like this, so I have confidence even when I'm typing, that my code is going to be correct.

These two features are great timesavers, and claw back a lot of time you might have spent setting up TypeScript and debugging issues you might have as a result."

Why Astro matters

Tue, 20 Jul 2021 00:00:00 GMT

Next, Nuxt, Gatsby, SvelteKit ... there's been an explosion of frontend application frameworks lately. I've tried many (but not all) of them, and I've got to say, it's never been a more delightful experience to spin up a new project. So much so, that I've got hundreds of unfinished ones lying around everywhere.

Recently, Astro, another new frontend application framework, launched itself on the unsuspecting JavaScript public.

Whilst many of us may have been tempted to say "oh no not another one", this framework really stood out to me.

What's the point of difference with this one? Why does it "matter" so much? Well, consider this:

1. Frontend can be one happy family again #

Astro could be considered the first frontend "meta framework".

What's one of those then? It's a "set of core interfaces for common services and highly extensible backbone for integrating components this is already Java thing by the way.

Astro is essentially a "bring your own frontend" approach to modern web frameworks. You can use whatever framework (oh, ok "library" then) you know and love, and still spin up a performant app that you can host almost anywhere.

Think about the potential here. Astro could be the place the frontend finally comes together. It no longer matters (as much) what framework you use. Use them all if you like 🤷‍♂️.

Love Vue? You can love Astro. React? Same. Svelte? You'll find no argument from Astro, because Astro is the glue that underpins how we build websites and applications.

Great, innit? It'll probably never happen but I can dream, can't I?

2. Astro pushes the boundaries for every javascript framework* #

(* oh, ok library then)

Take a look at this tweet from Evan You, the creator of Vue:

Is it a coincidence that Vue now can do a similar thing to Astro? did Astro get Evan to start thinking more about this problem? Could the same be said for the other frameworks too?

Better hydration is something I've been wanting ever since the present generation of frontend application frameworks came out.

I know the React team have been working on it for a long time. I even opened (very prematurely it turns out!) this issue on the GatsbyJS repo around 2 years ago.

React 18's hydration prioritisation is a good step forward, however the whole DOM tree still need to be hydrated. Won't it be great when we need only attach JavaScript generated elements to the DOM when components really need them?!

It would be wonderful to think that partial rehydration could be everywhere, it would certainly level the playing field and even things up a lot for the next 1 billion web users.

Check out Astro #

If you care about performance (you care right?) please check out this gamechanger. I'm so excited for the potential here.

https://astro.build"

Why Atomic Design Makes Sense

Mon, 23 Nov 2015 00:00:00 GMT

Although I've really gotten into the SASS revolution, I haven't yet adopted a method for structuring elements of CSS. I guess it's because none of them strike me as particularly visual, and being a former print designer, I tend to think in those terms. But something about Brad Frost's Atomic Design principles makes sense to me. Here's why.

When we're designing, we tend to think more about design principles instead of build principles. What I mean is that we think about rhythm, white space, alignment and these types of things. These are very necessary components of great web design as they are of design in other mediums.

But the web is still a unique animal. If we're truly going to design for the web, we need to think about the medium itself, much as a printer considers the stock, the inks, and the situation artwork is going to be displayed in.

https://atomicdesign.bradfrost.com #

Atomic design really helps at this level: it helps us design repeatable elements that can be coded and re-used on different parts of our site. I don't intend to re-hash Atomic principles here, but take a look around, perhaps at this post, or read a pertinent excerpt from the book, to see what I mean.

This is going to really help us when it comes to building things out in code, especially if it's someone else building it.

Thinking Like Developers #

This is important. I don't personally think it's a requirement for designers to be able to code — but I do believe we need to think like developers. If we comprehend the way a developer approaches a project, we'll see that system-based thinking (rather than individual page-based thinking) actually helps us get a product that reflects our design intentions more completely.

And we'll also be able to empathise with our developers, which means greater understanding, greater cooperation and - again - much better result, not just in appearance but in cleaner, more efficient code that has performance benefits.

And website performance is part of our job too, right?

Atomic Design as a Developer #

As a developer, I'm wondering if I use SASS partials to structure my code. Perhaps defining files this way:

for atoms:

a-type
a-colors
a-inputs

for molecules:

m-card
m-navbar
m-lists
m-formfields

for organisms:

o-navbar
o-contactform
o-banner

and so on..

Hm, ok I'm going to try this. Hope to give you a development update in due course."

Why 'Delicious Reverie'?

Wed, 11 Feb 2015 00:00:00 GMT

I may have confused some of you with titling my personal site "Delicious Reverie". So I'm going to attempt to explain my decision, and discuss perhaps the greatest work of fiction ever produced in either French or English...

Why not benread.com? #

Yeah. Check it out: https://benread.com. That's not me. That's the other guy, named Ben Read, who happens to be a web developer. I'm sure he's a lot better at it than me.

Hilariously enough, I was contacted by a headhunting organisation recently who thought I was him; and I thankfully realised and straightened him out before things got really interesting.

A State of Mind #

So, I went with deliciousreverie.co.uk for this reason. At the time I conceived this blog, I was about 2 years into reading the formidable tome Les Miserables, in English, by Victor Hugo. I love that book. As well as the melancholy characters it contains, I loved the realism of France in the 1700s and was immediately drawn to Victor Hugo's thoughtful, at times poignant, prose.

There's one scene that I particularly loved, when a minor character, Monsieur Mabeuf, has been listening to his housekeeper relate a story from one of his old books. The old man appreciates the book anew in explaining to his housekeeper what she has been reading. The last line simply says:

"And Monsieur Mabeuf fell into a delicious reverie" #

It's out of these delicious reveries—or silent meditations if you like—that so many of our good ideas come. There is little more poignant than sitting and considering, of allowing the mind to think freely and contemplate whatever it will: a particular problem we're facing, an action we're about to take, or just to take a minute to appreciate strange life."

Why Performance Matters

Tue, 06 Oct 2015 00:00:00 GMT

Performance seems to be increasingly becoming a battleground for those of us who create websites. How has this situation arisen? How do we cope with these new requirements? How do we ensure our sites and web apps (sorry, Jeremy) don't suffer because of changing landscapes of user — and search engine — requirements.

"Everyone has broadband now. Therefore, we no longer need to be concerned about site load times" said nobody, ever. But we may have assumed that the advent of much beeefier data plans would mean that load times could become a lower priority.

This assumption ignores the fact that users could be on slower mobile networks. This has been discussed at length elsewhere. But a conversation I had recently highlighted another area that concerns a threat to the web as a whole.

When Mobile is Not Mobile #

Mobile device users are unique in that they have a choice ... they have apps on their devices which might be able to perform a similar task to what they're trying to achieve on your website.

A good example of this is a booking system. Does your website load — and respond — quickly enough for them? That window of opportunity is now measured in milliseconds, not seconds. Will they decide to use a booking system they already have on their device instead? Will they use a search, throwing up all kinds of alternative possibilities for an entertaining evening out.

This 'user abandonment' doesn't just impact on your site, on solely your service. It has wider implications.

A Performance Win is A Web Win #

If people are turning away from their browsers and towards their devices, then how is that going to impact the web in the future? Could the web eventually be relegated to a second-fiddle service, which users only turn to as a backup?

Haven't we seen that model before though? When there was no wifi, where users would have apps on their local devices simply because they couldn'taccess the web on the go. I remember taking screenshots of Google Maps data so that I could find my way around the wilds of Mid Wales. Although that's perhaps going back a step too far, I personally don't want to lose the web — not for all of its uncontrollable, unfathomable aggregation of data, truthful and otherwise, that abounds.

The web is still the portal to knowledge, to education, to opinion. It's the unique frontier, where anything can happen. If we were to lose the sense of what that means to the average user, then we have lost something quite unique, perhaps forever."

Wordpress theming with Backbone.js

Wed, 06 Jan 2016 00:00:00 GMT

You'll notice that there's not a huge amount of posts about JavaScript on my site. That's not an accident, I'm really only beginning to gain my feet with the language. Despite this I took the plunge and decided to build a Wordpress theme that uses Backbone.js to render the posts in a list. Here's a bit about the project.

Backbone.js is one of the most stable of the Javascript frameworks, which is one of the reasons I chose to focus on it. I wanted to start with a stable framework that would allow me to get to grips with it properly, and that I didn't have to worry about a huge amount of refactoring when the next version came out. But it's also one of the smallest frameworks. Even with recommended dependencies jQuery and Underscore it's still faster than some of the larger frameworks out there. This appeals to my sense of what users are most in need of from us as developers.

Where I succeeded

I'm going to talk as little as I can about the tools here. I'm exhausted at having to learn someone else's toolset to enable me to accomplish my work. I'm just going to say that it's good practice to break down your files into small blocks of code, this will really help you when it comes to debugging and maintenance of your code later on. So here's my JavaScript project folder setup:

-src
  |
  --js
    |
    ---1 underscore.js
    ---2 backbone.js
    ---3 models.js
    ---4 views.js
    ---5 collection.js
    ---6 router.js

note: I'm loading jQuery in separately from a CDN.

Structuring my project folder this way allows me to load the JS in the correct order. Now, when they get parsed by your task runner they'll be in the correct order to run.

After a few false starts I had gotten the post feed into my view, and it is outputting as I wanted. This felt like such a victory! I can now see all of my posts and click through ... but this simple bit of functionality is currently far from finished.

Where I have yet to succeed #

Documentation is patchy. I started off by being able to retrieve the basic JSON feed from /wp-json/, but couldn't find out why I can't see the posts. After some digging, I found out that the JSON endpoints haven't been implemented in Wordpress core yet. We still need to use the plugin. In fact, I enjoy what I do because I can research things easily on Stack Exchange. But there's not many people using Backbone with Wordpress, and I was distinctly on my own trying to find this out.
No links. If you click on one of the links in the list view, Wordpress takes over and you will be shown the posts' PHP page. This is because I have yet to wrap my head around how routing works.

These are some of the things I can't do yet. It really bothers me but until I find some time or it becomes a necessity, I can't justify more time on this project.

Here's the theme if you want to pick over my code."

Words Fail Me.

Fri, 27 Feb 2015 00:00:00 GMT

I'm a great fan of Virginia Woolf's Stream of Consciousness writing, but that isn't why this post is here.

As developers, the hardest thing to do is to name things. Why is that? Why is "semantic code" so hard to achieve.

Virginia Woolf, in this recording from 1937 entitled "Words Fail Me" hits the nail squarely on the head."

How to test JavaScript API Calls

Thu, 09 Apr 2020 00:00:00 GMT

In the January 2020 issue of Net Magazine, we walked through how to use React testing library to write basic unit tests for your React components. In this article I'm going to dive a little deeper and show how to write tests for some code that fetches data from an API.

This is an important distinction from what we covered previously because writing tests for UI components is very different from tests like this, and I hope that you'll learn some more things to help you ensure that all of your code is production ready, which will give you and your stakeholders more confidence when publishing new code.

Step 0. Decide What to Test #

Before we even begin writing tests it's good to decide what needs to be tested. We need to set clear boundaries before we begin, otherwise we could waste time writing tests unnecessarily. Read through your code and see what different outcomes might be generated by your code.

In our example of fetching data from an API, the API call could be successful, that counts as one outcome. But what if it's not successful? And what should happen if the call is successful, but it returns no data? That's three different possible outcomes already!

Let's look at our imaginary API call to see what outcomes exist. Here's the code we're going to test:

import env from "./ENV";
import axios from "axios";

const getApiData = (
  parameters = {}, 
  domain = env.domain
) =>  axios.get(`${domain}/api/v1/data/?${parameters}`).then(function (response) {
  // handle success
  if (!Array.isArray(data) || !data.length) {
    return [];      
  }
  return data;
  }).catch(function (error) {      
    // handle error      
    console.log(error);   
  });

Looking at my code, I can see the following outcomes:

Fetch api data
Fetch data with parameters specified
Return the data if the call was successful
Return an empty array if no data was received
Log an error if the request was unsuccessful

Looking at your code in the beginning like this often reveals other issues to you that you may not have noticed before, and which prompts you to revisit your original code and improve it.

Let's handle the first 4 tests first, then come back to the last two and see how we can improve our code.

To begin, I'll create a new file to write my tests in. The name of the file is usually the same as the module. So if my module is called GetApiData.js, my test should be GetApiData.test.js.

Setup and Mocking #

1. Mock the API #

Although this test is about fetching data from the API, I don't want to actually call the data from the API. There are several reasons for this: Primarily, it's because I'm not testing the API, I'm testing the code I have written. But also there could be a cost involved each time I contact the API, I don't want or need that cost to be incurred. Finally, I don't want to wait for the API query to resolve for my tests to finish!

To do that, I'm going to "mock" this function. When you "mock" something you essentially overwrite the function with a fake function. Let's first import the code that was written to fetch data from that API, and also the library that we used to connect to the API, Axios:

import GetApiData from "./GetApiData";
import axios from "axios";

After importing it, we can overwrite the functionality of axios like this:

jest.mock("axios");

const mockedAxios = axios.get;

Now, every time we call GetApiData in this file, and that calls Axios, it'll use our mocked implementation. Using it in the variable mockedAxios will help us identify clearly what we're doing when we write our tests.

The last thing we want to set up in regard to our API is the domain. This would be a parameter that is passed via our configuration, or part of our environment variables. But we're not testing our environment variables, so we should mock that domain too:

const domain = "https://fakeapi.com/";

2. Mock the console #

The next thing we want to mock is what we would have used in our code to log out errors: console.log(), for similar reasons we mentioned above: we're not testing the functionality of the console. Also, we don't want to actually log the errors to the console as we're running tests, but instead somewhere we can test the output.

const mockedConsole = jest.spyOn(global.console, "error");

By using Jest's SpyOn function, we can examine when that function was called, and what it was called with ... it's actually is a spy function, reporting back to us (thankfully!).

3. Mock the data that should be returned #

Finally, because we're not contacting the api, we need to provide mocked data to test against as if though it did:

const mockedDataOne = {
  id: 1234,
title: "Super Blog Post",
categories: ["1"],
_embedded: {
  
  term: [[{ name: "Category" }]],

author: [{ name: "Author" }],
},};const mockedDataTwo = {
  id: 165,
title: "Super Post Two",
categories: ["2"],
_embedded: {
  
  term: [[{ name: "Category" }]],

author: [{ name: "Author" }],
},};

Right! Let's begin our tests with a wrapping description:

describe('GetApiData() Source data so we can consume it', () => {

4. Clean ups #

Last piece of setup here: we want to reset our mocked API call and console log before each new test, otherwise we'll have stale data left over from the previous test, which could cause subsequent tests to fail:

beforeEach(() => {
  mockedAxios.mockReset();
  mockedConsole.mockReset();
});

Right, now we've set up our tests, and mocked the important stuff, let's dive into our first test ...

Test 1: Fetch api data #

Let's begin our tests with a wrapping description:

describe('GetApiData()', () => {

This wrapping function describes the component, or makes a short statement to help us understand what these tests are for. If your function name adequately describes what it does, and you don't need a longer description, that's a good sign that you have named your function well!

it("Should get api data", async () => {
  mockedAxios.mockResolvedValueOnce({ 
    data: [{ test: "Hi I worked!" }] 
  });
  const data = await getApiData(domain);
  expect(mockedAxios).toBeCalledTimes(1);
});

First thing to note: this is an asynchronous function! axios.get is already an async function so it makes sense to test it asynchronously too. It's best to make api calls async because you have a callback even if something fails, rather than the request simply hanging indefinitely, which is bad for user experience.

mockResolvedValueOnce() is a built-in function in Jest that, well, mocks the resolved value of the API call just once.

Here we're mocking the result of the mocked axios call. We're not testing the contents of the data, so I've just added a dummy object to the result of the mockResolvedValueOnce() function, since that's adequate for what we're testing.

You can now run this test, and you should see 1 passing test. Go you!

So ... it worked! We can stop there right?

Well ... how do we know our code contacted the right API endpoint? How do we know it sent the correct parameters, if we need any?

Test 2: Return the data if the call was successful #

Our next test will check that we have the data we expected in the return value of the GetApiData() function:

it('Should get data from the api', async () => {	mockedAxios.mockResolvedValueOnce({ 
  data: [ mockedDataOne, mockedDataTwo ] 
})

This time we're mocking the return value containing the two objects we originally set up.

const data = await getApiData(domain);
expect(mockedAxios).toBeCalledTimes(1);

Just as before, I like to check that we did actually call the mockedAxiosfunction. Next I'm going to check one of the data objects to make sure it has the same id as mockedDataOne:

expect(data[0]).toEqual(
  expect.objectContaining({
    id: mockedDataOne.id
  })
)})

You could do more tests, perhaps making sure that data[1] also has the corresponding ID, but this is enough to convince me that the data is returning correctly.

Now this does seem a little ... "circular" at first. You might think "of course it contains it! That's what you told it to contain!", but think about it for a minute: we haven't just returned that data. We've used our preexisting code (minus the actual API calls and real data) to return it. It's like throwing a ball, then our code caught it, and threw it back.

If nobody threw our ball back, then something is very wrong with the code we're testing: it's not working as we expected.

Test 3: Fetch data with parameters specified #

Here's our next assertion. We want to make sure our code passed the parameters we wanted, and returned the value we expected.


it('should get data using parameters', async () => {
    const params = {
    categories: ['2'],
  }

So this time our params contain an array specifying category 2 should be fetched. Remember we mocked some data in our setup? How many of those mocked data sets has the category of 2? Only one of them:mockedDataTwo.

mockAxios.mockResolvedValueOnce({ 
  data: mockedDataTwo 
})
await GetApiData(domain, params)
expect(mockAxios).toHaveBeenCalled()
expect(mockAxios).toBeCalledWith(`${domain}/api/v1/data/`, {
  params: {
    categories: params.categories,
  },
})
})

Okay, so if this test passes, our code is passing the categories correctly. Great! But does the data reflect that?

expect(data[0]).toEqual(  expect.objectContaining({    categories: ["2"],  }));

If this test passes, then great! We have successfully obtained data with the correct parameters.

Another check to do here is that the data only contains items with this category, and not any other. I'll leave that one for you to figure out.

These next two tests are to verify we have captured two significant branches, or outcomes, of our code: failures.

Test 4: Return an empty object if no data was recieved #

If there hasn't been any data sent back to us after the API call, we have returned an array as a fallback so that we don't have an exception in our data layer. that can be used by our UI to provide a fallback - once the API call has been resolved.

it("Should return an empty array if no data was recieved", async () => {  const data = await GetApiData(domain, params);  mockAxios.mockResolvedValueOnce({ data: null });  expect(mockAxios).toBeCalledTimes(1);  expect(Array.isArray(data)).toBeTruthy;});

We're mocking a data object with a null value here to represent no values being returned from the API call. We're using Array.isArray because that is far more robust than using isArray, which is an older method that returns true for a number of different cases (don't ask...).

Test 5: Log an error if the request was unsuccessful #

Logging errors is a vital part of a robust application. It's a great way of being able to respond to API failures or application exceptions before users get to see them. In this test, I'm just going to check for a console.log()call, but in a production app, there would be an integration with some external logging system that would send an email alert to the dev team if it was a critical error:

Our final test uses our consoleMock from our initial setup (see above):

it("Should log an error if the request was unsuccessful", async () => {  const error = new Error("there was an error");  mockAxios.mockRejectedValue(error);  await GetApiData(domain);  expect(mockAxios).toBeCalledTimes(1);  expect(mockedConsole).toBeCalledTimes(1);  expect(mockedConsole).toBeCalledWith(error);});

the consoleMock function allows us to mock the functionality of the console.log object. Because we're testing that an error is thrown by our code, we need to use the Error object to test the output correctly.

So there we are ... we now have a suite of tests to give us more confidence that our code is production ready ... as long as the tests don't fail in our pipeline, we can be confident that we have met the core criteria for our GetApiData function.

Conclusion #

There's a lot to these functions and it can take quite a bit of time to get used to writing this much code:- more than our actual function! But what is the price of confidence? ... if you think about it, by spending the time writing this code, we could have saved our company hundreds of thousands of pounds from lost income if it was broken!

I would say that thoroughly testing your code is an important step, along with static typing, quality checking, and pre-release validation, to ensuring that your code is indeed production ready!

Boxout: The price of confidence #

Developers will spend more time writing tests than writing the components they’re building. That makes sense if you think about it: you need to test every possible outcome of the code that’s being written. As is demonstrated in this article, one API call with some basic functionality can result in a number of differing outcomes.

The benefit of adding tests to your code can easily override the time spent by developers following this practice. If your business or customers needs the confidence that things won’t break, then testing is definitely a good practice to introduce at the start of a project.

Other ways that testing can benefit a project include during refactors. Often project requirements will change after the code has been written. That introduces more risk into the codebase because on revisiting the code a developer might decide to refactor to make it simpler … which could include deleting things that were actually needed! Looking at the test serves as documentation: developers can see that there was a decision behind every code outcome that has been written.

Boxout: Scoping outcomes #

The hardest part of finding out what to test is knowing what your code actually does. This becomes harder with the more time that passes between when you write tests to when you write the actual code. So I recommend writing tests alongside the component, or even before you write your component.

When you’re doing this you’ll be more clearly able to think about all of the different outcome possibilities that your code offers: what variables might change? What different return values are possible?"

2015 Year in review

Thu, 10 Sep 2015 00:00:00 GMT

For the past year and a bit, I've worked at Blaze Communication, a marketing agency in Enfield. I've learned so many new things at this company. My knowledge and understanding is far, far different than it was a year ago. Here are some of my observations.

All of these things could be blog posts in their own right; but I'm a fan of brevity.

How to Sell Change #

Right when I started working for Blaze, I got really annoyed and frustrated because I wanted to change how things were done to the way I was used to ... I'd come from an organization working to very different ways of planning and development, and I wanted to implement all of them straight away.

I soon found out that change needed to be justified in terms of benefits. So I looked for opportunities to raise the issues I was most concerned about.

After a few strong - but not unpleasant - conversations in which I challenged the validity of certain processes, I got some of what I wanted.

But I also had to see things from the company's perspective. They weren't interested in implementing things unless they could be convinced of a clear benefit. And I realised that some of the things I wanted to do weren't going to fit.

I guess I learned that Agile also needs to be Agile.

How to Convince with Tact #

It was clear from day one that design skills (or UX skills) were not to be part of my role. However, I could see a gap in the skills of the team when it came to understanding the users' perspective.

I needed to find a way of doing this tactfully, so I wouldn't offend their sensibilities. I eventually succeeded in doing this by being patient, building trust with my design colleagues, and complimenting them when I felt there was a basis for doing so.

Eventually, I was glad to see more talk of "what's best for the user" near the end of my tenure. I really hope they continue to learn UX based thinking after I move on.

How to perform basic SysAdmin tasks #

The answer: With Great Caution. The first time I transferred in a domain from somewhere else, I knocked a site out for a few days. Also: because they say DNS propagation can take up to 48 hours doesnt mean it will. It's usually done in an hour or so.

In the last few days I've finished setting up a new server for the company, although a third party did a lot of the leg work it gave me a lot of insight to how these things happen.

Deciding which version of PHP to install, and the different methods of running it, was a real eye opener for me.

Migrating a server with 20+ live sites and many DNS records can be fun :-)

How to effectively Maintain older sites #

The main thing is, don't be too precious about them. If you need to add a lot of extra Javascript just to get things to perform the way the beief describes, do it.

If it's an older site and otherwise badly needs updating, it shows that the one responsible doesn't care too much about it's upkeep. So why should you invest your care to a greater degree?

How to use CSS Animations #

Because design thinking wasn't part of my remit, I had to find another outlet for my creative thinking. I found plenty of scope in CSS Animations.

When I could see from a design a case for animating something, I would seek a way of doing that in a subtle way. I found that softening page transitions and hover states gave me a way of using time as a canvas.

Other stuff #

How to build a Wordpress theme from scratch, how to use SASS, how to remove a hacked site from blacklists (once cleaned), how to build extensibly, how to manage & support remote developers, how to build a theme that's easily translateable, how to train clients in the use of their CMS.

And probably tonnes more things."