Elizabeth Hudnott reflects on her recent Remote Nest Residency.
Hi! I’m an artist who makes interactive digital artefacts which seek to create complexity from simple rules. I love geometric shapes, bold colours, mathematics and computer programming. I invite the reader to become an integrated component in my creations by customizing the pieces to produce your own unique renditions.
I’ve built a software environment that contains a selection of sketches, which are at varying degrees of development. Each sketch contains two parts. Obviously there is computer code containing drawing instructions necessary to produce the artwork as I intend it to be. But when you load a sketch you only see one configuration of the artwork among many. Each sketch also creates a user interface that bestows the observer (or rather the participant) with the ability to adjust many parameters that feed into my code to extract one chosen image from a near infinite sea of possibilities. Indeed, it’s a requirement that the participant explores the parameter space because the default settings just replicate textbook images.
My creation effort is divided into two distinct parts, the sketches and the environment in which they operate. The latter provides functionality that is shared between sketches, such as video rendering.
Now that the larger stage has been set I can say that I took a single sketch that began simply and that this particular project mostly focussed on my journey to seek out increased complexity and variety for that chosen sketch. Truchet tilings have the property that every line that touches a tile’s perimeter does so at one of a small number of permitted points on that perimeter and furthermore rotating the tile through any multiple of 90° retains this property. For example, if we choose to allow lines that pass through any corner of a tile then rotating a tile will cause a line to point towards a different corner. Strictly speaking, the definition of Truchet tiles is limited to the particular collection of four tiles that Sébastien Truchet designed, but it’s this particular property about his designs that fascinates me. Cyril Stanley Smith developed alternative designs inspired by Truchet and my designs are an extension of Smith’s.
The sketch originally used three tile designs, two of which replicated a pair from Stanley Smith’s work and the third was a blank tile. An obvious way of extending the concept is to design more tiles. I began by sketching eight sets in a paint package, each consisting of several dozen designs. Two of those sets were taken forward to the coding step. These two sets are interoperable with one another because they only differ with respect to whether straight lines or curves are used to connect the points of interest. The new designs have lines touching the perimeter at the mid-points along the sides of the tile. This switch to different anchor points and away from the corner points which I had used previously will become significant later. The term “anchor points” is not a mathematical term but is rather a piece of linguistic innovation created by my internal monologue. Another of the eight sets of designs is an extension of the original diagonal scheme, so the prior work will be resurrected at some point in the future. Eventually I hope to have all eight schemes implemented.
I decided to use my design drawings purely to collect my thoughts together and communicate them to the Talking Birds team. I could have loaded these graphics directly into my software but I want more flexibility than than fixed graphics allow. Therefore I had to add written descriptions of each tile to the program, expressly mentioning each line or curve and the coordinates of its start point and its end point. Although through judicious use of
If… then… else… this was actually realized as a single giant recipe with multiple branches that can produce almost any tile design in the set. Implementing the new designs involved reorganizing the code as a whole. Previously the layout of the code was very simple, as the following pseudocode demonstrates. Pseudocode is made up language that’s more rigid than English but isn’t sufficiently detailed to be a runnable program.
r = random() If r < threshold then Draw a leftward leaning line Else Draw a rightward leaning line End If
This would have become unwieldy as more designs were added and the tile selection process became more complicated. Therefore I replaced it with a model where one section of code describes the design of a tile and the drawing commands needed to produce it but isn’t aware of any details about how the tile is being incorporated into a larger design. A second section of code arranges tiles into a design but doesn’t contain any drawing commands. A third section of code sits halfway between organizing tiles without knowing what they look like and being a tile graphic without a context. That third position represents a specific tile that’s involved in the layout and it has particular colours chosen for it. The tile designs themselves are expressed as abstract lines without colours, although they can indicate suggested or prototypical colours. More on that later. The three parts have to communicate with each other in very precise ways to ask questions of one another and supply answers. For example, the layout algorithm will want to ask a tile design how many of the four possible anchor points it uses and which ones those are. This sketch expanded to become my largest sketch yet in terms of the number of lines of code needed. Therefore, developing a competent engineering design that adequately supported my artistic aspirations was definitely a piece of this project and these two sides, artistic and technical, grew and expanded together. That said, my code does mostly just flow off the tip of my tongue in whatever way I feel expressing my artistic creativity, and while there certainly are software architectural decisions involved the product is certainly not designed with the kind of robustness required for a business application or teamwork project.
My master to-do list for this sketch has a section about incorporating different tile shapes. I’d like to add hexagon shaped tiles and even Penrose tilings into the mix. But hexagons work differently from rectangles. Grid lines are particularly easy to calculate for rectangular grids and so for now hexagons remain something for the future. Squares, rectangles, parallelograms and chevrons all share a property that makes them easy to lay out. Hexagonal and triangular tessellations aren’t excessively difficult to implement but my methodology is to build out in successive generalizations, one at a time. I was about to implement a rectangular grid but I knew the next logical generalization of that component would be extension to incorporate parallelograms and chevrons. As I sat down to write the line drawing instructions for the new tile designs I also knew that if I calculated the coordinates of all the points using an assumption of a rectangular grid then it would be particularly fiddly and cumbersome to go back and edit the code to make a parallelogram or a chevron generalization later. So I went ahead and implemented them straight away. During my meeting with Talking Birds it had been said that I shouldn’t feel constrained by my original pitch and it felt really good to be able to do what was best for my overall project vision without feeling I was going against my supervisor’s wishes, as would have been the case under a more conventional business relationship. I hate cognitive dissonance, so avoiding that was definitely a plus.
Unfortunately the parallelogram shaped tiles initially turned out to be a massive flop that failed to live up to my over inflated expectations. My reaction when I first saw them was, “Well done Elizabeth, you’ve put the whole picture into italics! What good is that?” At least, that was my initial impression when I viewed the early parallelogram based images having “Random” colouring and 100% colour flow, which are two parameter settings I’ll describe in the next section. Applying those terms to this early stage of development is anachronistic however, because all the lines were being drawn in a single colour at this point. I recently discovered that other people have the ability to see images in their mind’s eye but I do not. I’d always assumed the phrase was just a figure of speech that people say when they’re intensely concentrating on something. So aphantasia sounds like a good excuse for my minor failure and I’m going to roll with that whether or not it was truly the cause of the mishap! But it’s interesting to see how these parallel journeys that I’m on intersect, growing from a hobbyist into a “proper artist” while also developing understanding and coming to terms with my disabilities and trying to think of ways to adapt things to work better for me, but also valuing the things that do come easily for me. Early on I quickly and easily got this complex sketch mapped out, with its many desirable features and components categorized and organized hierarchically in my “Master To Do List”, and that’s a different kind of visualization.
In contrast to the parallelograms, the chevrons were a spectacular hit from my perspective, which was totally unexpected as I mostly threw them into the game plan for completeness and I had felt they’d be inferior to the geometric purity of rectangles and parallelograms. Clearly I learned nothing from reading Flatland! Chevron shaped tiles gave the images a kind of non-uniformity that had been missing up until this point. And they have a pleasant flowing, calligraphic look about them. So I’m very pleased that I went on the detour to make chevron shaped tiles. I’d really like to build further on this feature in the future. Tiles will tessellate with any kind of extrusion glued onto the basic rectangular shape so long it’s created by cutting out a correspondingly shaped intrusion from the opposite side, as was done so spectacularly by Escher. The parallelogram and the chevron are just the most basic examples of this. I think a quadratic curve would be the next logical step. So far I’ve been distorting my square tile designs to fit other shapes, but specifically designing other shaped tiles could also be an option going forward.
As for the two parallelogram parameters, which may be adjusted via the controls labelled “Bottom” and “Right” on the Grid tab, which are short for, “Bottom edge x-offset”, and, “Right edge y-offset” respectively, they’re still there and they aren’t completely useless. In retrospect it’s quite useful to be able to shift a little bit away from rectangular tiles by adding a slight slant to break up the 90° angles. The same controls can be used to refine the chevron shape also. It’s just that the parallelogram isn’t the radically different alternative shape choice that I thought it was going to be, whereas chevrons are.
Previously the sketch only supported two colours, one for the leftward leaning strokes and another for the rightward leaning strokes. These colours were customizable and each “colour” could actually be a colour gradient that blended between two colours. But essentially there were two colours and wherever a particular tile design occurred in the tessellation it was always drawn in the same colour. The colours didn’t consistently follow the lines as they passed off of the edge of one tile and onto the next, twisting and changing directions.
I wanted to make the colours flow. Implementing this proved more difficult than I expected but I don’t remember why. If I’d retained the diagonal lines then the task would have been slightly more complicated still, because a line that goes to a corner can abut up against three other lines drawn on adjacent tiles in the maximal case. Using the mid-points as anchor points constrains each anchor point to have just one corresponding anchor point on one adjacent tile (or none at all at the edges of the picture). The right side of one tile connects to the left side of another, etc.
The sketch now supports both colouring modes. In the “As Designed” mode, which is the default, the layout algorithm takes the prototypical colours offered by the tile designs as uses those as the actual colours. Colours don’t flow from tile to tile, not all of the time anyway, although the specific details of the designs and particular organizations of tiles may permit some small localized flow of colour to occur occasionally. In “Random” mode the prototypical colours are ignored and the system picks a random colour and makes that colour flow onto any adjacent tiles, then their adjacent tiles, and so on.
Prior to this project I discovered it’s often a bad idea to create parameters with discrete options because they produce sharp transitions during animation, which sounds like an obvious thing but the reality is more nuanced than that. Moreover, I discovered a discipline of reimagining things that I initially thought of as discrete parameters as continua instead. This discipline led to the conception and subsequent implementation of a parameter that controls the probability of colour flow. When set to 1 then colours always flow from tile to tile. When set to 0 a new colour is always selected when a line crosses a tile boundary, which looks similar to but differs in construction from the “As Designed” mode. Then the feedback process from idea to thought to code to visualization and back to a new idea led to a refinement of this flow / no‑flow concept. Rather than selecting any new colour from the palette when a no-flow situation occurs, I instead decided to organize the colours into groups so that a yellow line can change into a blue line but not into a green line, for instance, because green is grouped with red. This involved another new parameter to decide how many colours should be in each group. I find the final result quite pleasing.
The flow probability slider did not resolve my struggles with animation though, because “As Designed” versus “Random” wasn’t the only sharp contrast hanging around. Unlike my other sketches, this sketch has many aspects which involve forms of interconnectedness that produce drastic changes between successive images as a result of only small changes to the input parameters. Animations become trippy messes without narratives. Consider this change that’s easy to visualize. The “Random” colouring algorithm works by answering the question, “Which tiles are connected to each other?” As in, “Is it possible to trace a line from this tile to that tile, possibly via some other intermediate tiles?” But if I change one tile from depicting a vertical line to having a horizontal one instead then suddenly lots of tiles that were connected to each other are no longer connected and simultaneously it could be the case that many tiles that weren’t previously connected now become connected. It remains an open question as to how many of these animation issues can or could be resolved by making some adjustments to the algorithms. In the previous example we could potentially force the modified tile to incorporate a no-flow rather than always deciding randomly and that would mitigate sudden changes in colour. But then what if the two shapes which are becoming connected previously belonged to two different colour groups? The interconnectedness makes designing and implementing this sketch very tricky. Ideas that sound simple turn out not to be so.
I wanted the sketch to have a retro computing feel. I really like the way that people made compelling video games in the 1980s on machines that restricted artists to 4, 8 or 16 colours. I like the bold colours of the blocks in Tetris and I remember the sixteen colours of the default EGA palette. There’s something wonderfully symmetric about powers of two and having three primary colours and three secondary colours, together with black and light grey for a total of eight colours, then having a second set consisting of eight brighter versions of the same. The plan was to permit the user to create pieces with between two and sixteen colours and to let them choose the colours because that’s an obvious way to add parametrization. And the current version does indeed permit between one and fifteen customizable colours plus a background, which can be either another colour or an image.
Even though the colours are customizable, I still wanted a default palette that I liked. My plan was to take the EGA palette and tweak each of the colours so they were somewhat displaced from their usual positions but retained their boldness without being recognizable. I experimented with tweaking the colours for a considerable amount of time without being satisfied. Until the realization suddenly clicked that it wasn’t the EGA palette that I desired but it was the colours of the London Underground map that my brain was craving.
Another consideration was how to parametrize colours. I could have gone with the three primary colour RGBA system (Red, Green, Blue and Alpha, i.e. transparency) but in the end I settled on the HSLA system (Hue, Saturation, Lightness, Alpha). I’d originally wanted to flow not just blocks of colour from tile to tile but also have fading between say lighter and darker versions of the same colour along the length of the line. I unfortunately haven’t managed to pull that one off yet, but one day it will come. I needed to make a colour chooser and I was already considering how the existence of colour fades would affect it. Unfortunately the HSLA system isn’t great for producing smooth transitions of colour because fading along the hue axis will generate rainbows. Which is great for my inner gay but sometimes one does want to be able to fade between red and blue without passing through orange, yellow, green and turquoise! Fading between opposite colours like blue and yellow in the RGBA system will produce a a bunch of muddy greys in the middle. I find the HSLA system very useful for my artwork because its concepts like saturation are pretty easy and intuitive to grasp, though the RGBA system is acceptable too. But neither is great for colour gradients. This issue affects my other sketches too. My sketch that draws Julia Set and Mandelbrot Set fractals sometimes produces weird colour gradients because it’s based on HSLA. Another colour space is needed. CIELAB (Luminosity, A* and B*) seems promising but the A* and B* dimensions are less intuitive than HSLA and at the time of writing web browsers don’t support LAB as a built-in feature (but future support is planned). If I wanted to use it today then I’d have to program the necessary calculations to convert numbers from LAB into RGBA myself.
Another issue which afflicts both RGBA and HSLA is that until recently computers have been able to display bright reds and greens but their range of blues has been more limited. It doesn’t make sense to place red and blue measurements on the scales we’ve always historically used, which is a particular problem for HSLA because
hsla(0, 100%, 50%, 1) and
hsla(240, 100%, 50%, 1) don’t just differ because the former has a red hue and the latter has a blue hue as should be the case because that’s what those descriptions say. But in reality they also differ in brightness because 100% saturation and 50% lightness doesn’t mean the same for red as it does for blue, because the measuring system is based upon the historical limitations of displays rather than the full range of human vision.
Probably the eventual solution to my needs in the medium term will be to continue accepting input in HSLA colour space and then convert those colour measurement into LAB space, perform my colour gradations and animations in LAB space, and finally convert into the computer’s native RGBA space to generate output. But there’s more to this sketch than the colour gradations that I didn’t end up implementing and HSLA is good enough for now. Valuable research for future work though.
There’s currently a glitchy behaviour associated with the alpha colour dimension. When you have a line meeting a curve on the same tile and the colours aren’t fully opaque then you get an effect that’s like looking through two overlapping coloured lenses and the intersection piece comes out darker than it “should”. I did regard this as a bug, though it could also be considered a feature. It warrants further investigation for creative potential.
Originally the plan had been to use my initial drawings of tile designs as thumbnails and have users choose tiles from a broader catalogue of designs. But the number of designs had grown quite large and I became concerned that if I implemented the catalogue approach then the user interface could end up being cumbersome. Noting that my designs are built from simple components, I determined there would be sufficient constraints imposed on the user that creating a tile designer widget and embedding it inside the sketch would be eminently feasible and would provide a more direct way of engaging in tile construction than scanning through a list. And so I implemented the tile designer widget. You can click in any of eight areas to toggle lines on and off, convert lines into curves, or change colours. And I think it’s a great success because very quickly I was playing around and creating new designs that were not on my original list.
Initially the tile designer only supported drawing lines in a single colour. When it came to extending it to support multiple colours I hit upon a new challenge. My original designs adhered to a discipline whereby whenever two lines on a tile touched one other then those lines were either the same colour or they crossed over one another clearly at 90°. But now that I had created this open ended tile designing interface it would be arbitrary and difficult to impose the same restriction on others. So I had to go back to pen and paper and draw out every possible tile design and decide how the different coloured parts were going to mesh together. Then I went back and rewrote most of the tile drawing code so it could accommodate the necessary new additional design elements.
The tile designer has some limitations. Sometimes it’s a bit fiddly to use. When it’s not doing what I intend then I find I need to click a little further away from the middle of the tile and a bit closer towards an edge or a corner. Sometimes creativity may lead you to attempt something that the tile designer won’t let you do. In order to work with the “Random” colouring mode every line needs to connect with one of the tile’s anchor points so that it has somewhere where colour can flow in from. Sometimes you’ll attempt to say draw a yellow line but it’ll insist on drawing a blue line instead because a blue line already exists in an adjacent position on the tile. However, there isn’t a good reason for imposing this constraint when the tile designs are being used with the “As Designed” mode. This problem also suggests that a hybrid mode would be desirable, whereby when the “Random” method is chosen then colours would flow in from the anchor points whenever possible but any inaccessible lines would be rendered “As Designed”.
Furthermore, what application of colour in the tile designer even means with a “Random” colouring mode is not explained clearly within the software, so I’ll explain it now. If you have a design with some red lines and some green lines then it means that red lines will be drawn in the same colour as one another and the green lines will also be drawn in a consistent colour. However, what those colours actually are has nothing to do with which colours were used in the tile designer were because the whole point is that the colours are chosen randomly. So red and green or blue and yellow, it doesn’t matter. All that matters is that two colours are being used. When the tile is placed into the picture “red” and “green” may even end up being the same colour! Though it might be handy if there were some way of choosing whether or not to force the system to choose two distinct colours. Enforcing additional constraints is a broad area that offers a rich tapestry that’s ripe for further innovation, as we shall see.
The tile designer permitted new designs that weren’t in my notebooks (on paper or digitally). But conversely I also had a few designs in my notebooks that didn’t fit within the simplistic expectations of the tile designer. I think the best approach going forward will be to offer a smaller catalogue of “specials” that can supplement the tile designer when needed, but I haven’t quite reached that stage yet.
By this point I was close to reaching the suggested time commitment for this project and I’d achieved all of the objectives I set myself in my pitch except for one thing, which is that I wanted to have lines with fading transitions of colour. But when I started thinking about how I would achieve this I realized that it wasn’t very straightforward and would involve editing many different portions of the program. I didn’t want to run out of time and have the task half completed because half complete code doesn’t look half complete, it just doesn’t run. So instead I decided to look for low hanging fruit. I added a second line width parameter so that horizontal lines can have a different thickness to vertical ones, which enhances the calligraphic feel. I also wanted to address an issue that I had on my master plan as a later follow up item but which became a pressing issue now that the algorithms were close to producing images that looked a step up from where I’d started.
The biggest challenge to adding colour gradients is that lines often loop back on themselves, so getting seamless joins at every point of intersection requires some thought. Secondly, each tile doesn’t contribute an equally long piece to the total line. A horizontal segment is as long as the width of the tile, a vertical one is as long as its height. A diagonal line will follow Pythagoras’ theorem. That’s assuming rectangular tiles, but what about those parallelogram and chevron shaped tiles? Then there’s the curved segments. They look like quarter doughnuts but that characterization is an approximation because I actually draw them using cubic Bézier curves and in the future I want to give users the ability to change them into different looking curves by moving the control points, much as you would do in a program such as Photoshop.
Introducing colour flows had started giving my images some form. But there were often instances where a tile didn’t connect with any adjacent tiles. For example, these could be curved pieces that didn’t connect to anything at either of their two ends. I named these tiles “wispy bits”. I wanted a way of ensuring each tile looked like it had a purpose, so I devised a plan whereby I would be able to configure a minimum number of connections for each type of tile. This allows me to express, for instance, that a straight piece or a curved piece should be connected to something at least one of its two ends or that a T-junction should be connected at least two of its three points.
Implementing this proved more difficult than I’d anticipated because the layout algorithm selects tile designs to go into spaces by scanning from left to right and top to bottom, but sometimes it would get stuck in a situation where none of the tile designs in use could be placed in a space because the combination of tiles around it didn’t permit it due to the constraints. I adjusted the algorithm so that when it gets stuck it will now go back and make a fresh tile selection for the space to left of the current one. And if that strategy also fails then it’ll make a fresh selection for the space above the current one. But sometimes even these accommodations are not flexible enough, in which case my algorithm just gives up and violates the constraints. It’s intellectually dissatisfying knowing that there are cases where a valid picture exists which meets all of the stated requirements but my algorithm isn’t sophisticated enough to discover it. On the other hand these failures usually occur when it’s presented with a limited choice of tiles combined with high degrees of constraint, which isn’t the norm.
Adding constraints took a bit longer than I expected because my initial attempt contained some programming errors, as tends to happen, but ironing out the creases in this particular section of code was more difficult than usual because there was a lot of data that wasn’t easily visible concerning combinations of tiles that were being tried and which ones were being accepted and which were being rejected, and all of these decisions happen before anything gets drawn on the screen.
The result is partially satisfying. Wispy bits in the original sense can now be avoided when desired. However, it’s still possible to form very short lines that only span two tiles. More sophisticated kinds of constraint will be needed before a level of control can be attained which fully satisfies me.
Enhancing The Environment
My long term goals for the software are:
- To add more sketches, and to add more parameters to existing sketches.
- To add social elements that will let users can share their creations with each other.
- To grow the user base and increase product recognition.
- To facilitate users to create their own sketches.
- To enhance the sensory experience using sound and motion.
I envisage my software will arouse interest for four purposes, which I definitely hope are not mutually exclusive. In no particular order:
- To produce abstract art pieces that are shown and appreciated online or in print like other art.
- To offer a space to experiment, visualize, develop intuitions, form personal theories and learn the maths behind the sketches, for those who have an interest in recreational mathematics.
- To provide sensory stimulation for sensory seeking neurodivergent individuals with conditions such as autism, as a relaxing activity.
- To stimulate interest in coding.
To satiate the third need I would like to add an ability to translate the evolution of musical qualities into changes to the visual parameters. I think synchronization with music along with the ability to control parameters using hand gestures would be two really fulfilling features for me to enjoy as a person with neurodivergent traits and for others like me.
I’m not currently knowledgeable on the topic, but I wonder if my music visualization aspirations could also be adapted into a useful tool for deaf and hard of hearing people. As a hearing person I remember around the turn of the millennium when products such as Microsoft Windows Media Player 7 were launched and music visualization algorithms were an exciting new way of understanding music. But the novel element wore off quickly. I’m seeking to create a much richer experience in terms of customizability and personalization but I need to research if there’s genuine potential to communicate useful information in an accessible way or not.
So far the tool has been entirely grounded in me and my needs. There’s still a considerable amount of work to be done in terms of engaging with online enthusiasts, teachers, and other disabled people.
As an artist I’ve got many ideas for new sketches that I’m eager to get stuck into. But taking the environment to the next level by making it into something special will/would be exciting too. It frustrates me that I can’t be working on both simultaneously!
When I began this project, the sketch had five limitations that I wanted to address.
- Only one set of tiles was available and that tile set only included two tiles (plus a blank tile).
- There were only four points available on each tile (the corners) to join lines between tiles.
- The lines were always straight.
- The colours didn’t always flow naturally from one tile into another.
- The computer code was rigid because it was written to draw those two shapes only. The code didn’t contain an explicit model of the high level properties.
- Eight styles of tiles have been invented and two of these are available for use. They’re now styles rather than sets of tiles because the tile designer interface doesn’t constrain the user to a small predetermined collection anymore.
- There are still only four anchors in active use, though they’re different ones from previously. Internally however though the code has been rewritten in an abstract way so that it may support any number of anchor points in future, with sixteen already defined.
- Lines can now be straight or curved.
- Colours do flow naturally from one tile into another, though not with colour gradients yet.
- The code is more flexible. New tiles can easily be coded separately and plugged into the general algorithm.
- Chevron and parallelogram shaped tiles are possible. Users can design their own tiles using the built in tile designer and refine the layout by imposing constraints. An initial proof of concept for user programmed sketches has been created.
Overall, the project has been a success. The next task is to keep the momentum going and plan what comes next. Working with Talking Birds has boosted my aspirations and raised interesting questions about possible future directions for my work.