Skip to main content

How Visual is Your Language? Semantic Mutation Testing


What is Mutation Testing?

Mutation analysis is defined as using well-defined rules defined on syntactic structures to make systematic changes to software artifacts.

Mutation testing is defined as using mutation analysis to design new software tests or to evaluate existing software tests.

It consists on mutating a program and checking if the results differ from the original.

Visual Languages

When creating a visual language we can use many dimensions (syntax) like color, position, order, shape, orientation, texture, size and others to define language semantics.

The thing is that when drawing on a grid of pixels we will "accidentally" use other dimensions that add no extra meaning to our programs.

Visual Mutation Testing

To test how visual our language is and to notice which dimensions we are "wasting" I propose to do mutation testing on those dimensions and see if the meaning of the program changes.


Imagine a textual language that required us to name things or write symbols/sections that aren't used for anything in the program.

If a visual language doesn't give any meaning to absolute and relative position of items and the length and shape of the connections carries no extra meaning, then why have them?

How many "box and arrow" languages are just scratch with arrows between the blocks?

Seen from the other side, each wasted dimension is an opportunity to encode meaningful information about the program.

Below are some "rules" to apply and check if the dimension has any meaning.

Mutations by Dimension


  • Display in black and white

  • Display in grayscale

  • Changing Hue, Saturation and/or Lightness of colors


  • Move one value in the canvas by X and/or Y


  • If A and B are related and A comes before B horizontally and/or vertically change their order


  • Swap shapes

  • Turn a shape into another


  • Make a connection shorter, longer or of length 0

  • Make the connection wider or narrower

  • Change connection shape/angles/path

  • If a connection has a direction, revert it or remove it

    • There are languages were the flow direction is specified by something else, like node type


  • Rotate shape by N degrees


  • Change/Remove textures


  • Make things bigger/smaller

Dimension Redundancy

  • If a mutation is applied to one of the dimensions above, must another dimension be altered to keep the new version equivalent to the original one?

    • Not necessarily a bad thing, having a redundant dimension for something like color may be useful for people with color vision deficiency

Any others you can think of?

What's the most/least visual language according to this criteria?

Let me know @warianoguerra

Found it interesting?

Subscribe to instadeq's monthly newsletter to stay updated