How Visual is Your Language? Semantic Mutation Testing
What is Mutation Testing?
Mutation analysis is defined as using well-defined rules defined on syntactic structures to make systematic changes to software artifacts.
Mutation testing is defined as using mutation analysis to design new software tests or to evaluate existing software tests.
It consists on mutating a program and checking if the results differ from the original.
Visual Languages
When creating a visual language we can use many dimensions (syntax) like color, position, order, shape, orientation, texture, size and others to define language semantics.
The thing is that when drawing on a grid of pixels we will "accidentally" use other dimensions that add no extra meaning to our programs.
Visual Mutation Testing
To test how visual our language is and to notice which dimensions we are "wasting" I propose to do mutation testing on those dimensions and see if the meaning of the program changes.
Why?
Imagine a textual language that required us to name things or write symbols/sections that aren't used for anything in the program.
If a visual language doesn't give any meaning to absolute and relative position of items and the length and shape of the connections carries no extra meaning, then why have them?
How many "box and arrow" languages are just scratch with arrows between the blocks?
Seen from the other side, each wasted dimension is an opportunity to encode meaningful information about the program.
Below are some "rules" to apply and check if the dimension has any meaning.
Mutations by Dimension
Color
Display in black and white
Display in grayscale
Changing Hue, Saturation and/or Lightness of colors
Position
Move one value in the canvas by X and/or Y
Order
If A and B are related and A comes before B horizontally and/or vertically change their order
Shape
Swap shapes
Turn a shape into another
Connections
Make a connection shorter, longer or of length 0
Make the connection wider or narrower
Change connection shape/angles/path
-
If a connection has a direction, revert it or remove it
There are languages were the flow direction is specified by something else, like node type
Orientation
Rotate shape by N degrees
Texture
Change/Remove textures
Size
Make things bigger/smaller
Dimension Redundancy
-
If a mutation is applied to one of the dimensions above, must another dimension be altered to keep the new version equivalent to the original one?
Not necessarily a bad thing, having a redundant dimension for something like color may be useful for people with color vision deficiency
Any others you can think of?
What's the most/least visual language according to this criteria?
Let me know @warianoguerra