A deep dive into an unexpected occurrence.
It is the time of Covid. We are stuck in the UK. We are installing a digital art show called Microworld in Norway using RemotePC so we can see what the people in Norway can see. With this software it is important to never turn the remote computers off – to leave them connected to Wifi all the time. If you turn off the computer then the RemotePC software can’t see the computer any more because it is not on Wifi, and therefore you can’t turn it back on. It is gone for good. So you have to contact a human at the other end in Norway and ask them to climb up a ladder and turn the PC back on manually. Which is a pain.
So we have been experimenting with leaving the computers on all the time and just turning off the projectors each night.
One morning we remotely looked at Hector & the Sunstars program, which was left on all night. The program was running fine but we noticed the blue sunstar was missing!
Did Hector eat it? No that’s impossible within the program.
So what happened? We have never seen this behaviour before – did the sunstar manage to escape off screen? Maybe processing glitched after being on for so long? But the rest of the program is running fine, Hector and the red and green sunstars are behaving normally. What happened to the missing blue sunstar?
This didn’t upset us by the way – it excited us! This is a great example of the artwork surprising the artist. Which, believe it or not, is one of the main aims of Microworld, and we would say generative / interactive art in general. For all of us to see new things.
A painting is fixed. A film changes over time but is still the same film. Digital art is the best because it has the potential to be different every time it runs.
In Microworld with multiple interacting pieces – we are looking to create an ecosystem of actions and reactions that are so complex they are impossible to predict. The pieces adapt to what is going on in the space and emerge into previously unseen states. A missing sunstar is very unexpected.
This unexpectedness or emergence is the holy grail of interactive ecosystem art. We once returned to an exhibition in Leicester after a month away. This is before we used RemotePC when we had to physically visit out exhibitions! We were astonished to discover that one of the pieces which comprised of hundreds of digital ants leaving trails had made these new patterns on the wall. Black and white flickering dots. This should have been impossible in the program – had the ants behaviour evolved somehow? The explanation was more prosaic. The cleaners in the exhibition space were having troubles with their hoovers interfering with the projectors – so the projectors had just been left on the entire time of the exhibition. This is back in 2014 before long-life LED projectors we use now. Eventually after continuous use the projectors had just burnt out in places – causing what is know as dead pixels.
Eliminate the possible
So to solve this case we’ll need to take a much closer look at the crime scene – or in this case the code. You should get a good idea of how we debug a program. The step by step process of eliminating things that might have gone wrong. We should say that we don’t know the solution in advance, so following the advice of Sherlock Holmes, we are going to eliminate some possible theories.
There are 3 ways that an object which is drawn to the screen on every loop, could go missing in Processing
1) Has it wandered off the screen ?
2) Has it become too small to see ?
3) Has it become too transparent to see ?
1) Has it has wandered off the screen?
There are 3 sunstars in the program, red, green and blue. They all use the same code and have their own position, scale, colour, and many other variables which make them tick.
Most computer programs have a central loop, which just goes round and round until the program is stopped by the user. Hector and the Sunstar’s loop contains these functions :
doWebcam();
movePlankton();
moveSunstars();
moveHector();
drawPlankton();
drawSunstars();
drawHector();
There is no escaping this central loop. 20 times a second each of these functions are called in order top to bottom. Every time – no exception the sunstars are moved. At the end of each move, in the moveSunstars() function, there are the following 2 lines of code:
posx = constrain(posx, 150, width-150); //well away from edges
posy = constrain(posy, 150, height-150);
This is the position of each sunstar so you can see it is constrained within a box. You can see the box in red here. So it is impossible to wander off the screen.
2) Has it become too small to see ?
Each sunstar has a size which depends on how much plankton it has eaten. The more plankton it eats the bigger it gets, if it doesn’t find any to eat it shrinks. This is a way we add variety to the artwork because we can’t possibly tell how much plankton it will get hold of and munch. The amount of plankton depends on what the webcam sees which depends on what is going on in the space. If someone enters the gallery with a bright red coat it may be that there is more red plankton available so if the red sunstar can find them it will get big.
The sunstars are drawn with these two lines of code :
sg.scale(sunstarscaler);
sg.scale(munchscale);
There are 2 scalers which can both vary. The scaler is just a number. If it equals 2 then the sunstar will be twice as big. sunstarscaler is applied to all three sunstars – it is the way we can balance the size of the sunstars for different situations. In this case we have a line of code
float sunstarscaler = 0.4;
This never changes. munchscale however does change depending on how much plankton food the sunstar has eaten. Each sunstar eats different coloured plankton. Red eats red plankton and so on. munchscale is only ever change in this line of code
munchscale = map(munch,0,1000,1.0,2.0);
This says that munchscale varies between 1.0 and 2.0. The more food it has eaten (which is stored in the munch variable) determines where munchscale is between 1.0 and 2.0. The more it has ate the bigger it gets. Although it looks as if munchscale is constrained between 1.0 and 2.0 this is not the case. What if the sunstar has eaten 2000 plankton. Is this even possible? We need to look ore closely at the munch variable which is controlled.
munch = clamp(munch-1,0,1000); //shrink over time
This lines does 2 things; firstly it stops munch ever being greater than 1000, (which in turn means that the munchscale can never be above 2.0), but secondly it reduces munch, until it becomes 0 (which sets the munchscale to 1.0). You could say this is a very crude homeostatic system. It continuously keeps the sunstar’s size within a set range.
So it is impossible to be too small.
3) Has it become too transparent to see ?
In Processing you can add an alpha channel to colours, which is how see through something is. If alpha is set to 0 then an object is invisible. However in this program we fix the alpha to 220 which means the sunstars have a touch of transparency but are always visible.
NaN
So we have removed the three most likely reasons for the missing sunstar. We will have to think outside of the box and gather more evidence. As well as position and scale there is one other variable which changes the sunstars appearance : posa.
This is the angle it is rotated. Sunstars are symmetrical and turn around in response to the space. Could it be that this angle has broken? What do we mean by broken? Sometimes Processing variables will be given unsuitable values. What would happen if the blue sunstar was given an unsuitable angle? Angles will accept any positive or negative numerical value, but what about infinity? What does it do if told to rotate by an infinite amount? And how could it possibly be told to do such a thing? Well infinity is surprisingly easy to produce in computer programs – just divide something by 0 and you have infinity, or try and find the square root of a negative number. If you run the following 2 line program
float s = sqrt(-1);
println(s);
You will get the print out “NaN“. This stands for Not a Number. It doesn’t crash the program but it does do funny things. What if we were trying to make the sunstar deliberately disappear – can we use NaN to do this? Yes we can. If we add the line
s3.posa = sqrt(-1);
Then the blue sunstar (s3) does indeed disappear, but the rest of the program runs on OK. This type of debugging is more active than usual. Typically we debug in a reactive way – we see which line crashed and try to fix that line. Processing is a friendly language and often tells us which line is causing the problem. But sometimes we have to deliberately try and make it crash.
So we have a possible theory – the blue sunstar is getting a NaN into posa. How could this happen?
Back to the code. We have a function which repels the sunstars from each other and also from Hector:
void repel(float rx, float ry, float repelForce){
float dx = rx – posx;
float dy = ry – posy;
float d = sqrt(dx*dx + dy*dy);
if(d < 150){
velx -= (dx/d) * repelForce;
vely -= (dy/d) * repelForce;
vela += (dx – dy) * 0.0002;
if(randomint(0,5)==0){ playSound(randomint(0,6),0.2); }
}
}
Here we can see there is a division by d. A likely culprit! It doesn’t affect posa but does affect velx and vely which are used to move the sunstars. If they get NaN then there’s going to be trouble. What if d is zero? This would be true if the sunstars were right on top of each other, or if Hector was right on top of them. The sunstars repel each other pretty well – they have lots of tentacles providing pushing force, but Hector is different – it is just using collision detection on one point on its head. Hector tends to push the sunstars around and out of the way – but it is imaginable that Hector could get boxed in by the sunstars in which case it might run over one of them. And then there is a small chance that it moves exactly on top of them.
Lets see if we can simulate this. In the code just before calling the repel function we will set the blue sunstar’s position to be the same as Hector’s head (which is coded as bellyx[0],bellyy[0]).
s3.posx = bellyx[0];
s3.posy = bellyy[0];
s3.repel(bellyx[0], bellyy[0], hectorRepelForce);
Yes it worked – the blue sunstar is missing.
We have solved the case of the missing sunstar!
Now we have to fix it
An easy way to do this is to make sure that d is never zero. We can add a tiny amount to the sqrt line
float d = sqrt(dx*dx + dy*dy) + 0.01;
This tiny amount will only change the position by a fraction of a pixel which is negligible in terms of what we see, but importantly it will stop the NaN error.
A few further observations. Why is it possible that Hector and the blue sunstar can be in exactly the same position? This is because of the constraints we put on the sunstars position – the keeping them in that box. If there is a pile up in the top right hand corner then the blue sunstar will be positioned at [width-150,150] because of these lines which we dismissed earlier
posx = constrain(posx, 150, width-150); //well away from edges
posy = constrain(posy, 150, height-150);
Hector’s position is also constrained – it too can sometimes be at [width-150,150].
So the moral of the story turns out to be this: be careful when putting ‘living’ things in a box, they will tend to get stuck in the corners.
Epilog
After a week of finding the blue sunstar had disappeared overnight, and wondering why – we eventually found one morning the green sunstar had disappeared. This made a welcome change, but was not particularly surprising because all the sunstars use the same code. We theorise that the reason the blue sunstar disappears more often is that it is moved and drawn third. Perhaps it gets boxed into a corner more often because it moves last. But the exact details of why that is, now don’t matter.
We have solved the case.
Genetic Moo May 2021
Hector and the Sunstars is on display in Microworld Kristiansand, at the Sorlandets Kunstmuseum in Norway until Sept 25th 2021. We are also developing a virtual LIKELIKE space as an art residency which is based on this case, open from June the 9th. https://www.skmu.no/en/utstillinger/microworld-kristiansand/
