In seiner Funktionalität auf die Lehre in gestalterischen Studiengängen zugeschnitten... Schnittstelle für die moderne Lehre
In seiner Funktionalität auf die Lehre in gestalterischen Studiengängen zugeschnitten... Schnittstelle für die moderne Lehre
In this project course of two weeks time I tried to harmonize overly dramtic climate images by using neural networks and the help the US administration under Donald Trump.
From the input talks about the matter of comparing images I was interested the most in the quantitative method of finding similarities in groups of pixels within an image via Convolutional Neural Networks (CNN). Especially interesting I found the fact that the actual computer vision / mashine learning activity stays mostly concealed to the user, due to the gigantic training set it is based upon, leading to the so called term „black box“.
This of later on made me want to at least get a little closer to understanding what's going on by slightly changing inputs and comparing the different outputs.
Slide by Paul Heinicker
From the 30 image-excerpt of the climate image dataset we were given I first tried to find a way to process them algorithmically in different ways and different tools. However, since they were of varying dimensions, different file-sizes and even completely diffent contents (photos, diagrams, screenshots etc.) I had a bit of a rough start. Also the xml tags seemed rather unhelpful to me. Nevertheless I liked the idea of textually tagging the image data, which I came back to later in the process. With no outcome so far, I took a step back and looked at the images from a more „innocent“, „human“ perspective and came to find that they all had one thing in common: they were all pretty dramatic.
While it might be a phenomenon of today's time that -- within digital media in particular -- most imagery we daily perceive is competing for our attention and therefore using exaggeration, signal colours and so on to gain our attention. For me this observation also plays a role with regard to the amount of drama we perceive.
As a starting point I dealt a little more with the origin and use of the term „drama“ which helped me better understand the means used in modern imagery to generate drama. However, processing images for drama with comon digital tools was more difficult than one might think. While a high contrast, the use of signal colours and certain xml-keywords might suggest an image to be of high dramatic value, I decided to manually preselect which images actually show drama and which don't. This worked best here, since in this case the actual drama often times was „hidden“ in the context of the scenario, relating to the climate change phenomenon.
Not quite the idyllic nature scene one might expect looking at the keywords generated by Google Could Vision.
Since I moved from a scientific approach to a more artistic one using Goodle Cloud Vision the actual visual measuring of the drama on a particular image was just perfect, even though the outcome can not be described as perfectly accurate. Google Could Vision is a computer vision algorithm based on a CNN, allowing to generate non-biased keywords, based on what the CNN recognizes, i.e. „beliefes“ what is being shown on the image. Along with each of the keywords a probability („certainty“) percentage value is generated.
Now, having defined drama in the beginning, I again manually evaluated the generated keywords for dramatic or non-dramtic (in the context of climate change).
Above is an image depicting the oil well fire rage in Kuwait from 1991
Rex Tillerson. US Secretary of State (US-Außenminister) under Donald Trump and former CEO of ExxonMobil, the world's largest oil and gas company.
To counteract the dramatization of climate images I decided to find a way of dedramtizing them. However reflecting what I was about to do, I was reminded of Donald Trump and his many statements neglecting climate change and refusing scientific proof. Now, since I of course do not want to support this propaganda activity, I decided to also move away from scientific proof and „fight“ against climate change denial using the denialers' own means and methods of switching to an alternate reality where common sense does not apply. On top of their tools, I used those people themselves.
For that I used the Deep Dream Generator, a „reversed“ CNN with the ability to recognize patterns in an image an have them ephasized or style-transfered to another image. Recommended read from the Google-Research blog: [Inceptionism: Going Deeper into Neural Networks](https://research.googleblog.com/2015/06/inceptionism-going-deeper-into-neural.html „ Inceptionism: Going Deeper into Neural Networks “)
Now, regarding my process: First of I choose an image depicting a cause or direct effect of climate change. Next, I had it run through Goodle Cloud Vision to count the amount of dramatic keywords generated. At the same time I made Google's Deep Dream algorithm „dream“ about a certain politian from Donald Trump's administration with the climate image as basis. The new, dreamerly image-output I then recoursivly ran through Google Cloud Vision and again went deeper into the dream.
Interestingly, the Deep Dream Generator as a „reversed CNN“ gave me an output which I ran through Google Could Vision as a „linear CNN“, basically making the two CNN's clash into each. Later on this reminded me of having an early tranlator engine translate a sentence from one language to another a few times until the outcome is non-understandable for the engine itself.
The image of the Kuwait oil well fire rage after dreaming of Rex Tillerson five times.
Interestingly all the apocalypticness is gone and the dystopic scenery transformed into a „fun“ „recreational“ world in an instant. Thanks Rex Tillerson.
Ecosystem, Drought, Shrubland, Steppe
Vice President Mike Pence. „Climate Change is just an issue for the left.“
The drought image after deep dreaming about Mike Pence five times. Art, Fun, Spring, Recreation
Wildfire, Smoke, Fire, Natural Desaster, Desaster
US president Donald Trump. „The concept of global warming was created by the Chinese [...]“
The wildfire image after deep dreaming about Donald Trump five times. Fun, Grass, Recreation, Child, Tree